We classify the twisted tensor products of a finite set algebra with a two elements set algebra using colored quivers obtained through considerations analogous to Ore extensions. This provides also a classification of entwining structures between a finite set algebra and the grouplike coalgebra on two elements. The resulting $2$-nilpotent algebras have particular features with respect to Hochschild (co)homology and cyclic homology.