Cell-penetrating peptides are short basic peptide sequences that might display amphipathic properties. These positively charged peptides internalize into all cell types, albeit with different efficiency. Cell-penetrating peptides use all routes of pinocytosis to internalize, in addition to direct membrane translocation that requires interaction with lipid membrane domains. These differences in internalization efficiency according to the peptide sequence and cell type suggest that the cell-penetrating peptides interact with different molecular partners at the cell surface. This review will first report on data that describe the molecular interaction of the most popular cell-penetrating peptides (penetratin, Tat and oligoarginine) with carbohydrates and lipids. The second part of the review will be dedicated to cell studies that have reported how cell surface composition influences cell internalization. Discussion will focus on the gap between in vitro and in cellulo studies, and more specifically to which extent the interaction with molecules found in membranes reflect the internalization efficiency of the peptides.