Cancer is a complex disease whose outcome depends largely on the cross-talk between the tumor and its microenvironment. Here, we review the evolution of the field of tumor immunology and the advances, in lockstep, of our understanding of cancer as a disease. We discuss the involvement of different immune cells at distinct stages of tumor progression and how immune contexture determinants shaping tumor development are being exploited therapeutically. Current clinical stratification schemes focus on the tumor histopathology and the molecular characteristics of the tumor cell. We argue for the importance of revising these stratification systems to include immune parameters so as to address the immediate need for improved prognostic and/or predictive information to guide clinical decisions.