We study the radiative heat transfer and the Casimir-Lifshitz force occurring between two bodies in a system out of thermal equilibrium. We consider bodies of arbitrary shape and dielectric properties, held at two different temperatures and immersed in environmental radiation at a third different temperature. We derive explicit closed-form analytic expressions for the correlations of the electromagnetic field and for the heat transfer and Casimir-Lifshitz force in terms of the bodies' scattering matrices. We then consider some particular cases which we investigate in detail: the atom-surface and the slab-slab configurations.