We develop a functional analytic approach to the study of the Kramers and kinetic Fokker-Planck equations which parallels the classical $H^1$ theory of uniformly elliptic equations. In particular, we identify a function space analogous to $H^1$ and develop a well-posedness theory for weak solutions of the Dirichlet problem in this space. In the case of a conservative force, we identify the weak solution as the minimizer of a uniformly convex functional. We prove new functional inequalities of Poincar\'e and H\"ormander type and combine them with basic energy estimates (analogous to the Caccioppoli inequality) in an iteration procedure to obtain the $C^\infty$ regularity of weak solutions. We also use the Poincar\'e-type inequality to give an elementary proof of the exponential convergence to equilibrium for solutions of the kinetic Fokker-Planck equation which mirrors the classic dissipative estimate for the heat equation.