We present in this paper a generic and parameter-free algorithm to efficiently build a wide variety of optical components, such as mirrors or lenses, that satisfy some light energy constraints. In all of our problems, one is given a collimated or point light source and a desired illumination after reflection or refraction and the goal is to design the geometry of a mirror or lens which transports exactly the light emitted by the source onto the target. We first propose a general framework and show that eight different optical component design problems amount to solving a Light Energy Conservation equation that involves the computation of Visibility diagrams. We show that these diagrams all have the same structure and can be obtained by intersecting a 3D Power Diagram with a planar or spherical domain. This allows us to propose an efficient and fully generic algorithm capable to solve the eight optical component design problems. Our solutions can satisfy design constraints such as convexity or concavity and are always graphs over the plane or the sphere. We show the effectiveness of our algorithm on numerous simulated examples.