We consider integer linear programs whose solutions are binary matrices and whose (sub-)symmetry groups are symmetric groups acting on (sub-)columns. Such structured sub-symmetry groups arise in important classes of combinatorial problems, e.g. graph coloring or unit commitment. For a priori known (sub-)symmetries, we propose a framework to build (sub-)symmetry-breaking inequalities for such problems, by introducing one additional variable per considered sub-symmetry group. The derived inequalities are full-symmetry-breaking and in polynomial number w.r.t. the number of sub-symmetry groups considered. The proposed framework is applied to derive such inequalities when the symmetry group is the symmetric group acting on the columns. It is also applied to derive sub-symmetry-breaking inequalities for the graph coloring problem. Experimental results give insight into how to select the right inequality subset in order to efficiently break sub-symmetries. Finally, the framework is applied to derive (sub-)symmetry breaking inequalities for Min-up/min-down Unit Commitment Problem with or without ramp constraints. We show the effectiveness of the approach by presenting an experimental comparison with state-of-the-art symmetry-breaking formulations.