We generalize Milnor link invariants to all types of surface-links in 4-space (possibly with boundary) and more generally to all codimension 2 embeddings. This is achieved by using the notion of cut-diagram, which is a higher dimensional generalization of Gauss diagrams, associated to codimension 2 embeddings in Euclidian spaces. We define a notion of group for cut-diagrams, which generalizes the fundamental group of the complement, and we extract Milnor-type invariants from the successive nilpotent quotients of this group. We show that the latter are invariant under an equivalence relation called cut-concordance, which encompasses the topological notion of concordance. We give several concrete applications of the resulting Milnor concordance invariants for surface-links, comparing their relative strength with previously known concordance invariants, and providing realization results. We also obtain several classification results up to link-homotopy, as well as a criterion for a surface-link to be ribbon. In connection with the k-slice conjecture proved by Igusa and Orr, we show that a classical link has vanishing Milnor invariants of length $\leq 2k$ if and only if it bounds disjoint surfaces in the $4$-ball having vanishing Milnor invariants of length $\leq k$. Finally, the theory of cut-diagrams is further investigated, heading towards a combinatorial approach to the study of surfaces in 4-space.