Voiculescu's notion of asymptotic free independence is known for a large class of random matrices including independent unitary invariant matrices. This notion is extended for independent random matrices invariant in law by conjugation by permutation matrices. This fact leads naturally to an extension of free probability, formalized under the notions of traffic probability. We first establish this construction for random matrices. We define the traffic distribution of random matrices, which is richer than the *-distribution of free probability. The knowledge of the individual traffic distributions of independent permutation invariant families of matrices is sufficient to compute the limiting distribution of the join family. Under a factorization assumption, we call traffic independence the asymptotic rule that plays the role of independence with respect to traffic distributions. Wigner matrices, Haar unitary matrices and uniform permutation matrices converge in traffic distributions, a fact which yields new results on the limiting *-distributions of several matrices we can construct from them. Then we define the abstract traffic spaces as non commutative probability spaces with more structure. We prove that at an algebraic level, traffic independence in some sense unifies the three canonical notions of tensor, free and Boolean independence. A central limiting theorem is stated in this context, interpolating between the tensor, free and Boolean central limit theorems.