Let $H$ be a Krull monoid with class group $G$ such that every class contains a prime divisor. Then every nonunit $a \in H$ can be written as a finite product of irreducible elements. If $a=u_1 \cdot \ldots \cdot u_k$, with irreducibles $u_1, \ldots u_k \in H$, then $k$ is called the length of the factorization and the set $\mathsf L (a)$ of all possible $k$ is called the set of lengths of $a$. It is well-known that the system $\mathcal L (H) = \{\mathsf L (a) \mid a \in H \}$ depends only on the class group $G$. In the present paper we study the inverse question asking whether or not the system $\mathcal L (H)$ is characteristic for the class group. Consider a further Krull monoid $H'$ with class group $G'$ such that every class contains a prime divisor and suppose that $\mathcal L (H) = \mathcal L (H')$. We show that, if one of the groups $G$ and $G'$ is finite and has rank at most two, then $G$ and $G'$ are isomorphic (apart from two well-known pairings).