While sharing multimedia content on Online Social Networks (OSNs) has many benefits, exposing other people without obtaining permission could cause Multiparty Privacy Conflicts (MPCs). Earlier studies developed technical solutions and dissuasive approaches to address MPCs. However, none of these studies involved OSN users who have experienced MPCs, in the design process, possibly overlooking the valuable experiences these individuals might have accrued. To fill this gap, we recruited participants specifically from this population of users, and involved them in participatory design sessions aiming at ideating solutions to reduce the incidence of MPCs. To frame the activities of our participants, we borrowed terminology and concepts from a well known framework used in the justice systems. Over the course of several design sessions, our participants designed 10 solutions to mitigate MPCs. The designed solutions leverage different mechanisms, including preventing MPCs from happening, dissuading users from sharing, mending the harm, and educating users about the community standards. We discuss the open design and research opportunities suggested by the designed solutions and we contribute an ideal workflow that synthesizes the best of each solution. This study contributes to the innovation of privacy-enhancing technologies to limit the incidences of MPCs in OSNs.