We consider physical network coding on a two-way relay channel. Two source nodes send their packets simultaneously over time-varying MIMO channels to a relay node. Assuming that the relay employs the joint decode-and-forward strategy, the relay must be able to decode the individual source packets. This task requires a robust estimation of the source-to-relay channels. We design a message passing algorithm based on factor-graphs to solve this problem efficiently.