We consider how to tell the time-ordering associated with measurement data from quantum experiments at two times and any number of qubits. We define an arrow of time inference problem. We consider conditions on the initial and final states that are symmetric or asymmetric under time reversal. We represent the spatiotemporal measurement data via the pseudo density matrix space-time state. There is a forward process which is CPTP and a reverse process which is obtained via a novel recovery map based on inverting unitary dilations. For asymmetric conditions, the protocol determines whether the data is consistent with the unitary dilation recovery map or the CPTP map. For symmetric conditions, the recovery map yields a valid unitary and the experiment may have taken place in either direction. We also discuss adapting the approach to the Leifer-Spekkens or Process matrix space-time states.