Episodic memory serves as a store of individual experiences and allows for flexible adaptation to environment volatility and goal changes. The selection of episodic memories to recall is often considered to be driven by external sensory cues. Experimental studies suggest that this process is also influenced by internal cues, and that projections from the medial prefrontal cortex to the hippocampus play a role in this contextual modulation. In order to make sense of the biological configuration of prefrontal-to-hippocampus connectivity, we investigate the effectiveness of modulating various layers of a hippocampus-inspired neural network in a contextual memory task. Our results reveal that providing context information to the most downstream regions (i.e. last layers) of the model leads to better performance. In addition, the best average performance is obtained when contextual connections target the regions corresponding to the biological subfields that receive information from the prefrontal cortex, which provides a normative account of the biological connectivity. We relate this work to the need for augmenting reinforcement learning with flexible episodic memory. We make the code available at https://github.com/HugoChateauLaurent/opportunistic-PFC.