Mesoscale eddies significantly influence ocean circulation, nutrient distribution, and climate patterns globally.  A thorough reconstruction of the eddy field is therefore important, yet classical eddy detection algorithms based on sea level anomaly (SLA) suffer from the low coverage of the current altimetry network. In this work, we evaluate the efficacy of deep learning techniques in enhancing the oceanic eddy field reconstruction of an operational ocean forecasting system. We use two ocean models from an Observing System Simulation Experiments (OSSE): a free-run high-resolution ocean circulation model representing the ‘truth’ and a second one constrained by synthetic observations mimicking the altimetry network through assimilation techniques to approximate the state of the ’truth’ model.  We train a neural network model that takes sea surface temperature, sea surface height, and ocean surface currents inputs from the data-assimilation model to recover eddies identified in the ‘truth’ model, which are generated with py-eddy-tracker, a sea surface height-based eddy detection algorithm. Our investigation centers on a semantic segmentation problem using the U- Net architecture to classify pixels for a given map into non-eddy, cyclonic eddy, and anticyclonic eddy. Our study focuses on the Gulf Stream region, an area renowned for its dynamic oceanic conditions. We find a higher detection rate of eddies and reduced inter-class misclassification when compared to eddy fields reconstructed from the data-assimilated model using the traditional SLA-based algorithm.  Our data-driven method improves the detection of ‘true’ eddies from degraded data in an OSSE framework, and shows potential for application in operational analysis and forecasting systems.