This article proposes a new approach to train physics-augmented neural net-works with observable data to represent mechanical constitutive laws. To trainthe neural network and learn thermodynamics potentials, the proposed methoddoes not rely on strain-stress or strain-free energy pairs but needs only partialstrain or displacement measurements inside the structure. The neural networkis trained thanks to an unsupervised procedure in which the modified con-stitutive relation error (mCRE) is minimized. The mCRE functional providesa bias-aware data assimilation framework with a rich physical sense as theconstitutive relation error (CRE) part can be interpreted as a modeling errorcontinuously defined over the structure, and can be used as a prediction qualityin the inference phase. This article also extends previous works on the mCREby introducing a new minimization procedure in the case of nonlinear statelaws. As typical structural health monitoring applications may require that theneural networks should be trained online, an important focus is thus madeon automatic and adaptive tuning of sensitive hyperparameters (learning rate,weighting between losses, number of epochs and initialization). It is shown thatwhen the training database is rich enough with respect to the loading cases,the proposed method achieves remarkable performance regarding the qual-ity of the learned model, noise robustness, and low sensitivity to user-definedhyperparameters. The method is evaluated on two test cases: a non-quadraticpotential in the small strain regime with synthetic optic fiber measurements, and a Mooney-Rivlin model in the hyperelastic case with synthetic digital image correlation observations.