This article proposes a new approach to train physics-augmented neural networks with observable data to represent mechanical constitutive laws. To train the neural network, the proposed method does not rely on strain-stress or strain-free energy pairs but needs only partial strain or displacement measurements inside the structure, as well as boundary conditions. The neural network is trained thanks to an unsupervised procedure in which the modified Constitutive Relation Error (mCRE) is minimized. The mCRE functional provides a rich physical sense as the Constitutive Relation Error (CRE) part can be interpreted as a model error continuously defined over the structure, and can be used as a prediction quality in the inference phase. This article also extends previous works on the mCRE by introducing a new minimization procedure in the case of nonlinear state laws. As typical structural health monitoring applications may require that the neural networks should be trained online, an important focus is thus given on automatic and adaptive tuning of sensitive hyperparameters (learning rate, weighting between losses and number of epochs). It is shown that when the training database is rich enough with respect to the loading cases, the proposed method achieves remarkable performances regarding the quality of the learned model, noise robustness, and low sensitivity to user-defined hyperparameters. The method is evaluated on two test cases: a non-quadratic potential in the small strain regime with synthetic optic fiber measurements, and a Mooney-Rivlin model in the hyperelastic case with synthetic digital image correlation observations.