This work aims at improving the workers’ wellbeing by providing them with skill-based personalized assistance in the context of physical Human-Robot Collaboration (pHRC). Past researches usually assume that each person will respond equally to assistance and therefore do not update their assistance policy online. However, since the focus of our work is on humans in pHRC, intra- and inter-individual variations are to be considered. Thus, we propose a new hybrid approach that combines reinforcement learning and a symbolic approach based on an ontology to guide humans towards skills improvement using solely internal robot data without any additional sensor. The advantage of this combination is to handle constant adaptation of users needs while reducing the learning process. This reduction is insured by the use of a knowledge base to choose the most suitable assistance, as well as a pre-training of the learning algorithm in simulation. In addition, including human feedback in the learning algorithm speeds up learning and ensures that unwanted assistance is not provided to the operator. Finally, since acquiring a skill involves both theory and practice, we offer two types of assistance, textual advice, along with a change of the robot behavior. We have demonstrated through simulations and a real-world experimentation that our approach leads the learner more quickly to the mastery of skills and thus eases the on-the- job training.