In this paper, we provide a comparative performance evaluation study of Deep-Q-Network (DQN) and Dueling DQN in the context where we address network slicing in 5G networks and beyond through solving SFC orchestration problem leveraging Software Defined Networking (SDN) and Network Function Virtualization (NFV) capabilities and using Deep Reinforcement Learning (DRL) approach aiming to maximize Quality of Experience (QoE) while meeting Quality of Service (QoS) requirements. We intend through such investigation to highlight how the DRL agent behaves along the training phase while orchestrating each network slice (or Service Function Chain (SFC)) on a Physical Substrate Network (PSN) in terms of reaching a suitable compromise between performance and convergence. The network slice orchestration is achieved by deploying the corresponding SFC request composed of a set of ordered Virtualized Network Functions (VNFs) linked through virtual links that packets need to traverse within a network slice to achieve specific service requirements. We show throughout numerical experiments how Dueling DQN outperforms DQN in this scenario and how we can compare its performances with those of reference algorithms referred to as violent and random. The investigated performance evaluation study is based on two performance metrics concerning the QoE score and the rejection ratio (RR). Furthermore we assess the quality of learning for the two metrics by testing the ability of the DRL agent to reach a near-optimal solution, along the last 100 runs of the learning phase, quantified by a pre-defined QoE threshold score.