We focus on the minimal time control problem for single-input control-affine systems $\dot{x}=X(x) + u_1 Y_1 (x)$ in $\R^{n}$ with fixed initial and final time conditions $x(0) = \hat{x}_0$, $x(t_f) = \hat{x}_1$, and where the scalar control $u_1$ satisfies the constraint $|u_1(\cdot)| \leq 1$. For these systems a concept of conjugate time $t_c$ has been defined in e.g. \cite{Agrachev_Stefani_Zezza_2002,MaurerOsmolovskii_2004,Noble_Schattler_2002} in the bang-bang case. Besides, theoretical and practical issues for conjugate time theory are well known in the smooth case (see e.g. \cite{Agrachev_Sachkov,Milyutin_Osmolovskii_1998}), and efficient implementation tools are available (see \cite{Bonnard_Caillau_Trelat}). The first conjugate time along an extremal is the time at which the extremal loses its local optimality. In this work, we use the asymptotic approach developed in \cite{Silva_Trelat} and investigate the convergence properties of conjugate times. More precisely, for $\varepsilon > 0$ small and arbitrary vector fields $Y_1, ..., Y_m$, we consider the minimal time problem for the control system $\dot{x}^\varepsilon = X(x^\varepsilon) + u_1^{\varepsilon} Y_1(x^\varepsilon)+ \varepsilon \sum_{i=2}^m u_i^{\varepsilon} Y_i(x^\varepsilon)$, under the constraint $\sum_{i=1}^m (u_i^\varepsilon)^2 \leq 1$, with the fixed boundary conditions $x^\varepsilon(0)= \hat{x}_0$, $x^\varepsilon(t_f) = \hat{x}_1$ of the initial problem. Under appropriate assumptions, the optimal controls of the latter regularized optimal control problem are smooth, and the computation of associated conjugate times $t_c^\varepsilon$ falls into the standard theory; our main result asserts the convergence, as $\varepsilon$ tends to 0, of $t_c^\varepsilon$ towards the conjugate time $t_c$ of the initial bang-bang optimal control problem, as well as the convergence of the associated extremals. As a byproduct, we obtain an efficient algorithmic way to compute conjugate times in the bang-bang case.