Given polynomials $g$ and $f_1,\dots,f_p$, all in $\Bbbk[x_1,\dots,x_n]$ for some field $\Bbbk$, we consider the problem of computing the critical points of the restriction of $g$ to the variety defined by $f_1=\cdots=f_p=0$. These are defined by the simultaneous vanishing of the $f_i$'s and all maximal minors of the Jacobian matrix associated to $(g,f_1, \ldots, f_p)$. We use the Eagon-Northcott complex associated to the ideal generated by these maximal minors to gain insight into the syzygy module of the system defining these critical points. We devise new $F_5$-type criteria to predict and avoid more reductions to zero when computing a Gr\"obner basis for the defining system of this critical locus. We give a bound for the arithmetic complexity of this enhanced $F_5$ algorithm and compare it to the best previously known bound for computing critical points using Gr\"obner bases.