The paper deals with the controllability of finite-dimensional linear difference delay equations, i.e., dynamics for which the state at a given time $t$ is obtained as a linear combination of the control evaluated at time $t$ and of the state evaluated at finitely many previous instants of time $t-\Lambda_1,\dots,t-\Lambda_N$. Based on the realization theory developed by Y.~Yamamoto for general infinite-dimensional dynamical systems, we obtain necessary and sufficient conditions, expressed in the frequency domain, for the approximate controllability in finite time in $L^q$ spaces, $q \in [1, +\infty)$. We also provide a necessary condition for $L^1$ exact controllability, which can be seen as the closure of the $L^1$ approximate controllability criterion. Furthermore, we provide an explicit upper bound on the minimal times of approximate and exact controllability, given by $d\max\{\Lambda_1,\dots,\Lambda_N\}$, where $d$ is the dimension of the state space.