Motivated by the discrete dipole approximation (DDA) for the scattering of electromagnetic waves by a dielectric obstacle that can be considered as a simple discretization of a Lippmann-Schwinger style volume integral equation for time-harmonic Maxwell equations, we analyze an analogous discretization of convolution operators with strongly singular kernels. For a class of kernel functions that includes the finite Hilbert transformation in 1D and the principal part of the Maxwell volume integral operator used for DDA in dimensions 2 and 3, we show that the method, which does not fit into known frameworks of projection methods, can nevertheless be considered as a finite section method for an infinite block Toeplitz matrix. The symbol of this matrix is given by a Fourier series that does not converge absolutely. We use Ewald's method to obtain an exponentially fast convergent series representation of this symbol and show that it is a bounded function, thereby allowing to describe the spectrum and the numerical range of the matrix. It turns out that this numerical range includes the numerical range of the integral operator, but that it is in some cases strictly larger. In these cases the discretization method does not provide a spectrally correct approximation, and while it is stable for a large range of the spectral parameter λ, there are values of λ for which the singular integral equation is well posed, but the discretization method is unstable.