Let $A$ be a matrix of size $M\times N$ (a dictionary) and let $\|.\|$ be a norm on $\RR^N$. For any $d\in\RR^N$ we consider the sparsest vector (i.e. the one with the smallest number of non zero entries) $u\in\RR^M$ such that $\|Au-d\| \leq \tau$, for a parameter $\tau>0$. We say that $u$ is a $K$-sparse solution if it has less than $K\in\NN$ non zero entries. In this paper, we give a precise geometrical description of the data sets yielding a $K$-sparse solution. We parameterize and measure these sets. More precisely, we measure their intersection with a ball defined by any given norm $\delta$ and a radius $\theta$. These measures are expressed in terms of the constituents of the optimization problem. This is the core of a new methodology, called Average Performance in Approximation (APA), inaugurated in this work. By way of application, we give the probability of obtaining a $K$-sparse solution, when $d$ is uniformly distributed in the $\delta$-ball of radius $\theta$. Analyzing the obtained formulas reveals what are the most important features of the dictionary and the norm defining the data fidelity, to obtain sparse solutions. This important question is largely discussed. We also provide an example when both $\|.\|$ and $\delta$ are the Euclidian norm. Some among the wide-ranging perspectives raised by the new APA methodology are described as well.