Over several decades, many contributions have been made to understand the physics and to develop models of aggregates of metals and composites. Three books `Modelling small deformations of polycrystals' (eds. J. Gittus and J. Zarka) and `Modelling large deformations of solids' (eds. J. Gittus, J. Zarka and S. Nemat-Nasser) and `New approach of inelastic analysis of structures' (eds. J. Zarka et al.) gave already a general view of the state of the art. Nemat-Nasser, particularly in his impressive book `Micromechanics: overall properties of heterogeneous materials', improved and is still greatly improving our knowledge in this fundamental area. In this paper, which is dedicated to his 60th anniversary, we want to give again and for the last time, our own view of the problem based on (may be) not well known works and also some unpublished theses of some students which were done between 1964 and now. In the first part, we shall underline what the physical quantitative description of inelastic behavior of single crystals means for us by answering the following questions: (1) are the various glides on each crystal slip plane, as introduced in the multiple plastic potential theory, potential internal variables? (no); (2) can we build a real physical model for the crystal in correlation with the observations of the evolution of the crystalline defects, such as the dislocations, the vacancies, the precipitates ... and the thermally activated processes? (yes); (3) can we define the physical hardening parameters (yes) and is the latent hardening higher than the self-hardening? (no); (4) is it possible to reach a description which fits the experimental results? (may be!). Then, in the second part, assuming a given behavior for the crystals, for the global inelastic behavior of polycrystals, we shall try to answer the questions: (5) how many different orientations of crystals are necessary to reach the description of an initially isotropic polycrystal? (three well selected ones!); (6) are the simplified models, such as the self-consistent model, sufficient to represent the interactions between crystals? (yes); (7) are the actual numerical simulations based on rate-dependant or rate-independent crystal plasticity, useful for the industrial applications? (no, unhappily!). Finally, in the third part, we shall give our new general framework of `intelligent' modelling of aggregates where we take into account not only the local behaviors of the crystals but also their size, their shape and their relative distribution. In this framework, it is needed: (i) To build a data base i.e. to obtain some experimental, real or simulated, results where the experts identify all variables or descriptors which may relevant to the given problem. This is, at first, done with some primitive descriptors x which are usually in a limited number. Then, the data are transformed into intelligent descriptors XX in a larger number, using the existing knowledge and theories which are still always insufficient. The descriptors may be numbers, boolean, alphanumeric, name of files which gives access to data bases, or treatments of curves, signals and images. The results or conclusions may be classes (good, not good, ...) or numbers (Young modulus, cost, weight, life time, ...). Usually, the data base may contain roughly 30 to 150 examples with 10 to 1000 descriptors and 1 to 20 conclusions. (ii) To generate the rules with any automatic learning tool. The intelligent descriptors help these learning algorithms. Each conclusion is explained as function or set of rules of some among the input intelligent descriptors with a known reliability or accuracy. If this reliability is too low, it implies that either there is not enough data or there are bad, missing descriptors or the problem was not well described. (iii) To optimize at two levels (inverse problems): (1) Considering the intelligent descriptors as independent; it is possible to get the optimal solution satisfying the special required properties and allowing the discovery of new mechanisms; (2) considering the intelligent descriptors linked to primitive descriptors, it is possible to obtain the optimal solution which is technologically realizable. So, not only a practical optimal solution is obtained, but also the experts may learn the missing parts, may build models or theories based only on the retained intelligent descriptors and guided by the structures of the rules or relationships. We shall illustrate our framework by treating step by step the problem of the global elastic behavior of concrete with a cement matrix and soft or hard inclusions. This problem has been the subject of several papers, providing simplified models, simple bounds of the elastic moduli and sophisticated theories ... were produced. Our aim is to give the results to the engineers in such a practical way that they could: (1) estimate the elastic properties within a few percent of error for any concentration, any shape, or distribution of the inclusions; (2) select the preparation of the aggregate to reach any pre-assigned elastic properties (even at the lowest cost or weight).