We here propose simultaneously fast and in-place algorithms for problems where the result of some formula has to be linearly accumulated. In other words, some of the output variables are also input variables, with a linear dependency between these both read and write accesses. Fundamental examples include the accumulated multiplication of polynomials or matrices, C += AB, that is where there is a side-effect on the output C which receives in fine C + AB. The difficulty is to combine in-place computations, here the output variables are not even directly available to store intermediate values, with fast algorithms, that usually come at the expense of (potentially large) extra temporary space. We first propose the automatic design of fast and in-place accumulating algorithms for any bilinear formulae (and thus for polynomial and matrix multiplication) and then extend it to any linear accumulation of a collection of functions. This generation requires a relaxation of the in-place model to any algorithm that is allowed to modify its inputs provided that those are restored to their initial state afterwards. This is a less powerful model than when the input is purely read-only, but it turns out to be crucial in our case, especially when we have accumulation operations. From this generic method we are for instance able to derive unprecedented in-place accumulating algorithms for fast polynomial multiplications as well as for Strassen-like matrix multiplications; the latter coming with an associated optimality result.