This paper investigates an innovative building material based on rapeseed concrete. This material is a non-load-bearing insulating concrete, which is intended for use in the construction of wood-frame walls thanks to its thermophysical properties. It is composed of particles of rapeseed straw, lime, and cement. First, this work proposes to characterize rapeseed straw aggregates according to the place of cultivation, the year of harvest, and the size of the straw strands. For this purpose, straws of three different origins and different years of harvest were chosen. Aggregate sizes of 10 mm and 20 mm in length were selected. In a second step, this study focuses on the effect of the type of rapeseed straw aggregates on the mechanical resistance and thermal conductivity of bio-based concrete. The results obtained showed that the influence of the different parameters on the compressive strength was stronger than that on the thermal conductivity. On the one hand, rapeseed concrete made with 10 mm straw exhibited a lower thermal conductivity, averaging at 0.073 W.m−1.K−1. On the other hand, concrete manufactured with the 20 mm size aggregates demonstrated a higher mechanical strength, which remained relatively low and closer to 0.22 MPa. Finally, 20 mm-long aggregates offered the best compromise between mechanical and thermal resistance.