The Brownian motion of a particle hotter than its environment is an iconic out-of-equilibrium system. Its study provides valuable insights into nanoscale thermal effects. Notably, it supplies an excellent diagnosis of thermal effects in optically levitated particles, a promising platform for force sensing and quantum physics tests. Thus, understanding the relevant parameters in this effect is critical. In this context, we test the role of particles' shape and material, using optically levitated nanodiamonds hosting NV centers to measure the particles' internal temperature and center-of-mass dynamics. We present a model to assess the nanodiamond internal temperature from its dynamics, adaptable to other particles. We also demonstrate that other mechanisms affect the nanodiamond dynamics and its stability in the trap. Finally, our work, by showing levitating nanodiamonds as an excellent tool for studying nano-thermal effects, opens prospects for increasing the trapping stability of optically levitated particles.