Reconstruction of the circumstellar environment and direct detection of exoplanets in the vicinity of nearby stars is challenging due to the very high contrast between the host star and the sought objects. In addition to the use of an extreme adaptive optics and a coronagraph, dedicated processing methods combining images recorded with the pupil tracking mode of the telescope are required to eliminate efficiently the nuisance component (speckles + noise) corrupting the signals of interest. Concerning the reconstruction of the circumstellar environment, we proposed in previous works the REXPACO algorithm tailored for angular differential imaging (ADI). It estimates jointly the sought objects and the statistics (mean and covariance matrix) of the nuisance following an inverse problem approach. It demonstrates state-of-the-art performance with ADI, but room for improvements remains, especially for reconstructing fine structures near the star and disks being partly rotation-invariant (which prevents their unmixing without additional diversity). We will present our most recent developments in that direction (Flasseur+ in prep). In particular, the new method takes benefit of the spectral diversity of the data to improve (drastically) the reconstruction fidelity. Contrary to the most advanced counterpart algorithms dedicated to point-source extraction, spatial and spectral correlations are modeled jointly and directly from the data. The algorithm yields a deconvolved estimate of the objects in an unsupervised fashion. In terms of results, we will first exemplify from simulated disks that a joint modeling of the spectral diversity (even at a low resolution and with a limited bandwidth) is a “game-changer” to improve the reconstruction of disks, without Reference Differential Imaging. We will then show from real data from SPHERE-IFS that the proposed temporo-spatio-spectral model captures accurately the nuisance to eliminate it efficiently. Finally, we will show several reconstructions of emblematic disks with unprecedent quality. Fully exploiting the high spectral diversity of the future ELT instruments will be a challenge to explore the inner environment of nearby solar-type stars. Concerning exoplanet detection, most of the current processing techniques proceed in two sequential steps: (i) a suppression of the stellar continuum by filtering, (ii) a spectral cross-correlation with a model of the sought planets. Based on HARMONI simulations, we will present detection sensitivity of PACO ASDI modeling statistically the nuisance component and making any assumption on the exoplanet’s spectrum. We will also discuss preliminary results obtained on these simulations with PACOME (Dallant+ A&A 23) that efficiently combines observations of the same star while accounting for the Keplerian motion of the sought exoplanets within an end-to-end statistical detection formalism.