Multi-core multi-node parallelization of the radio interferometric imaging pipeline DDFacet - Archive ouverte HAL
Conference Papers Year : 2022

Multi-core multi-node parallelization of the radio interferometric imaging pipeline DDFacet

Abstract

The next generation of radio telescopes, such as the Square Kilometer Array (SKA), will need to process an incredible amount of data in real-time. In addition, the sensitivity of SKA will require a new generation of calibration and imaging software to exploit its full potential. The wide-field directiondependent spectral deconvolution framework, called DDFacet, has been successfully used in several existing SKA pathfinders and precursors like MeerKAT and LOFAR. This imager allows a multi-core execution based on facets parallelization and a multinode execution based on observations parallelization. However, because of the amount of data to be computed, the data on a single observation will have to be distributed on several nodes. This paper proposes the first two-level parallelization of DDFacet in the case of a single observation. A multi-core parallelization based on facets and a multi-node parallelization based on frequency distribution grouped in Measurement Sets. We show that this multi-core multi-node parallelization has successfully reduced the total execution time by a factor of 5.7 on a LOFAR dataset.
Fichier principal
Vignette du fichier
SiPS_IEEE_2022_Monnier.pdf (1.01 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03729202 , version 1 (20-07-2022)

Identifiers

Cite

Nicolas Monnier, David Guibert, Cyril Tasse, Nicolas Gac, François F. Orieux, et al.. Multi-core multi-node parallelization of the radio interferometric imaging pipeline DDFacet. SiPS22-IEEE Workshop on Signal Processing Systems, Nov 2022, Rennes, France. ⟨10.1109/sips55645.2022.9919239⟩. ⟨hal-03729202⟩
178 View
182 Download

Altmetric

Share

More