GUESS: monitorinG join qUery Execution in Serverless and Serverful spark - Archive ouverte HAL
Communication Dans Un Congrès Année : 2024

GUESS: monitorinG join qUery Execution in Serverless and Serverful spark

Résumé

This paper proposes a monitoring system called GUESS to compare the performance and energy consumption of join query processing on Spark in Serverless and Serverful environments. The system collects metrics on resource utilization, query execution times, and power usage through Prometheus, Grafana, Spark History Server, and Open- Manage Enterprise Power Manager. These metrics are visualized through an intuitive web dashboard to enable easy comparison between Serverless and Serverful Spark workloads. Experimental results using the TPC-H benchmark show that the Serverless environment consumes less energy than the Serverful environment due to on-demand resource allocation. However, the Serverful environment exhibits better query performance, especially for workloads with known resource requirements. GUESS provides insights into optimizing resource efficiency and query performance when deploying Spark analytic workloads.
Fichier principal
Vignette du fichier
demo_1329_paper.pdf (1.39 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04544429 , version 1 (12-04-2024)

Identifiants

  • HAL Id : hal-04544429 , version 1

Citer

An-Truong Tran Phan, Laurent d'Orazio, Thuong-Cang Phan, Le Gruenwald. GUESS: monitorinG join qUery Execution in Serverless and Serverful spark. International Conference on Database Systems for Advanced Applications (DASFAA), Jul 2024, Gifu, Japan. ⟨hal-04544429⟩
77 Consultations
115 Téléchargements

Partager

More