Apache Spark and Apache Kafka at the rescue of distributed RDF Stream Processing engines - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Apache Spark and Apache Kafka at the rescue of distributed RDF Stream Processing engines

Résumé

Due to the growing need to timely process and derive valuable information and knowledge from data produced in the Semantic Web, RDF stream processing (RSP) has emerged as an important research domain. In this paper, we describe the design of an RSP engine that is built upon state of the art Big Data frameworks, namely Apache Kafka and Apache Spark. Together, they support the implementation of a production-ready RSP engine that guarantees scalability, fault-tolerance, high availability, low latency and high throughput. Moreover, we highlight that the Spark framework considerably eases the implementation of complex applications requiring libraries as diverse as machine learning, graph processing, query processing and stream processing.
Fichier principal
Vignette du fichier
paper43.pdf (564.57 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01740515 , version 1 (22-03-2018)

Identifiants

  • HAL Id : hal-01740515 , version 1

Citer

Xiangnan Ren, Olivier Curé, Houda Khrouf, Zakia Kazi-Aoul, Yousra Chabchoub. Apache Spark and Apache Kafka at the rescue of distributed RDF Stream Processing engines. 15th International Semantic Web Conference ISWC 2016, 2016, Kobe, Japan. ⟨hal-01740515⟩
276 Consultations
234 Téléchargements

Partager

Gmail Facebook X LinkedIn More