Apache Spark and Apache Kafka at the rescue of distributed RDF Stream Processing engines - Archive ouverte HAL Access content directly
Conference Papers Year :

Apache Spark and Apache Kafka at the rescue of distributed RDF Stream Processing engines

Abstract

Due to the growing need to timely process and derive valuable information and knowledge from data produced in the Semantic Web, RDF stream processing (RSP) has emerged as an important research domain. In this paper, we describe the design of an RSP engine that is built upon state of the art Big Data frameworks, namely Apache Kafka and Apache Spark. Together, they support the implementation of a production-ready RSP engine that guarantees scalability, fault-tolerance, high availability, low latency and high throughput. Moreover, we highlight that the Spark framework considerably eases the implementation of complex applications requiring libraries as diverse as machine learning, graph processing, query processing and stream processing.
Fichier principal
Vignette du fichier
paper43.pdf (564.57 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01740515 , version 1 (22-03-2018)

Identifiers

  • HAL Id : hal-01740515 , version 1

Cite

Xiangnan Ren, Olivier Curé, Houda Khrouf, Zakia Kazi-Aoul, Yousra Chabchoub. Apache Spark and Apache Kafka at the rescue of distributed RDF Stream Processing engines. 15th International Semantic Web Conference ISWC 2016, 2016, Kobe, Japan. ⟨hal-01740515⟩
243 View
219 Download

Share

Gmail Facebook Twitter LinkedIn More