Strider: A Hybrid Adaptive Distributed RDF Stream Processing Engine - Archive ouverte HAL Access content directly
Conference Papers Year :

Strider: A Hybrid Adaptive Distributed RDF Stream Processing Engine

Abstract

Real-time processing of data streams emanating from sensors is becoming a common task in Internet of Things scenarios. The key implementation goal consists in efficiently handling massive incoming data streams and supporting advanced data analytics services like anomaly detection. In an ongoing , industrial project, a 24/7 available stream processing engine usually faces dynamically changing data and workload characteristics. These changes impact the engine's performance and reliability. We propose Strider, a hybrid adaptive distributed RDF Stream Processing engine that optimizes logical query plan according to the state of data streams. Strider has been designed to guarantee important industrial properties such as scalability, high availability, fault tolerance, high throughput and acceptable latency. These guarantees are obtained by designing the engine's architecture with state-of-the-art Apache components such as Spark and Kafka. We highlight the efficiency (e.g., on a single machine machine, up to 60x gain on throughput compared to state-of-the-art systems, a throughput of 3.1 million triples/second on a 9 machines cluster, a major breakthrough in this system's category) of Strider on real-world and synthetic data sets.
Fichier principal
Vignette du fichier
iswc-2017-camera.pdf (754.18 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01740499 , version 1 (22-03-2018)

Identifiers

  • HAL Id : hal-01740499 , version 1

Cite

Xiangnan Ren, Olivier Curé. Strider: A Hybrid Adaptive Distributed RDF Stream Processing Engine. ISWC, Oct 2017, Vienna, Austria. ⟨hal-01740499⟩
54 View
101 Download

Share

Gmail Facebook Twitter LinkedIn More