Tracking Consistency over Data Streams with InkStream
Résumé
Establishing robust frameworks to safeguard data consistency in streaming applications is a strategic imperative. Nevertheless, existing methods cannot deal with the infinite nature of streaming data. On the other hand, Stream Processing Engines, systems that serve as the infrastructure for executing continuous queries efficiently, were never leveraged for data consistency management. Indeed, handling data consistency by enforcing constraints over data streams may compromise the strict performance requirements that streaming applications impose on latency and throughput. In this demonstration, we introduce InkStream, a novel system utilizing provenance-based techniques to track the consistency of streaming data. InkStream enriches each record in the input stream with provenance annotations, encoding its consistency across a set of constraints. These annotations are propagated through query operators, enabling the quantification of the impact of the consistency on the query results. Users can engage with InkStream through interactive visualizations, real-time monitoring of query outputs, and runtime monitoring of consistency metrics. Through InkStream, monitoring data consistency over streams becomes accessible and actionable during runtime, unlike traditional post-hoc approaches.
Domaines
Base de données [cs.DB]Origine | Fichiers éditeurs autorisés sur une archive ouverte |
---|