Safely Managing Data Variety in Big Data Software Development - Archive ouverte HAL
Communication Dans Un Congrès Année : 2015

Safely Managing Data Variety in Big Data Software Development

Résumé

We consider the task of building Big Data software systems, offered as software-as-a-service. These applications are commonly backed by NoSQL data stores that address the proverbial Vs of Big Data processing: NoSQL data stores can handle large volumes of data and many systems do not enforce a global schema, to account for structural variety in data. Thus, software engineers can design the data model on the go, a flexibility that is particularly crucial in agile software development. However, NoSQL data stores commonly do not yet account for the veracity of changes when it comes to changes in the structure of persisted data. Yet this is an inevitable consequence of agile software development. In most NoSQL-based application stacks, schema evolution is completely handled within the application code, usually involving object mapper libraries. Yet simple code refactorings, such as renaming a class attribute at the source code level, can cause data loss or runtime errors once the application has been deployed to production. We address this pain point by contributing type checking rules that we have implemented within an IDE plugin. Our plugin ControVol statically type checks the object mapper class declarations against the code release history. ControVol is thus capable of detecting common yet risky cases of mismatched data and schema, and can even suggest automatic fixes.
Fichier principal
Vignette du fichier
icse.pdf (383.19 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01207655 , version 1 (01-10-2015)

Identifiants

  • HAL Id : hal-01207655 , version 1

Citer

Thomas Cerqueus, Eduardo Cunha de Almeida, Stefanie Scherzinger. Safely Managing Data Variety in Big Data Software Development. 1st IEEE/ACM International Workshop on Big Data Software Engineering, May 2015, Florence, Italy. ⟨hal-01207655⟩
144 Consultations
266 Téléchargements

Partager

More