Book Sections Year : 2023

Algorithm vs Processing Manipulation to Scale Genetic Programming to Big Data Mining

Abstract

In the era of petabyte, robust machine learning tools are needed to cope with the volume and high dimensionality of data to min. Evolutionary Algorithms (EA), such as Genetic Programming (GP), are powerful machine learning techniques with great potential to deal with big data challenges. To better exploit their capacities, additional manipulations can help the EA to alleviate the computation cost and then better look insight the large data sets. This chapter summarizes some solutions and trends to address difficulties when training EA/GP on big data sets and proposes a taxonomy to classify these solutions on three categories: Processing manipulation, algorithm manipulation and data manipulation. Two approaches are then presented and discussed. The first one, from the processing manipulation category, parallelizes GP over Spark. The second one, from the algorithm manipulation category, extends GP with active learning using dynamic and adaptive sampling. For each approach, some guidelines of implementation into the GP loop over an EA Python framework are given. A combination of the two approaches is also presented. The efficiency of these solutions is then discussed according to some published experimental studies.

No file

Dates and versions

hal-04669411 , version 1 (08-08-2024)

Identifiers

Cite

Sana Ben Hamida, Hmida Hmida. Algorithm vs Processing Manipulation to Scale Genetic Programming to Big Data Mining. Metaheuristics for Machine Learning, pp.179-199, 2023, ⟨10.1007/978-981-19-3888-7_7⟩. ⟨hal-04669411⟩
36 View
0 Download

Altmetric

Share

  • More