Data Pipeline System Designs for In-network Learning - Archive ouverte HAL
Conference Papers Year : 2024

Data Pipeline System Designs for In-network Learning

Abstract

This paper introduces the design of a data pipeline system (DPS) integrated with artificial intelligence (AIF) functions to support real-time AI learning and operations for network automation in 5G/6G systems. To enhance the DPS for network learning, we design it as a chain of functions, namely ingress and egress Network Data Broker Function (iNDBF and eNDBF) and Network Data Preprocessing Function (NDPPF). To take into account the distributed nature of the network architecture of 5G and beyond, we design the DPS to be integrated seamlessly with a distributed learning system such as the federated learning. We implement a realistic evaluation framework, employing a real dataset from a national mobile operator to simulate the network architecture. Additionally, a federated learning framework for anomaly detection is integrated with the DPS to assess the effectiveness of our proposal. Evaluation results show that delays in end-to-end data transmission and preprocessing to the AIF locations can cause a distributed learning process to work with stale data. The results also highlight how these delays can lead to desynchronisation of the distributed learning process, and thus generate AI models with low accuracy.
Fichier principal
Vignette du fichier
PNtumba_et_al_CNSM_2024_v8.pdf (524.13 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04681121 , version 1 (14-09-2024)

Identifiers

  • HAL Id : hal-04681121 , version 1

Cite

Patient Ntumba, Nour-El-Houda Yellas, Salah Bin Ruba, Fehmi Ben Abdesslem, Stefano Secci. Data Pipeline System Designs for In-network Learning. 20th International Conference on Network and Service Management Prague, Czech Republic // 28 - 31 October 2024, Oct 2024, Prague, Czech Republic. ⟨hal-04681121⟩
53 View
0 Download

Share

More