Steering undulatory micro-swimmers in a fluid flow through reinforcement learning - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue European Physical Journal E: Soft matter and biological physics Année : 2023

Steering undulatory micro-swimmers in a fluid flow through reinforcement learning

Résumé

This work aims at finding optimal navigation policies for thin, deformable microswimmers that progress in a viscous fluid by propagating a sinusoidal undulation along their slender body. These active filaments are embedded in a prescribed, non-homogeneous flow, in which their swimming undulations have to compete with the drifts, strains, and deformations inflicted by the outer velocity field. Such an intricate situation, where swimming and navigation are tightly bonded, is addressed using various methods of reinforcement learning. Each swimmer has only access to restricted information on its configuration and has to select accordingly an action among a limited set. The optimisation problem then consists in finding the policy leading to the most efficient displacement in a given direction. It is found that usual methods do not converge and this pitfall is interpreted as a combined consequence of the non-Markovianity of the decision process, together with the highly chaotic nature of the dynamics, which is responsible for high variability in learning efficiencies. Still, we provide an alternative method to construct efficient policies, which is based on running several independent realisations of Q-learning. This allows the construction of a set of admissible policies whose properties can be studied in detail and compared to assess their efficiency and robustness.

Dates et versions

hal-04058675 , version 1 (05-04-2023)

Licence

Paternité

Identifiants

Citer

Zakarya El Khiyati, Raphaël Chesneaux, Laëtitia Giraldi, Jérémie Bec. Steering undulatory micro-swimmers in a fluid flow through reinforcement learning. European Physical Journal E: Soft matter and biological physics, 2023, 46 (6), pp.43. ⟨10.1140/epje/s10189-023-00293-8⟩. ⟨hal-04058675⟩
36 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More