Communication Dans Un Congrès Année : 2025

GRAMMAR REINFORCEMENT LEARNING: PATH AND CYCLE COUNTING IN GRAPHS WITH A CONTEXT-FREE GRAMMAR AND TRANSFORMER APPROACH

Apprentissage par renforcement grammatical : comptage des chemins et des cycles dans les graphes à l’aide d’une grammaire hors-contexte et d’une approche par transformeur.

Résumé

This paper presents Grammar Reinforcement Learning (GRL), a reinforcement learning algorithm that uses Monte Carlo Tree Search (MCTS) and a transformer architecture that models a Pushdown Automaton (PDA) within a context-free grammar (CFG) framework. Taking as use case the problem of efficiently counting paths and cycles in graphs, a key challenge in network analysis, computer science, biology, and social sciences, GRL discovers new matrix-based formulas for path/cycle counting that improve computational efficiency by factors of two to six w.r.t state-of-the-art approaches. Our contributions include: (i) a framework for generating gramformers that operate within a CFG, (ii) the development of GRL for optimizing formulas within grammatical structures, and (iii) the discovery of novel formulas for graph substructure counting, leading to significant computational improvements.

Published as a conference paper at ICLR 2025 As mentioned in Section 1, path/cycle counting has been extensively tackled in the literature. In the early 1970s, Harary and Manvel (1971) introduced algorithms for counting cycles up to length five at the graph level. Two decades later, Alon et al. (1997) refined these algorithms, extending cycle counting to lengths of up to seven, and conjectured that these methods can also be adapted to count cycles at the node level. Later Voropaev and Perepechko (2012) established a relationship between the counting of l-cycles at the edge level and the counting of (l -1)-paths at the edge level using a simple formula. By deriving explicit formulae for the counting of paths of length up to six at the edge level, they were able to compute the number of cycles of length up to seven. More recently, Jokić and Van Mieghem (2022) rediscovered the formulae for paths of length up to four from Voropaev and Perepechko (2012). In contrast, Giscard et al. (2019) proposed an algorithm capable of counting cycles and paths of arbitrary lengths. However, they acknowledged that their method is slower than those presented by Alon et al. (1997) and Voropaev and Perepechko (2012). Specifically, since the latter algorithms are based on matrix multiplication, they exhibit a computational complexity of O n 3 , where n is the number of nodes. As noted by Giscard et al. ( 2019), these matrix-based approaches remain the most efficient known methods for counting paths and cycles of lengths up to six and seven, respectively.

CONTEXT-FREE GRAMMAR.

Throughout this paper, we employ standard formal language notation: Γ * denotes the set of all finite-length strings over the alphabet Γ, and ε represents the empty string. The relevant definitions used in this context are as follows:

Definition 2.1 (Context-Free Grammar) A Context-Free Grammar (CFG) G is defined as a 4-tuple (V, Σ, R, S), where V is a finite set of variables, Σ is a finite set of terminal symbols, R is a finite set of production rules of the form V → (V ∪ Σ)

* , and S is the start variable. Note that R fully characterizes the CFG, following the convention that the start variable is placed on the top left and that the symbol | represents "or". Definition 2.2 (Derivation) Let G be a CFG. For u, v ∈ (V ∪ Σ) * , we define u =⇒ v if u can be transformed into v by applying a single production rule, and u * =⇒ v if u can be transformed into v by applying a sequence of production rules from G.

Fichier principal
Vignette du fichier
4538_Grammar_Reinforcement_Lea.pdf (7.15 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05096789 , version 1 (04-06-2025)

Licence

Identifiants

  • HAL Id : hal-05096789 , version 1

Citer

Jason Piquenot, Maxime Bérar, Pierre Héroux, Romain Raveaux, Sébastien Adam, et al.. GRAMMAR REINFORCEMENT LEARNING: PATH AND CYCLE COUNTING IN GRAPHS WITH A CONTEXT-FREE GRAMMAR AND TRANSFORMER APPROACH. The Thirteenth International Conference on Learning Representations, Apr 2025, Singapour, Singapore. ⟨hal-05096789⟩
498 Consultations
229 Téléchargements

Partager

  • More