Eﬃcient plurality consensus, or: The beneﬁts of cleaning up from time to time

Plurality consensus considers a network of n nodes, each having one of k opinions. Nodes execute a (randomized) distributed protocol with the goal that all nodes adopt the plurality (the opinion initially supported by the most nodes). Communication is realized via the Gossip (or random phone call) model. A major open question has been whether there is a protocol for the complete graph that converges (w.h.p.) in polylogarithmic time and uses only polylogarithmic memory per node (local memory). We answer this question affirmatively. We propose two protocols that need only mild assumptions on the bias in favor of the plurality. As an example of our results, consider the complete graph and an arbitrarily small constant multiplicative bias in favor of the plurality. Our first protocol achieves plurality consensus in O(log k · loglog n ) rounds using log k +Θ(loglog k ) bits of local memory. Our second protocol achieves plurality consensus in O(log n · loglog n ) rounds using only log k + 4 bits of local memory. This disproves a conjecture by Becchetti et al. (SODA’15) implying that any protocol with local memory log k +O(1) has worst-case runtime Ω( k ). We provide similar bounds for much weaker bias assumptions. At the heart of our protocols lies an undecided state , an idea introduced by Angluin et al. (Distributed Computing’08).


Introduction
Reaching plurality consensus is a fundamental problem in distributed computing.We consider this problem in a networked setting, where a graph is given in which each node initially holds one of k ∈ N opinions.The objective is the design of an efficient distributed protocol that ensures that eventually all nodes agree on the initial plurality opinion, which is the opinion that is initially supported by the most nodes.This problem is also referred to as majority consensus or proportionate agreement [1,3,19].In accordance with [5] and others, we prefer to refer to it as plurality consensus, so as to make clear that the opinion eventually to be attained by all nodes need not initially have been absolute majority.One need not stray too far from the core of distributed computing to come across direct applications of plurality consensus: the handling of fault tolerance in parallel computing or the implementation of majority-based conflict resolution for CRCW PRAMs (and derivative models) are immediate examples.
Natural metrics for plurality consensus protocols are running time and memory overhead.The latter is the additional amount of (local) memory needed by each node above and beyond the bits XX:2 Efficient Plurality Consensus required to store its current opinion.Results are typically expressed in terms of the number of nodes, the number of initial opinions, and the initial bias (between plurality and remaining opinions).Of particular interest, to us and in general, are fast protocols with small memory overhead.As in [5] we assume a synchronous Pull-based Gossip communication model on the underlying graph.Here, in each discrete round, every node may contact one neighbor and query that neighbor's opinion.We mostly assume the complete graph, but using a construction of [5] we can easily extend our results to regular expanders.We define the (relative) plurality gap γ >1 as the ratio between the plurality opinion and the second most common opinion.We analyze two protocols in this paper: A protocol with running time O logk•loglog γ n+loglogn and memory overhead Θ(loglogk).A protocol with running time O logn•loglog γ n and memory overhead 4.
Plurality consensus is a member of the class of population dynamics, which are of great interest in fields as varied as epidemiology, physics, statistics, biology, chemistry, or sociology.All these have in common an initial population of agents with some initial properties and a protocol (dynamics) that in some manner changes the properties of given agents usually based on those of other agents.Specific models are as varied as the problems themselves; for instance, we may or may not have an underlying graph structure, a prescribed timing model, or restrictions on the amount and nature of communication.Other related dynamics are the lately en vogue voting protocols and Moran-type processes.

Related Work
In [3], Aspnes et al. consider k =2 initial opinions {x,y} and the complete graph as neighborhood structure.They introduce a third state, referred to as blank, b, which is a crucial ingredient in their protocol and analysis.Their protocol works such that an activated node u picks another node v at random.Given the opinions of those two nodes, the transition now proceeds as follows: If u has a non-blank opinion and sees in v the other non-blank opinion then it changes to b, if it has a non-blank opinion and sees in v the same non-blank opinion or b then it maintains its opinion, and if it has the blank opinion then it just copies whatever it sees in v.The authors show that with high probability all n nodes reach consensus within O(nlogn) many interactions (corresponding to parallel convergence time O(logn)), and the consensus value is the plurality value provided its (absolute) initial bias is at least ω( √ nlogn).Each node needs to be able to store one of three values, x, y, or b.In the case of two opinions the plurality problem can be solved by calculating the median of the opinions.In [11] the authors present such a protocol that converges in O(logn) rounds and has constant memory overhead, if the initial difference bias c 1 −c 2 is at least Ω √ nlogn .In [2] the authors consider the plurality problem in a sequential setting where only one node can change its opinion at a time.They present a new protocol called Average and Conquer (AVC) that solves plurality exactly, in sequential time O(nlogn/(s )+lognlogs), where n ( >1/2) is the size of the plurality opinion and s the number of states.In [7] the authors generalize the former result to general networks and k opinions.They introduce protocols that solve the plurality consensus problem that are based on an interesting relationship between plurality consensus and distributed load balancing.
In [5] Becchetti et al. take the model of [3] and generalise it to k ≥2 initial opinions.They still use the blank state, which they however refer to as undecided.The authors express their results in terms of md(c), the monochromatic distance of configuration c.Formally, md(c)= k i=1 c i /c 1 , where c i is the number of nodes with opinion i and c 1 ≥ c 2 ••• ≥ c k .Note that md(c) is always between O(1) and O(k).Then authors show almost-tight bounds on convergence time.Formally, let k =k(n) be any function such that k =O (n/logn) 1/3 , and consider any initial configuration with c 1 ≥ (1+α)•c 2 , where α ≥ 0 is any arbitrarily-small constant.Their protocol converges in O(md(c)•logn), rounds (w.h.p.) and it has only constant memory overhead.They also show that for k =O (n/logn) 1/6 and any initial configuration the convergence time of their protocol is (w.h.p.) linear in the monochromatic distance.Finally, they show how to adapt their results to regular expanders using random walks to sample the opinion of nodes.As with [3], they require one state more than is necessary to store the actual opinion values.They conjecture that any protocol using logk+O(1) bits of memory has runtime at least linear in k in the worst case -as discussed later, we partially refute this conjecture.In [6] the authors consider the 3-majority dynamics.They show that for k ≤n α (with constant α) the 3-majority dynamics converges to an almost-consensus state in time O (k 2 √ logn+kllogn)(k+logn) .An almost-consensus state is defined as a state where all but a subset of size O(n γ ) (for constant γ <1) of the nodes support the same opinion.In [4] the authors consider the undecided dynamics in complete graphs in an asynchronous setting.They derive the time of convergence and an upper bound for the probability of error.
A line of research which is related to the plurality consensus problem is the voting problem.The setting is the same, a network with n nodes is given and initially every node has one of k opinions.
Here the goal is that all nodes agree on one opinion, which is not necessarily the plurality opinion.
is the ratio of the square of the sum of node degrees over the sum of the squared degrees.The authors of [9, 10] consider a modification of the standard voter model with two opinions, which they call two-sample voting.In every round, each node chooses two of its neighbors randomly and adopts their opinion only if they both agree.For regular graphs and random regular graphs, it is shown that two-sample voting has a consensus time of O(logn) if the initial imbalance between the nodes having the two opinions is large enough.In [13] the authors consider a 2-choice voting protocol for k opinions in the complete graph.Their protocol converges to the majority opinion in time O(k•logn), with high probability, if k=O(n ) for some small >0, and the initial absolute gap between largest and second-largest opinion is Ω √ nlogn .They also show that there exist initial configurations where the Θ(k) bound on the run time is matched.Independently, they also give a protocol which is similar to our simple, first protocol (cf.Section 3) and has roughly the same voting time.Other related papers from literature about sensor networks include [12] (which considers binary interval consensus, which can be used to solve majorization) and [17] (which considers the plurality problem in a different distributed model and for constant k).

Our Contribution
In Section 3 we present and analyze protocol RepeatedCleanup, which in a complete graph of n nodes and k opinions works in O logk•loglog γ n+loglogn rounds and has a memory overhead of O(loglogk) bits.The protocol is a synchronized version of the protocol from [6], in which the nodes "lose" their opinion and adopt the state "undecided" whenever they sample a node with a different opinion.Then they will adopt the "real" opinion of the next (i.e., not undecided) node they sample.The nodes keep changing their opinion until all nodes agree on the plurality opinion.In contrast, our protocol works in O loglog γ n many phases, each of length Θ(logk).In the first round of every phase the nodes sample a node and lose the opinion if the sampled node has a different opinion.They then use the rest of the phase to find a new "real" opinion.In [5] the authors show that for k =O (n/logn) 1/6 and any initial configuration the convergence time of their protocol is linear in the monochromatic distance, which can be as large as O(k).Hence, our protocol outperforms the lower bound for the protocol of [5].Interestingly, the speed-up is reached by synchronizing the protocol, which can also be regarded as slowing it down.In our protocol a node that just found a real opinion again waits until the beginning of the next round to sample another node, instead of doing that immediately.
The drawback of our first protocol is that it uses a counter to determine the end of a phase.In Section 4 we present a protocol that works in O logn•loglog γ n rounds and has a memory overhead of only 4 bits.The main idea of the protocol is to slow down the progress in the individual phases by having nodes toss a biased coin with success probability 1/n, basically replacing the deterministic counter by a probabilistic counterpart.Our result shows that a conjecture by Becchetti et al. [5], implying that any protocol with constant memory overhead has worst-case runtime Ω(k), does not hold if nodes have access to such a coin.Note that the coin toss is not necessary if nodes can decide whether they sampled themselves (or a marked node/leader).
A very recent, independent result by Ghaffari and Parter [14] suggests a protocol for plurality consensus with similar time and memory bounds as ours.They employ the same basic idea of cleanup and decision-accumulation rounds (cf.Section 3), which they name selection and recovery steps.Their final protocol differs in that they use some of the undecided nodes as clock nodes (which use the logk bits normally used to store the opinion to count time) to help synchronize other nodes.

Model & Notation
We consider protocols in the complete graph with n∈N nodes.Each node u has one of k ∈N opinions op u ∈{1,2,...,k}.We write op u =⊥ to indicate that node u is undecided.Time is modelled in synchronous, discrete and parallel rounds and we assume a Pull-based Gossip model for communication (nodes can request information from one other node chosen uniformly at random).Note that each node needs at least logk bits of local memory (to store its current opinion).Any additional number of bits per node needed by a given protocol is called the protocol's memory overhead.
Notation.In the following, • 1 and • 2 denote the L 1 and L 2 norms, respectively, that is, for an n-dimensional vector x.For a real value x > 0, its binary logarithm is denoted by logx and its natural logarithm by lnx.For an integer i, the shorthand [i]:={1,2,...,i} denotes the set of the first i integers.The phrase with high probability (w.h.p.) refers to probabilities of the form 1−n −Ω(1) .At any point in time, the system can be described by a k-dimensional vector x=(x 1 ,x 2 ,...,x k )∈ {0/n,1/n,...,1} k ⊆[0,1] k , where the i-th entry x i ∈[0,1] denotes the current fraction of nodes with opinion i.We call such a vector x a configuration.Note that 1− x 1 ∈ [0,1] is the fraction of undecided nodes.X i (t) is defined as the random variable that takes as its values the configuration produced by the protocol at the end of round t ∈ N 0 .We use x := X(0) to denote the initial (fixed) configuration.The random variable Y (t):=1− X(t) 1 with values in ∈[0,1] denotes the fraction of undecided nodes at the end of round t.
To measure how far we are from plurality consensus, we define two plurality gap notions: Assuming (w.l.o.g.) that 1 is a most common opinion, let ψ(x) := x 1 −max i =1 x i ∈ [0,1] be the absolute plurality gap.Similarly, γ(x):=x 1 /max i =1 x i ≥1 is the relative plurality gap.

Assumptions.
In order to guarantee convergence, we need some (mild) bias assumptions.Given the initial configuration x, without loss of generality (w.l.o.g.), we assume x1 ≥ xi for all i∈[k] and refer to (the initially most common) opinion 1 as the plurality opinion.For most of the analysis we assume Listing 1 Protocol RepeatedCleanup as executed by node u in round t∈N.It works in phases of length T =Θ(logk) and has memory overhead Θ(loglogk). (2) While Condition (1) is essential1 , Condition ( 2) is without loss of generality (it can be achieved by merging small opinions).We define ρ=ρ(n):=(logn) 2 / √ n and call a configuration x biased if ψ(x)=ω(ρ).Our analysis assumes n to be at least some sufficiently large constant.

Plurality Consensus with logk+Θ(loglogk) Bits
We divide time into phases, each consisting of T :=5+2logk rounds.In each round, every node u uses a Pull operation to sample a random node v and checks its opinion op v .We distinguish two types of rounds: Cleanup rounds represent the first round of each phase.Here, u becomes undecided if op v differs from u's own opinion op u (and keeps its opinion otherwise).Decision-accumulation rounds make up the remaining T −1 rounds of a phase.Here, only undecided nodes act and simply adopt the pulled opinion op v .
The synchronization of the steps in which nodes lose their opinion and adopt opinions of sampled nodes is key to the fast convergence.As we will see in the analysis they ensure that each phase increases the relative plurality gap exponentially.We call this protocol RepeatedCleanup.See Listing 1 for a formal description.In the remainder of this section we prove the following theorem.
We use the shorthand X(τ,t):=X((τ −1)•T +t) to denote the configuration at the end of round t in phase τ.Similarly, X i (τ,t) and Y (τ,t) denote the corresponding fractions of nodes.The following definition identifies opinions that are supported by only a small fraction of nodes (and, thus, are likely to vanish in a cleanup round), and configurations that do not have too many undecided nodes.
The following simple observation already hints at the basic change of configurations during a phase: in expectation, the cleanup round squares the relative plurality gap (but reduces the

XX:6
Efficient Plurality Consensus absolute support of each opinion), while the decision-accumulation rounds increase the absolute support of each opinion (not changing the relative plurality gap).
Observation 2. Consider a configuration x with y :=1− x 1 .If the configuration X results from a cleanup round on x and Y =1− X 1 , then Similarly, if X results from a decision-accumulation round, then Analysis Overview.The bound on the memory overhead in Theorem 1 immediately follows from the protocol description.The runtime bound is proven in three steps: Lemma 7 shows that all non-plurality opinions become negligible during the first O loglog γ(x) n phases.Lemma 8 proves that all these negligible opinions vanish within an additional constant number of phases.
Lemma 9 shows that any remaining undecided nodes vanish in another O(loglogn) phases.Before we prove these key lemmas (Section 3.3), we show how configurations evolve during single rounds (Section 3.1) and single phases (Section 3.2).

Change During a Single Round
Our first two claims show concentration for the expected configuration change in cleanup and decision-accumulation rounds we saw in Observation 2. They follow by standard Chernoff bounds.
Claim 3 (Cleanup Round).Consider a configuration x at the beginning of phase τ.Let a>0 be a constant.

Change During a Single Phase
Next, we use the effects of single rounds to show that (a) the property of being alive is (w.h.p.) invariant from phase to phase (Claim 5) and (b) the relative plurality gap of a biased configuration increases (w.h.p.) exponentially during a phase (Claim 6).

Wrapping up the Analysis
We now have the tools to prove the three key lemmas mentioned before (which immediately imply Theorem 1).We first show that after O loglog γ(x) n phases, (w.h.p.) all opinions i ≥ 2 are negligible and the configuration is still alive.
The next lemma shows that once we are in a configuration that is alive and where all opinions i ≥ 2 are negligible, (w.h.p.) all these negligible opinions vanish within a constant number of additional phases.
Lemma 8. Consider a configuration x that is alive and for which x i ≤ρ for all i =1.Define x 1 /ρ as in Claim 6.Since x 1 ≥1−e −1 but x i ≤ρ for all i≥2, we must have x 1 =Ω(1) (or x 1 ≤ k•ρ+o(1)=o(1) would contradict x being alive).In particular, we get γ ρ =Ω(1/ρ).Combining this with Claim 5 via a union bound, with probability at least 1−2n −3 configuration X(τ 1 +1,0) is alive and X i (τ 1 +1,0)<γ −3/2 ρ =O ρ 3/2 for all i≥2.Now consider the cleanup round of phase τ 1 +1 for an opinion i≥2 with X i (τ 1 +1,0)=O ρ 3/2 .The probability that even one node of such an opinion remains decided after the cleanup round is at most Repeating this for a constant number of phases (note that we can use our high probability bounds to guarantee that the configuration stays alive and the plurality gap high enough) and applying the geometric distribution, we get that the probability for an opinion i≥2 to survive c more phases is at most O ρ 3c n .The probability that even one of the k−1≤ √ n negligible opinions survives these c phases is O ρ 3c •n 3/2 =o n −(c−1)•5/4 .The claim's statement follows for c=2.
Our last lemma shows that once we reached a configuration that is alive and only the plurality opinion is left, (w.h.p.) all nodes adopt the plurality in O(loglogn) phases.

Plurality Consensus with logk+Bits
The non-constant memory overhead of RepeatedCleanup is due to the round counter used to synchronize phases.We now present a protocol that avoids this counter.Each node u stores its opinion op u ∈[k], a phase counter p u ∈N, and a state variable s u ∈{S,(U,I),(D,I),(U,A),(D,A)}.
Our description and analysis assume p u to be an arbitrary integer.While this would result in a non-constant memory overhead, we will prove that (w.h.p.) |p u −p v |≤1 for any two nodes u,v and any round.Thus, the actual implementation can restrict p u to {0,1,2}, such that we get a memory overhead of log(3•5)≤4 bits.We call this protocol ConstOverhead.Our main result is the following theorem: Theorem 10.ConstOverhead has a local memory overhead of 4 bits (15 states).If started on a biased configuration x with relative plurality gap γ :=γ(x), then (w.h.p.) plurality consensus is achieved in O logn•loglog γ n rounds.
The rest of this section describes the protocol as well as the underlying idea and gives a sketch of the analysis.More details can be found in the full version.Protocol Overview.Initially, each node u starts with p u =1 and s u =S.We call u undecided if s u = (U,•), decided if s u = (D,•), inactive if s u = (•,I), and active if s u = (•,A).At the start of a round, each node u samples a random node v and uses v's data to transition through its phase.Listing 2 gives the formal protocol description and Figure 1 an illustration.
We start with a high level description of the protocol which we gradually refine.Basically, ConstOverhead mimics the synchronized behavior of RepeatedCleanup.As in our first protocol, there are two ways to transition through a phase: if a node u samples another node v of the same opinion, it keeps its opinion and has to wait for the other nodes to catch up.While RepeatedCleanup implemented this waiting via a counter, ConstOverhead uses slow transitions (highlighted red/thick in Figure 1).Otherwise, if u samples a node v of a different opinion, it becomes undecided and keeps sampling nodes until it finds a new opinion.
Becoming decided or undecided is modeled by entering the inactive decided state (D,I) or inactive undecided state (U,I), respectively.This transition from S to one of these inactive states corresponds to cleanup rounds of RepeatedCleanup.Now, there is a slow transition to the active decided state (D,A) and active undecided state (U,A).This transition from inactive to active ensures that (a) decided nodes do not enter the next phase too early (which could require a large phase counter) and (b) undecided nodes do not sample decided nodes too early (which could result in a skewed distribution, since the number of decided nodes might be too small).Once an undecided node u becomes active, it keeps sampling nodes until it finds a new opinion.This corresponds to decision-accumulation rounds of RepeatedCleanup.Note that the transition from (D,A) to S (at which the phase counter is increased) is slow.This ensures that not too many nodes enter the next phase before all undecided nodes found a new opinion (which could, as before, require a large phase counter).Some Subtleties.While the above reflects the basic behavior of our protocol, we omitted some details.Let us make a few important and useful observations: A node u always checks whether the sampled node v is not too far behind.Nodes do not explicitly forget their opinion when becoming undecided but simply overwrite their old opinion when they find a new opinion.In particular, if a node from an earlier state asks for u's opinion while u is undecided, u answers with its most recent opinion (the provision of which does not cost us extra in terms of memory as we simply retain the information where it was; the undecided state is seperately encoded in the already accounted-for Θ(1) additional bits).Slow transitions s→s between two states s and s basically simulate Pull-rumor spreading [18]: Nodes in state s or later are "informed", while all other nodes are "uninformed".When u samples an informed node (a node in s or beyond), it can cross the slow transition (independent of the sampled opinion).The Bernoulli trial Ber(1/n) in slow transitions ensures that, eventually, at least one node is "informed".Without it, no node could cross such a transition.Analysis Overview.Given that ConstOverhead is based on the same principle as Repeat-edCleanup, namely employing synchronized cleanup rounds to increase the relative plurality gap exponentially from phase to phase, it is natural to use a similar analysis.The major difficulty stems from the fact that our synchronization primitive is now probabilistic (slow transitions) instead of deterministic (counter).In particular, in the case of ConstOverhead there is no guarantee that nodes wait at slow transitions for other nodes to catch up; in fact, there will typically be a few nodes that proceed early on over slow transitions.This might disturb our analysis in two ways: (a) if nodes could proceed arbitrarily far ahead, their phase counter could become arbitrarily high, resulting in a non-constant memory overhead, and (b) if a small group of nodes with a non-plurality opinion were "lucky" and proceeded fast, these nodes might cause more and more latecomers to adopt a non-plurality opinion.
The major tool to address both of these issues are two probabilistic synchronization results.The first shows that (w.h.p.) there is a period/stage of O(logn) consecutive rounds such that: If n − polylog(n) nodes are in state (D,A) of phase τ − 1 or state S of phase τ, and all remaining nodes are in one of the two inactive states (•,I) of phase τ, then at the end of this period, n−polylog(n) nodes are in one of the two inactive states (•,I) of phase τ, and all remaining nodes are in one of the two active states (•,A) of phase τ.
The second result shows that (w.h.p.) there is a period/stage of O(logn) consecutive rounds such that: If n−polylog(n) nodes are in one of the two inactive states (•,I) of phase τ, and all remaining nodes are in one of the two active states (•,A) of phase τ, then at the end of this period, n−polylog(n) nodes are in state (D,A) of phase τ or state S of phase τ +1, and all remaining nodes are in one of the two inactive states (•,I) of phase τ.
Note that the final condition of the first stage fits perfectly into the assumption of the second stage and vice versa.We call the first stage the cleanup stage and the second stage the decision-accumulation stage (in the style of the corresponding round in RepeatedCleanup).Since these stages are well-sparated, we can prove an analogue of Claim 3 and an analogue of Claim 4. Equipped with these, the remainder of the proof is basically identical to the proof of RepeatedCleanup (which was completely based on these concentration bounds).
A sequential version of the voter model was introduced in [16].The parallel voter model was first analyzed in [15].The authors of [15] bound the expected consensus time in terms of the expected meeting time T m of two random walks and show a bound of O(T m •logn)=O n 3 logn .The authors of [8] provide an improved upper bound of O 1/(1−λ 2 )•log 4 n+ρ on the expected consensus time for any graph G, where λ 2 is the second eigenvalue of the transition matrix of a random walk on G, and ρ= u∈V

Figure 1
Figure1Illustration how nodes transition through phases.Red (thick) state transitions highlight slow transitions that ensure that nodes do not get out of sync too much.