COVID-19 vaccines are important for individuals to avoid severe illness and collectively to prevent significant societal disruptions from uncontrolled disease spread. Vaccine adoption depends both on objective data about vaccine efficiency and on perceptions, which are shaped by individual characteristics and peer influences. Despite the abundance of Agent-Based Models (ABMs) models for COVID-19 and the long-term need for booster doses, ABMs have not yet accounted for the interplay of individual and collective drivers of vaccine adoption. In this explanatory study, we modify the validated COVASIM framework such that agents observe their peers’ characteristics (derived from several datasets), use machine learning to reflect and then take decisions based on their own characteristics. We show that specific decision-making errors are necessary to replicate the real-world prevalence of COVID-19 vaccine coverage in the USA. Specifically, agents must only observe simple features of their peers (e.g., age, sex) rather than personal information (e.g., comorbidities).