In many contexts individuals are subject to norms and decisions they disagree with ideologically. What is the effect of regularly being in an ideological minority on the propensity to confront majority norms and decisions? We study this in an ideologically-salient field setting – US appeals courts – using exogenous predictors of ideology and random assignment of judges. We find that ideological interaction silences extremists: Judges who are ideologically extreme relative to their peers are less confrontational – dissent less often – than other judges, despite shaping case outcomes the least. Considering many mechanisms, we find that a model of peer pressure where agents perceive concave ideological costs can explain the observations.