Pollution with trace metals (TM) has been shown to affect diversity and/or composition of plant and animal communities. While ecotoxicological studies have estimated the impact of TM contamination on plant and animal communities separately, ecological studies have widely demonstrated that vegetation is an important factor shaping invertebrate communities. It is supposed that changes in invertebrate communities under TM contamination would be explained by both direct impact of TM on invertebrate organisms and indirect effects due to changes in plant communities. However, no study has clearly investigated which would more importantly shape invertebrate communities under TM contamination. Here, we hypothesized that invertebrate communities under TM contamination would be affected more importantly by plant communities which constitute their habitat and/or food than by direct impact of TM. Our analysis showed that diversity and community identity of flying invertebrates were explained only by plant diversity which was not affected by TM contamination. Diversity of ground-dwelling (GD) invertebrates in spring was explained more importantly by plant diversity (27% of variation) than by soil characteristics including TM concentrations (8%), whereas their community identity was evenly explained by plant diversity and soil characteristics (2–7%). In autumn, diversity of GD invertebrates was only explained by plant diversity (12%), and their identity was only explained by soil characteristics (8%). We conclude that vegetation shapes invertebrate communities more importantly than direct effects of TM on invertebrates. Vegetation should be taken into account when addressing the impacts of environmental contamination on animal communities.