We prove that one can construct various kinds of automata over finite words for which some elementary properties are actually independent from strong set theories like Tn =:ZFC + "There exist (at least) n inaccessible cardinals", for integers n ≥ 0. In particular, we prove independence results for languages of finite words generated by context-free grammars, or accepted by 2-tape or 1-counter automata. Moreover we get some independence results for weighted automata and for some related finitely generated subsemigroups of the set Z ^{3×3} of 3-3 matrices with integer entries. Some of these latter results are independence results from the Peano axiomatic system PA.