In joint source-channel arithmetic coding (JSCAC) schemes, additional redundancy may be introduced into an arithmetic source code in order to be more robust against transmission errors. The purpose of this work is to provide analytical tools to predict and evaluate the effectiveness of that redundancy. Integer binary Arithmetic Coding (AC) is modeled by a reduced-state automaton in order to obtain a bit-clock trellis describing the encoding process. Considering AC as a trellis code, distance spectra are then derived. In particular, an algorithm to compute the free distance of an arithmetic code is proposed. The obtained code properties allow to compute upper bounds on both bit error and symbol error probabilities and thus provide an objective criterion to analyze the behavior of JSCAC schemes when used on noisy channels. This criterion is then exploited to design efficient error-correcting arithmetic codes. Simulation results highlight the validity of the theoretical error bounds and show that for equivalent rate and complexity, a simple optimization yields JSCACs that outperform classical tandem schemes at low to medium SNR.