Imitative and contrarian behaviors are the two typical opposite attitudes of investors in stock markets. We introduce a simple model to investigate their interplay in a stock market where agents can take only two states, bullish or bearish. Each bullish (bearish) agent polls m "friends" and changes her opinion to bearish (bullish) if (1) at least mρ hb (mρ bh) among the m agents inspected are bearish (bullish) or (2) at least mρ hh > mρ hb (mρ bb > mρ bh) among the m agents inspected are bullish (bearish). The condition (1) (resp. (2)) corresponds to imitative (resp. antagonistic) behavior. In the limit where the number N of agents is infinite, the dynamics of the fraction of bullish agents is deterministic and exhibits chaotic behavior in a significant domain of the parameter space {ρ hb , ρ bh , ρ hh , ρ bb , m}. A typical chaotic trajectory is characterized by intermittent phases of chaos, quasi-periodic behavior and super-exponentially growing bubbles followed by crashes. A typical bubble starts initially by growing at an exponential rate and then crosses over to a nonlinear power law growth rate leading to a finitetime singularity. The reinjection mechanism provided by the contrarian behavior introduces a finite-size effect, rounding off these singularities and leads to chaos. We document the main stylized facts of this model in the symmetric and asymmetric cases. This model is one of the rare agent-based models that give rise to interesting non-periodic complex dynamics in the "thermodynamic" limit (of an infinite number N of agents). We also discuss the case of a finite number of agents, which introduces an endogenous source of noise superimposed on the chaotic dynamics. "Human behavior is a main factor in how markets act. Indeed, sometimes markets act quickly, violently with little warning. [.. .] Ultimately, history tells us that there will be a correction of some significant dimension. I have no doubt that, human nature being what it is, that it is going to happen again and again." Alan Greenspan, Chairman of the Federal Reserve of the USA, before the Committee on Banking and Financial Services, U.S. House of Representatives, July 24, 1998.