A large portion of online display advertising inventory is sold through real time auctions. The bidding algorithms need to estimate precisely the value of each display. Many bidding models estimate this value as the probability that a sale is attributed to this display, but this approach does not capture that a user may be shown a sequence of several displays. By mixing tools from causal reasoning and reinforcement learning to model this sequence of auctions, we derive a simple rule to improve this estimate. We test the change online in a production environment and the results validate the approach. We believe this methodology could be adapted to tackle the notoriously difficult problem of building an incremental bidder.