The problem of constructing local bulk observables from boundary CFT data is of paramount importance in holography. In this work, we begin addressing this question from a modern bootstrap perspective. Our main tool is the boundary operator expansion (BOE), which holds for any QFT in AdS. Following Kabat and Lifschytz, we argue that the BOE is strongly constrained by demanding locality of correlators involving bulk fields. Focusing on 'AdS form factors' of one bulk and two boundary insertions, we reformulate these locality constraints as a complete set of sum rules on the BOE data. We show that these sum rules lead to a manifestly local representation of form factors in terms of 'local blocks'. The sum rules are valid non-perturbatively, but are especially well-adapted for perturbative computations in AdS where they allow us to bootstrap the BOE data in a systematic fashion. Finally, in the flat space limit, we show that the AdS form factor reduces to an ordinary QFT form factor. We provide a phase shift formula for it in terms of the BOE and CFT data. In two dimensions, this formula makes manifest Watson's equations for integrable form factors under certain extremality assumptions on the CFT. We discuss the eventual modifications of our formalism to account for dressed operators in AdS.