The macrophages from Crohn's Disease (CD) patients are defective to control the replication of CD-associated adherent-invasive E. coli (AIEC). We aimed to identify the host factors associated with AIEC replication focusing on polymorphisms related to autophagy. Peripheral blood monocyte-derived macrophages (MDM), obtained from 95 CD patient, 30 ulcerative colitis (UC) patients and 15 healthy subjects, were genotyped for several CD-associated polymorphisms. AIEC bacteria survival increased within MDM from CD patients compared to UC (p = 0.0019). AIEC bacteria survival increased in patients with CD-associated polymorphism IRGM (p = 0.05) and reduced in those with CD-associated polymorphisms XBP-1 (p = 0.026) and ULK-1 (p = 0.033). AIEC infection led to an increase of pro-inflammatory cytokines IL-1 beta (p < 0.0001) and TNF-alpha (p < 0.0001) in CD macrophages. ULK-1 expression increased in AIEC-infected MDM from CD patients compared to MDM from UC patients or healthy subjects (p = 0.0056) and correlated with AIEC survival (p = 0.0013). Moreover, the expression of ULK-1 phosphorylation on Serine 757 decreased following to AIEC infection (p < 0.0001). Short-term silencing of ULK-1 and IRGM genes restricted and promote, respectively, AIEC survival within MDM (p = 0.0018 and p = 0.0291). In conclusion, the macrophage defect to mediate AIEC clearance in CD patients is linked to polymorphisms related to autophagy such as IRGM and ULK-1.