Different countries have tried to define guidelines to quantify what levels of fungi are considered as inappropriate for housing. This retrospective study analyzes indoor fungi by cultures of airborne samples from 1012 dwellings. Altogether, 908 patients suffering from rhinitis, conjunctivitis, and asthma were compared to 104 controls free of allergies. Portuguese decree law no 118/2013 (PDL118), ANSES (a French environmental and health agency) recommendations, and health regulations of Besançon University Hospital were applied to determine the rates of non-conforming dwellings, which were respectively 55.2%, 5.2%, and 19%. Environmental microbiological results and medical data were compared. The whole number of colonies per cubic meter of air was correlated with asthma (P < 0.001) and rhinitis (P = 0.002). Sixty-seven genera and species were detected in bedrooms. Asthma was correlated to Aspergillus versicolor (P = 0.004) and Cladosporium spp. (P = 0.02). Thresholds of 300 cfu/m3 for A. versicolor or 495 cfu/m3 for Cladosporium spp. are able to discriminate 90% of the asthmatic dwellings. We propose a new protocol to obtain an optimal cost for indoor fungi surveys, excluding surface analyses, and a new guideline to interpret the results based on >1000 cfu/m3 of whole colonies and/or above threshold levels for A. versicolor or Cladosporium spp.