Conversion agricultural straws to building insulation materials has attracted great interest due to their good hygrothermal properties and low carbon footprint. In this study, fully bio-sourced composite materials were designed based on rice straw and natural binders. Sodium alginate and chitosan were used as the binders. A crosslinking method was proposed to prevent the water-soluble issue of the composites using sodium alginate as the binder. The influence of fiber size, binder type and binder ratio on the hygrothermal properties of the composite materials made of rice straw were investigated. It was found that all the materials are insulating with their thermal conductivity values between 0.038 and 0.047 W/(m⋅K). The alginate-based composites had the higher water vapor permeability and moisture sorption capacity than the chitosan-based composites. Larger fiber size and lower binder ratio resulted in lower thermal conductivity and higher water vapor permeability. However, fiber size and binder ratio had a negligible effect on moisture sorption capacity of the composite materials. Additionally, the mold growth was investigated by exposing the composite materials to severe conditions, i.e., 93% RH and 97% RH at 20 °C. Mold proliferation was found to be easier and quicker in the surface of alginate-based composites compared to the chitosan-based composites.