Four fields with areas ranging from 80 to 2000 arcmin(2) have been mapped with the photometer on board of the Infrared Space Observatory (ISO) at 90 and around 180mu m in regions of bright and faint cirrus. We examined the spatial characteristics of the infrared background emission with high spatial resolution and found that the fluctuations in the background emission limit the detection sensitivity of ISOPHOT for most of our observations. At 90mu m the power law relation between the power in the fluctuations and the spatial frequencies established from IRAS data could be extended to twice as high spatial frequencies. At 180mu m the small-scale fluctuations were studied for the first time by a cold space telescope with arcminute-resolution. A similar power law and spectral index down to spatial scales of 3arcmin as for the 90mu m component is found. For cirrus clouds the spatial frequency spectrum in the far-infrared has a similar shape as that derived from 21cm line observations of the interstellar neutral hydrogen. In faint regions the fluctuations are caused presumably by randomly distributed extragalactic sources. Future 3m class space telescopes surveying the sky around 200mu m will not be hampered by cirrus over most of the sphere. Based on observations with ISO, an ESA project with instruments funded by ESA Member States (especially the PI countries: France, Germany, the Netherlands and the United Kingdom) and with the participation of ISAS and NASA