The dorsal diencephalon (or epithalamus) of larval zebrafish displays distinct left-right asymmetries. The pineal complex consists of the pineal organ anlage and an unpaired, left-sided accessory organ – the parapineal. The neighboring brain nuclei, the left and right dorsal habenulae,show consistent differences in their size, density of neuropil and gene expression. Mutational analyses demonstrate a correlation between the left-right position of the parapineal and the laterality of the habenular nuclei. We show that selective ablation of the parapineal organ results in the loss of habenular asymmetry. The left-sided parapineal therefore influences the left-right identity of adjacent brain nuclei, indicating that laterality of the dorsal diencephalon arises in a step-wise fashion.