Recognizing objects using computational methods have become a popular research endeavor among researchers. In this paper, the recognition of fish based on GCFD (Generalized Color Fourier Descriptor) is introduced. The features are extracted using the GCFD technique which represents the image in a frequency domain. By analyzing the frequencies, the non-related information (backgrounds, not required lines or borders) are identified by performing some spectrum changes on the frequencies. The required objects in this study are the fish. In other words, the frequencies corresponding to the fish are maintained while other frequencies are removed from the frequency domain. After removing the non-related frequencies, the frequency domain is inversed in order to obtain the required image for further image processing. GCFD is used as a descriptor to extract the features of the fish as it is invariant to rotation and translation. A cultured fish tank installed with high-end video cameras is required to record the video from side and top views. Koi fish are chosen due to their active swimming behavior, variety of colors and easy-to-adapt habitat in the water. The evaluation of the technique is based on Bhattacharyya Distance. Some improvements were obtained in the recognition rate using the GCFD compared with existing color descriptors. The improvement can lead to better classifications of objects.