In this paper, we present an unsupervised state representation learning of spatio-temporally evolving sequences of autonomous agents' observations. Our method uses contrastive learning through mutual information (MI) maximization between a sample and the views derived through selection of pixels from the sample and other randomly selected negative samples. Our method employs balancing MI by finding the optimal ratios of positive-to-negative pixels in these derived (constructed) views. We performed several experiments and determined the optimal ratios of positive-to-negative signals to balance the MI between a given sample and the constructed views. The newly introduced method is named as Balanced View Spatial Deep InfoMax (BVS-DIM). We evaluated our method on Atari games and performed comparisons with the state-of-the-art unsupervised state representation learning baseline method. We show that our solution enables to successfully learn state representations from sparsely sampled or randomly shuffled observations. Our BVS-DIM method also marginally enhances the representation powers of encoders to capture high-level latent factors of the agents' observations when compared with the baseline method.