Multimodal learning involves the use of multiple senses (touch, visual, auditory, etc.) during the learning process to better understand a phenomenon. In the computational domain, we need systems to understand, interpret, and reason with multimodal data, and while there have been enormous advances in the field, many of the desired capabilities remain beyond our reach. The objective of such systems is to leverage different semantically related data types to output better predictions for a phenomenon of interest. For example, for users with sensory disabilities such as visual, to carry out daily tasks such as making a purchase or finding a place in a city, the visual information of their environment has to be transformed into a different modality with more semantic meaning. A system for this purpose could use auditory information provided by the user that specifies what information is required and that can be easily transformed into textual data, and visual information such as images obtained from its surroundings to help the user make a decision. Therefore, it would be a multimodal system leveraging information from three different modalities : auditory + text + images.This thesis focuses on advancing multimodal learning research through various scientific contributions : we simplify the creation of deep learning models by proposing frameworks that find a common semantic space for visual and textual modalities using deep learning as the backbone tool ; we propose competitive strategies to address the tasks of cross-modal retrieval, scene-text visual question answering, and attribute learning ; we address various data-related issues like imbalance and learning when not enough data is annotated. These contributions aim to bridge the gap between humans (such as non-expert users) and artificial intelligence to tackle everyday tasks. Our first contribution aims to evaluate the effectiveness of a multimodal system that receives images and text and retrieves relevant multimodal information (cross-modal system). The cross-modal feature allows the formulation of the queries in the form of images or text and retrieves relevant multimodal data. With this approach, we can evaluate the ability of the model to produce effective multimodal representations and to handle any multimodal query with a single model. Subsequently, in our second contribution, we adapt the system to perform a recent task called scene-text visual question answering (ST-VQA). The aim is to teach traditional VQA models to read the text contained in natural images. This task requires us to perform a semantic analysis between the visual content and the textual information contained in associated questions to give the correct answer. We find this task very relevant in the multimodal context since it truly forces us to jointly develop mechanisms that reason about visual and textual content. Our latest contributions point to data-related issues. Data is one of the most important factors in aiming for good performance. Therefore, we determined that a relevant skill is to understand how to properly clean and analyze data and create strategies that can take advantage of it.To evaluate our strategies, we consider the problem of attribute learning. Attribute learning can complement category-level recognition and therefore improve the degree to which machines perceive visual objects. In the first study, we cover two key aspects : imbalance and insufficient labeled data. We propose adaptations to classical imbalanced learning strategies that cannot be directly applied when using multi-attribute deep learning models. In the second study, we propose a novel strategy to exploit class-attribute relationships to learn predictors of attributes in a semi-supervised learning way. Semi-supervised learning permits harnessing the large amounts of unlabelled data available in many use cases in combination with typically smaller sets of labeled data.