Leonardo (1968-present), published by MIT Press, is the leading international peer-reviewed publication on the relationship between art, science and technology, making it an ideal dataset to analyze the emergence of such complex collaborations over time. To identify and analyze both the visible and latent interaction patterns, the research employs different granularities of data (article texts, images, publication dates, authors, their places of affiliation and disciplines) as part of a multimodal approach. Using a convolutional neural network, we examined the features of the images to analyze the modes of representing (and actually doing) art, science or engineering. We paired these features with information extracted using text mining to examine the relationships between the visual and the textual over time.