Understanding the irregular crowdmovement and social activities caused by urban events such as city festivals and concerts can benefit event management and city planning. Although various urban data can be exploited to detect such irregularities, the crowd mobility data (e.g., bike trip records) are usually in a mixed state with several basic patterns (e.g., eating, working, and recreation), making it difficult to separate concurrent events happening in the same region. The social activity data (e.g., social network check-ins) are usually oversparse, hindering the fine-grained characterization of urban events. In this paper, we propose a tensor cofactorization-based data fusion framework for fine-grained urban event detection and characterization leveraging crowd mobility data and social activity data. First, we adopt a nonnegative tensor cofactorization approach to decompose the crowd mobility tensor into several basic patterns, with the help of the auxiliary social activity tensor.We then use a multivariate-outlier-detectionbased method to identify irregularities from the decomposed basic patterns and aggregate them to detect and characterize the associated urban events.We evaluate the performance of our framework using real-world bike trip data and check-in data from New York City and Washington, DC, respectively. Results show that by fusing the two types of urban data, our method achieves fine-grained urban event detection and characterization in both cities and consistently outperforms the baselines