Abstract The origin and dispersal of the Austronesian language family, one of the largest and most widespread in the world, have long attracted the attention of linguists, archaeologists, and geneticists. Even though there is a growing consensus that Taiwan is the source of the spread of Austronesian languages, little is known about the migration patterns of the early Austronesians who settled in and left Taiwan, i.e. the “Into-Taiwan” and “out-of-Taiwan” events. In particular, the genetic diversity and structure within Taiwan and how this relates to the into-/out-of-Taiwan events are largely unexplored, primarily because most genomic studies have largely utilized data from just two of the 16 recognized Highland Austronesian groups in Taiwan. In this study, we generated the largest genome-wide data set of Taiwanese Austronesians to date, including six Highland groups and one Lowland group from across the island and two Taiwanese Han groups. We identified fine-scale genomic structure in Taiwan, inferred the ancestry profile of the ancestors of Austronesians, and found that the southern Taiwanese Austronesians show excess genetic affinities with the Austronesians outside of Taiwan. Our findings thus shed new light on the Into- and Out-of-Taiwan dispersals.