Double-stranded DNA viruses of the realm Varidnaviria (formerly PRD1-adenovirus lineage) are characterized by homologous major capsid proteins (MCPs) containing one (kingdom: Helvetiavirae) or two β-barrel domains (kingdom: Bamfordvirae) known as the jelly roll folds. Most of them also share homologous packaging ATPases (pATPases). Remarkably, Varidnaviria infect hosts from the three domains of life, suggesting that these viruses could be very ancient and share a common ancestor. Here, we analyzed the evolutionary history of Varidnaviria based on single and concatenated phylogenies of their MCPs and pATPases. We excluded Adenoviridae from our analysis as their MCPs and pATPases are too divergent. Sphaerolipoviridae, the only family in the kingdom Helvetiavirae, exhibit a complex history: their MCPs are very divergent from those of other Varidnaviria, as expected, but their pATPases groups them with Bamfordvirae. In single and concatenated trees, Bamfordvirae infecting archaea were grouped with those infecting bacteria, in contradiction with the cellular tree of life, whereas those infecting eukaryotes were organized into three monophyletic groups: the Nucleocytoviricota phylum, formerly known as the Nucleo-Cytoplasmic Large DNA Viruses (NCLDVs), Lavidaviridae (virophages) and Polintoviruses. Although our analysis mostly supports the recent classification proposed by the International Committee on Taxonomy of Viruses (ICTV), it also raises questions, such as the validity of the Adenoviridae and Helvetiavirae ranking. Based on our phylogeny, we discuss current hypotheses on the origin and evolution of Varidnaviria and suggest new ones to reconcile the viral and cellular trees.