Doubly polar molecules, possessing an electric dipole moment and a magnetic dipole moment, can strongly couple to both an external electric field and a magnetic field, providing unique opportunities to exert full control of the system quantum state at ultracold temperatures. We propose a method for creating a purely long-range doubly polar homonuclear molecule from a pair of strongly magnetic lanthanide atoms, one atom being in its ground level and the other in a superposition of quasi-degenerate opposite-parity excited levels [Phys. Rev. Lett. 121, 063201 (2018)]. The electric dipole moment is induced by coupling the excited levels with an external electric field. We derive the general expression of the long-range, Stark, and Zeeman interaction energies in the properly sym-metrized and fully-coupled basis describing the diatomic complex. Taking the example of holmium, our calculations predict shallow long-range wells in the potential energy curves that may support vibrational levels accessible by direct photoassociation from pairs of ground-level atoms.