We implement an experimental architecture in which a single atom of K is trapped in an optical tweezer, and is immersed in a bath of Rb atoms at ultralow temperatures. In this regime, the motion of the single trapped atom is confined to the lowest quantum vibrational levels. This realizes an elementary and fully controllable quantum impurity system. For the trapping of the K atom, we use a species-selective dipole potential, that allows us to independently manipulate the quantum impurity and the bath. We concentrate on the characterization and control of the interactions between the two subsystems. To this end, we perform Feshbach spectroscopy, detecting several inter-dimensional confinement-induced Feshbach resonances for the KRb interspecies scattering length, that parametrizes the strength of the interactions. We compare our data to a theory for inter-dimensional scattering, finding good agreement. Notably, we also detect a series of p-wave resonances stemming from the underlying free-space s-wave interactions. We further determine how the resonances behave as the temperature of the bath and the dimensionality of the interactions change. Additionally, we are able to screen the quantum impurity from the bath by finely tuning the wavelength of the light that produces the optical tweezer, providing us with a new effective tool to control and minimize the interactions. Our results open a range of new possibilities in quantum simulations of quantum impurity models, quantum information, and quantum thermodynamics, where the interactions between a quantized system and the bath is a powerful yet largely underutilized resource.