We consider the problem of designing adapted kernels for approximating functions invariant under a known finite group action. We introduce the class of argumentwise invariant kernels, and show that they characterize centered square-integrable random fields with invariant paths, as well as Reproducing Kernel Hilbert Spaces of invariant functions. Two subclasses of argumentwise kernels are considered, involving a fundamental domain or a double sum over orbits. We then derive invariance properties for Kriging and conditional simulation based on argumentwise invariant kernels. The applicability and advantages of argumentwise invariant kernels are demonstrated on several examples, including a symmetric function from the reliability literature.