Given an oriented surface bounding a handlebody, we study the subgroup of its mapping class group defined as the intersection of the handlebody group and the second term of the Johnson filtration; A n J2. We introduce two trace-like operators, inspired by Morita's trace, and show that their kernels coincide with the images by the second Johnson homomorphism r2 of J2 and A n J2, respectively. In particular, we answer in the negative a question asked by Levine about an algebraic description of r2(A n J2). By the same techniques, and for a Heegaard surface in S3, we also compute the image by r2 of the intersection of the Goeritz group g with J2.