Renormalizable nonanticommutative SYM theories with chiral matter in the adjoint representation of the gauge group have been recently constructed in [arXiv:0901.3094]. In the present paper we focus on the U*(1) case with matter interacting through a cubic superpotential. For a single flavor, in a superspace setup and manifest background covariant approach we perform the complete one-loop renormalization and compute the beta-functions for all couplings appearing in the action. We then generalize the calculation to the case of SU(3) flavor matter with a cubic superpotential viewed as a nontrivial NAC generalization of the ordinary abelian N=4 SYM and its marginal deformations. We find that, as in the ordinary commutative case, the NAC N=4 theory is one-loop finite. We provide general arguments in support of all-loop finiteness. Instead, deforming the superpotential by marginal operators gives rise to beta-functions which are in general non-vanishing. We study the spectrum of fixed points and the RG flows. We find that nonanticommutativity always makes the fixed points unstable.