the friction modifier is for the clutch plates only-it keeps them from chattering around corners, it has nothing to do with quieting the gears
like srothfuss said, the gear noise is more than likely backlash setting, or part of the break-in.
let me add a question, what brand gears did you choose, FRPP's have a wonderful history of being very quiet if installed and broken in properly, I've also heard good things about motive from local shops who install gears regularly. Richmond on the other hand, has a long history of being a noisy gearset no matter how well it's set up...i do know that one from experience
Also, did you do a proper break-in? proper break-in consists of approximately 300 miles with periods of continuous use not to exceed 45min or 50mph....basically city driving.
another measurement which will make some noise is pinion depth, instead of backlash where there is too much or too little "play" between the pinion and ring gear teeth, pinion depth is too shallow or too deep gear engagment....but it can make a similar noise.
if the noise gets any worse within your next two tanks of gas, i'd take it back, and have whoever installed them, ck all the tolerances (backlash, pinion depth, and carrier, and pinion bearing preload)
good luck!