It should push in snugly and need NO sealant of any kind. Shine a flashlight down there and see what's going on, be sure the dipstick is fully seated.
there is a chance it could be broken too. the end of the dipstick tube is double flanged, so if you pull the housing to the tube off, and only see one flange at the bottom, then it's broken off inside. If that's the case, don't stress. Forgot the name of the tool, but you bang it into the hole and turn it out with a ratchet, and the broken section comes right out.
also, the dipstick is supposed to bolt to one of the header bolts, and if you have aftermarket headers, then it likely is not anymore, however, it should still NOT leak.
