It sounds like the bung is there. Like you said, if it looked like someone just drilled a hole in the header, then it wouldn't be there. The bung is just the metal ring thats threaded, and welded onto or in the header. It provides structural support for the O2 sensor. THere is not enough metal in the header to hold the O2 sensor without the bung.
You can get the threads cut, or tap it yourself. Just make sure you get the right tap. Let us know what the vendor says. Sounds like that didn't tap the threads in when they were built.