Would it be worth it?

For a N/A 306 a 65 MM TB is fine. Putting anything bigger wouldn't give you a big boost in power. If you have pressurized induction, or a stroker or big cube engine, that could benefit from a larger TB.