Is it a knocking or ticking sound? Knocking could be broken parts, worn bearings, slapping pistons. Does it sound like a washing machine? Ticking/tapping could be misadjusted valve train, sticking lifters, bent pushrods, worn valve springs. Is the engine sludged up? What is your oil pressure? How many miles on the motor? Hydraulic or solid lifter? Stock or aftermarker, Pedestal or stud mount rockers?
Use a long piece of wood, pry bar, etc. and place one end on the passenger valve cover and the other to your ear and listen. Move it along the valve cover to try and isolate where the noise is coming from.
Pull the valve cover and inspect the valve train . I doubt it is the rockers and they would get quieter when the car warms up as the oil would flow more.
If it is in the bottom end, do a cylinder balance test followed by a compression/leak down test.