The oil pump is supposed to stay in the block. The shaft is supposed to have a spring washer on it that prevents it from being lifted out when you take the distributor out.
Solution for fixing an oil pump shaft that isn't engaging the pump but didn't fall down into the oil pan.