From what I have gathered, the lifters are pretty stout, being rollers and all. So you probably will not need to change those for a long time. Just inspect them while you are in there and get a set if you find one or more are worn. On the other hand, there will never be an easier time to replace them, so if you have an extra $200 to $250 burning a hole in your pocket, then go ahead and replace them. Your call.
The timing chain and pushrods are a different story. Both are fairly inexpensive to replace with new, and also are significantly lower durability than the lifters. So I would just plan on swapping those.
On the pushrods, isn't selecting the optimum pushrod length an issue when swapping heads and cam? It seems like you might need a change for that regardless. (I bought the TFS kit so I wouldn't have to worry about issues like selecting the right length pushrods.)