How often you planning to replace this seal? By removing the cover and replacing the seal, you solve the problem for $25. If you replace the cover to use the "easier" seal, this goes up another $100 or so. I dunno about you, but I replaced this seal over 20 years ago and it's still in there, doing the job. Not untouched, either, I've replaced the damper twice.
I really wouldnt worry about that happening. The oil has to pass through a screen and a filter before it even comes close to the inside of the lifters. The normal breakdown of bearings ending up in the bottom of the oil pan will do more damage than some soft silicone that may end up in your oil pan.