Rear Main Seals and retainers

Hi everyone.

I have a 4EFE EE101L Corolla, and I'm considering doing the rear main seal. I've always replaced rear main seals by prying it out with a screwdriver and driving it in carefully. I usually do it when I replace the clutch because I don't want to disassemble the whole gearbox again. But then I got thinking, can a leak develop at the retainer, where it meets the block? Should I take out the retainer and then reseal the surface behind it?

Thanks in advance!