Rattling Noise where Seatbelt is

A rattling noise is coming from where the passenger seatbelt is. I'm guessing it is simply a loose screw. I've removed an interior panel to get to it but discovered that the noise is coming from further up behind the panel above. This panel is where the seatbelt attachment goes through. For me to remove this panel I have to detach the seatbelt attachment. I've tried levering the seatbelt attachment apart with a screwdriver but haven't succeeded. I fear if I try more forcefully or proceed carelessly that I could damage things. So, I wonder if anyone here knows how to detach the seltbelt attachment in order to remove the panel? Most probably it's as easy as changing a lightbulb.