This sounds like your timing chain tensioner. Google for "corolla timing chain tensioner" and click images - does it look like these?
I'm assuming you have a 1998 Corolla, from your username.
This is a common issue with 8th generation Corollas. It is very easy to fix and you can either buy a new gasket (o-ring) at a dealership or buy a whole new tensioner assembly at Rockauto (maybe $15-20).
If you can post a picture, that would help narrow it down.