The camshaft cap is/was mounted the wrong way, that's why it worked it's way out.
If you look in the Repair manual (for the 2005-2007) at page E5 you'll find this little note.
NOTE: the collar (5) of the cap must engage in the cylinder head groove.
So you'll be good to go if you just mount it the other way around. :thumbup:
/Nicklas