My grandmother lived in the country, and as a result, there were lots of cats around, because people kept dropping (abandoning) them there. It was pretty common for the cats to bring small animals to the house. It's the cat's way of sharing with your or helping provide for who it considers its family. We had a cat that brought small animals to us, but didn't kill them. She'd just come set them loose in the house! Haha.
Anyway, your feral kitty is trying to do something nice for you. Personally, I would wait for her to be out of sight, and then bury the poor little creatures. Good luck to you and kitty!