Each of you buy a bag of balloons and some candy, write some sweet notes to each other etc, and put these things inside the balloons. Fill them with air and proceed to scatter them all around the room. At midnight, pop them and enjoy your partners little gifts to you. This is inexpensive and a lot of fun! (Just make sure you don't both get the same kind of balloons, might be hard to tell what is for who)
Depends on where you live and what the weather will be like, but stargazing or a bonfire is always nice. I live where it's cold, so we snuggle up under warm blankets together and drink hot cocoa.
Buy dessert for each other and cuddle up with your treats and a good movie.
Sometimes its nice just to get pretty for someone else. Dress up a little more than usual to make the evening a little more special.
Personally I've never been one for going out on New years eve. There are so many things you can do staying at home that can be so much more meaningful. :)