Don't listen to them. If you can afford a good car, get it. But don't go crazy! I would suggest you buy an older one (ie the Mk1 version not the outgoing Mk2). It is an excellent car and at the time represented the best of what Audi could offer in the sports car sector. The engine output I would suggest is the 1.8 Turbo, mated with the Quattro all-wheel-drive system. This car handles like a go-cart (note: that's a good thing) but also provides its passengers with exceptional luxuries, thus maintaining the high level of quality materials that Audi uses in its cars, especially in its interior. Also, the gas consumption is very good with numbers around 30 mpg for the highway. Also, the safety is not as good as that in a A4 of that time, but definitely beat the safety of the other competitors' cars, such as the BMW Z3 and Mercedes-Benz SLK.
Now, I must add that you would be better off with the coupe. The Roadster is a little risky due to its low height, something that can be very scary when you're driving next to a semi-truck.
It is up to you. I hope I helped. Good luck with your purchase.