A good big man will beat a good little man. Not 100% of the time, but probably 96% or better of the time.
As for your proposition that a no-rules contest means you can attack weak points, don't you think the big guys can do that, too? Check out Jon Hess vs. Andy Anderson from UFC 5. Hess, a much larger fighter, used eye-gouges on Anderson on his way to winning the match.
When skill levels reach equilibrium, other factors, like conditioning, size, power, speed, etc. become more of an issue. It's practically a mathematical equation. Two factors are equal (skill), but one guy has more of something (size, strength), the equation is tilted in his favor.
All things being equal, bigger guys hit harder (more mass), are more difficult to take down (more weight to move around), more difficult to maneuver in the clinch (again, more mass), and are more dominant on top on the ground (once again, it's hard to bench-press someone 30% larger than you). Do you really think Brock Lesnar is more technical than Randy Couture? He won in large part because, well, he's large.