I to think you have already made up your mind, but here is my answer anyway.
Size and strength do matter, but not to the extent that most people think. They definetly matter in a cage fight, or even in a street fight, but they are not as important for self defense.
They matter in the UFC because you have two highly trained individuals, and there is a saying that a good big man will beat a good small man.
However if you have a big guy with absolutly no training or very little, and a smaller person that is well trained, the size factor becomes much smaller.
Scenario: A big bad guy grabs hold of a smaller women to show her who's boss. Using her 10 years of training she immediatly delivers a palm heal to his chin, knee strike to the inner thigh or groin, elbow smash to the jaw, and then a hammerfist to the throat, followed by getting away and surviving, which is the ultimate goal of self defense. I have been lifted off of my feet ( I am 6" 2" and then was 110 lbs) by a palm heal under the chin by a lady 5' 4" and 120 lbs when I charged her to grab her.
Point is that yes all things being equal the bigger person has the advantage, but training will help negate some of that advantage.
Just my opinion from 18 years of training.
BTW to the poster of the question: How long have you trained and in what?