A a larger fella, I don't tend to think one way or the other. I am not a fan of the extremes especially when they involve poor health. At 6'5 and 280...at this height the smallest I had been was 222 when I was in great shape. I am most confident at...250 to 260 (my "playing" weight). 222 was too small, I was judging myself based on my skinny face etc. Anyways thats besides the point.
past that, its all about the confidence. If you have it, people see it. If you don't...people will look right past you. Find what works for you and work towards it.