I think it can be a variety of factors...but by far I see that it often comes down to some men dipping into the "settling" factor and/or jumping on the bandwagon. I don't think most people take time to truly realize what they are attracted to. True attractions may lie dormant as they are hardwired still, so they are recognized on some level, but nurture might have some men look over that for the most part due to social pressure to like what's deemed "understandable", expected and attractive.
Besides, beauty is not only a measure of a woman's level of "worth" to a large degree when you really dissect the issue, but men see this as well and want to be with a woman who is deemed powerful in her "feminine right" which is being attractive or sexually appealing to many. In return, that makes the man feel more masculine for what he can "accomplish" or retrieve. It shows that he’s got the goods to get a woman of that “caliber” so to speak.
As a result, that influences some men to "go for the gold" although they might not truly prefer that while not being flat out turned off by the conventional. This might explain why some men will find a larger woman (either to a slight or very generous degree) attractive and truly like and want to be with her but attempt to hide her around friends or deny the authenticity of the relationship. He starts to feel that showing an attraction to a larger woman says to society that he can’t do any better, thus he doesn’t “have the goods” to attract a powerful woman which in returns again validates his masculinity.
People also often expect certain types of people to be together. It is an eyesore when an "unlikely" pair is seen. People can't seem to shake being distracted or judgmental of that...and most won't even remotely attempt to become aware and analyze their actions.
It comes back down to the need to be liked and accepted...not marginalized. Here it takes on this form in a very complex but disturbing manner.