Ben Affleck's chief offense appears to be that he's perceived as the less-talented friend of Matt Damon. Affleck's acting talents have failed to convince many people, as the overwhelmingly negative reception to his casting as 'Batman' proved. In 2017 he confessed that he once inappropriately groped television host Hilarie Burton at an awards ceremony, which didn't do wonders for his reputation either, despite his belated apology for it. Those who vote Republican particularly dislike him for his strong and outspoken support for the Democrats, and for liberal causes in general.
Affleck's liberal views have inspired him to perform many charitable acts, though. His support for the gay and transgender communities has been strong and unwavering for years. Financially, he supports the East Congo Initiative, which attempts to secure better pay and conditions for farmers working within the country. He and Damon also include an 'inclusion rider' in any film they appear in, requiring both casts and crews to be ethnically diverse.