FORMER Men’s Fitness editor Neal Boulton, who sprinted out of the closet this year when he publicly made out with Jann Wenner outside the Time Warner Center, is being harassed – for acting too straight. “Some p – – – ed-off gay-activist types bombarded me over the weekend as I walked hand in hand with my children. One threatened to call Page Six to ‘out me’ as a straight fraud,” Boulton, who now edits gay-lifestyle magazine Genre, told us. “The other guy barked that I had no right editing a gay magazine because of all the women I slept with. I would have clocked them had my kids not been in tow. Yeah, I did spend the weekend at [my wife] Claire’s, but I do that all the time. I try to get as much time in with the kids and Claire . . . [Bleep], she’s the mother of my children.” Boulton assures us he is quite gay, and we’re inclined to believe him. He recently asked gay-porn king Michael Lucas for a bit part in an XXX video.