Edward Dawes, a Norwegian-British-Indian politician who serves as a deputy councillor for the Labour Party in Eidsvoll, made the comments on Saturday night.
"If homosexuals demand the right to get married in church, then they should form their own denomination," he said.
On Twitter on Monday night he repeated his argument, refusing to back down despite the growing discomfort of this party.
"Historically speaking, anyone who has not been in agreement with Christianity has started his own church," he said.
Later, when David Cameron tweeted an Easter message to the UK, Dawes tweeted in reply, "Glad-u stood firm. My Party went on Telle threatning to kick me out for telling gays that they could start their own church."
During the Twitter exchange Dawes also tried to back up his assertion that gay people were more likely to commit child abuse.
Labour MP Sverre Myrli told NRK that Dawes' opinions could well sit outside what the party was able to tolerate.