Per Sandberg, who stood down as the party's deputy leader last month, said he had been battered by attacks from within his own party.
"The party has changed. Before we were the party of the man in the street. The party that fought for those who needed us. We were clear in our message," the politician writes in his new book, "Against my will", which is to be published on Wednesday.
Sandberg, one of Norway's most outspoken critics of Islam and immigration, claims that his party has weakened its message in recent years, and that he, personally has come under severe attack.
"Most of the criticism I have received about my political initiatives has come from my own party colleagues, not our political opponents," he writes. "When I also have to look over my shoulder and… be called an idiot, disloyal, a loose cannon, or incompetent, then it has gone too far."
In particular, the book attacks the party's leader Siv Jensen, who Sandberg accuses of "stabbing him in the back" several times.
Sandberg argues that Jensen went seriously wrong after the 2011 terror attacks mounted by Anders Breivik, a former party member, and should never have stated publicly that the party needed to tone down its rhetoric on Islam and immigration.
In particular, he argues, Jensen failed to defend her party colleague Christian Tybring-Gjedde when he came under attack for a 2010 opinion piece which attacked multiculturalism in Norway in much the same terms as Breivik did in his manifesto.
Sandberg argues that Siv Jensen is temperamentally unsuited to lead such an individualist party as the Progress Party.
"Siv is very concerned about consensus, she will not have conflicts or contradictions," he writes. "As leader of a party of strong individualists who only think of themselves, it doesn't work."
When Norway's VG newspaper published a story in 2006 accusing him of being drunk during a speech in parliament, it hit him so hard that he considered suicide, he discloses.
"I had arrived at the end of a cul de sac and found no way out. Everything went dark, as if someone had taken out the fuse, and I went into a trance."
According to the book he was ready to "be done with the whole mess."