The 55-year-old mullah, whose real name is Najmeddine Faraj Ahmad, shouted “long live Islam” as he was led away to a waiting police vehicle.
The founder of the Ansar al-Islam group, Krekar has lived in Norway since 1991.
The Norwegian Security police, PST, said they wished to question Krekar about threats he is alleged to have made last weekend against Norwegian interests abroad, including police officers, on the paltalk.com web forum.
Police also searched his apartment in the Tøyen district in the early afternoon raid.
“Yesterday’s verdict is an important factor in his arrest, partly since it describes an established threat situation,” prosecutor Marit Bakkevig told news agency NTB.
Bakkevig said she was in dialogue with the police during the day was aware of the planned arrest.
“To the extent that there have been new criminal offenses, they must start in the district court in the usual manner.”
Krekar also looks likely to face more charges after he grabbed a microphone and lashed out at a camera when refusing to answer questions put to him by journalists at Norway’s TV 2 on Monday evening.
“It is important for us to report matters like this when our journalists are physically attacked or our equipment is destroyed,” said TV 2’s news editor Niklas Lysvåg.
An Oslo court on Monday sentenced Krekar to five years in jail for issuing death threats against a former government minister and others.
He was found guilty of threatening the life of Erna Solberg, an ex-minister who signed his expulsion order in 2003 because he was considered a threat to national security.
"Norway will pay a heavy price for my death," he said during a meeting with international media in June 2010.
"If for example Erna Solberg deports me and I die as a result, she will suffer the same fate," he said in Arabic.
"I don't know who will kill her: Al-Qaeda, Ansar al-Islam, my family, my children. I don't know… But she will pay the price."
The three Oslo court judges said the threat was aimed at pushing Norwegian authorities to retract the expulsion order.
Krekar, whose name is on terrorist lists drawn up by the United Nations and the United States, has avoided expulsion since the order was signed nine years ago since Norwegian law prevents him from being deported to Iraq until his safety can be guaranteed and as long as he risks the death penalty there.
Krekar was also accused of threatening three other Kurds living in Norway who had burned pages of the Koran or insulted it in another way.
The mullah has admitted to making the statements but has claimed his words merely referred to Islamic principles.
"The court does not reject sharia but, in the context of Norwegian law, sharia does not have a place," the Oslo court said in its verdict, referring to Islamic law.
Although the court cleared Krekar of charges he had called for the killing of American soldiers in Iraq, it granted the prosecution its request that he serve five years in prison.
The defence, which had demanded Krekar's release, appealed the verdict on the spot.
"The comments made by the accused were not threats nor terrorist acts and we think the court's interpretation is wrong," defence lawyer Brynjar Meling told AFP.
While Krekar acknowledges having co-founded Ansar al-Islam, which also figures on international lists of terrorist groups, in 2001, he insists he has not led the group since 2002.