Bhatti has appealed the decision of the authorities to hold him in custody, reports broadcaster NRK.
Oslo District Court believes that there are several grounds on which to suspect that Bhatti may attempt to destroy evidence should he be released, according to the report.
Several weapons were found in Oslo Saturday, but police declined to give further details of the circumstances of the discovery.
“Bhatti's name was quickly connected to the case and it was necessary for us to arrest him,” police lawyer Anne Cathrine Aga told NRK.
Bhatti was arrested Tuesday evening and is suspected of illegal involvement with firearms.
The Islamist, who has Pakistani parents, was born and brought up in Oslo, where he joined the Young Guns gang in the early 1990s. He later became radicalised, and was jailed for his role in shots fired at an Oslo synagogue in 2006.
After being released from prison in Pakistan in 2013, Bhatti warned Norway against joining the US-led coalition against the Islamic State terror group in 2014.
His stepson was last year reported to have been murdered by a rival, who was himself then executed by Isis for the crime.
The court also upheld Wednesday a police request that he be denied visits and letters during the custodial period.
“A number of weapons and ammunition have been confiscated.
“There are a number of statements and reports that connect the accused [Bhatti, ed.] with these weapons and show that the accused has been in contact with co-accused in the relevant period of time,” district court judge Jørgen Monn said in the custodial order.
Aga declined to confirm to NRK the nature of the connection between Bhatti and other individuals connected to the confiscation of the weapons.
Two others have been arrested, of which one remains in custody while the other has been released, reports NRK.
“We are engaged in an active investigation and several people are involved. One individual is already remanded in connection with the case, with interference with evidence also our focus here,” Aga told NRK.
Bhatti denies the accusation and any knowledge of the case, and has appealed against the decision to remand him in custody.
“He has been arrested [previously] several times just before [Norway's national day] May 17th, and if police wanted him out of Oslo before May 17th, they could have just asked him to leave,” defence advocate Bernt Heiberg said according to NRK.
Aga rejected claims the arrest was connected to May 17th.
“I can see that Bhatti believes the arrest is political, but this is a completely normal criminal case. We arrest criminals all year round,” Aga said.
Details of proceedings in court are protected from publication and Aga declined to confirm what police believe Bhatti's exact connection to the case is.
But the suspect refused to give blood samples, fingerprints, have his photograph taken or respond to police questions, saying he has provided police with these on a previous occasion, reports NRK.
He has several previous convictions for threatening behaviour, extortion and violence.
Aga said that police would go through the courts in order to secure the samples they need to continue their investigation.