The measures were part of proposals for a new Integration Agency launched on Monday by Carl Hagen, the former leader of the Progress Party, for Oslo Municipality.
"There are many good suggestions here," Solveig Horne, Minister of Children, Equality and Social Inclusion, told Norway's NRK channel.:"It should be up to each municipality, but I think Oslo's Progress Party is making important contributions to the integration debate. I shall take this with me in future work."
Marianne Marthinsen, who represents the Oslo Labour Party in the Norwegian Parliament, described the proposals as "absolutely insane".
But Horne appeared to have no problems with the most controversial suggestion: taking children into care if their parents fail to teach them Norwegian.
"I am very happy for children to go into care if there's neglect," she said. "Every parent has a responsibility to ensure that children can speak Norwegian by the time they start school."
Asked whether failing to teach children language constituted neglect, Horne was a little evasive.
"This is something the child welfare agency must consider in individual cases. But being able to speak the language before starting school is very important and all parents must ensure that their children can."
NRK's political commentator Lars Nehru Sand argued that both Hagen's radical proposals and Horne's response to them were a political "trick" to keep the party''s more radical anti-immigrant supporters onside without alienating the majority in Norway.
"The party is still resorting to the tried-and-tested, well-known and occasionally well-functioning political trick of double communication," he said.
"They might get a hearing at the Progress Party's National Congress, but these proposals will never be adopted as policy in Oslo, in the government or by parliament," he added.
He said Progress time and time again has had one of its more radical elements -- Per Sandberg, Christian Tybring-Gjedde, or this week Carl Hagen -- make radical far-Right proposals.
A more moderate figure then nuances the message, rephrasing it in a way more acceptable to the majority without rejecting it outright (as Horne did on Tuesday).
"This tells other voter groups that 'You can still count on us, we are still here for you,'" he argued.