The Conservative party and its allies won over voters with promises of tax cuts, better health and elderly care, and stricter rules on immigration.
With 99.7 percent of the votes counted, the four right-wing parties were projected to win 96 seats, compared to 72 won by the out-going centre-left coalition, well beyond the 85 needed to secure a majority in parliament.
"Today you have helped to make history!" Conservative leader Erna Solberg said as she took the stage at half past eleven. "Now it's our job to really do what we have promised."
Stoltenberg conceded defeat shortly before Solberg took the stage, saying he would resign as Prime Minister after presenting the next budget.
"The result must be viewed in light of the fact that we have been eight years in government," he said. It was, he said, "a very respectable" result, with
Labour remaining the biggest party in parliament, with some 54 seats and 30.4 percent of the vote.
While Solberg was gracious in victory, thanking the outgoing government, Siv Jensen, the leader of the populist Progress Party, began her speech with the words "Bye Jens" to the rapturous applause of her supporters.
She hinted that she might be willing to soften some of her party's hardline policies on immigration to achieve a coalition across the four right-wing parties.
"We must be tough bargainers, but we must also be realistic bargainers," she said.
Progress was set to receive 30 seats according to the projections, down 11 seats from what it received in the last election in 2009.
Election analyst Johan Giertsen said that the preliminary figures indicated that the Conservative party would not be able to rule with just the Progress Party, meaning they would have to bring in either the Christian Democrat or Liberal Parties, both of whom earlier in the election expressed unwillingness to enter a coalition with the Progress Party.
"Based on these figures, the Conservative Party and the Progress Party probably now need support from one or more other parties to form a new government," he told the NTB newswire.
Bernt Aardahl, professor of politics at the University of Oslo said before the poll that the outcome of coalition negotiations would be unpredictable.
"I think the negotiations after the elections will be tough, and it is still an open question over whether it will be a four-party government or a two-party government," he said. "If the Progress Party will be willing to modify their anti-immigration position in exchange for other policy goals that they have, that might simplify negotiations."
Outgoing Prime Minister Jens Stoltenberg was lauded for his leadership following the twin terrorist attacks mounted by far-Right terrorist Anders Breivik in 2011, and his government has also been commended for its success in bringing Norway through the global financial crisis.
Here is Statistics Norway's projection of the result, based on 99.7 percent of the vote counted.
Conservatives: 26.8, 9.6 (48 seats)
Progress: 16.3, -6.7 (29 seats)
Christian Democrats: 5.6, unchanged (10 seats)
Liberals: 5.3, 1.4 (9 seats)
Labour: 30.9, -4.5 (54 seats)
Centre: 5.5, -0.7, (10 seats)
Socialist Left: 4.1, -2.1 (7 seats)
Green: 2.8, 2.4 (1 seat)