The agreement signed by the United States, Russia, Canada, Denmark on behalf of Greenland, and Norway prohibits commercial fishing in a 2.8 million square kilometre (1.1 million square mile) area -- five times the size of France -- in the absence of international regulation.
The agreement comes amid fears the waters risk being targeted by commercial operators in future.
"Commercial fishing in the international part of the Arctic Ocean is not likely in the near future," Norway's ministry of fisheries said in a statement. "But developments must be followed closely and the coastal states therefore agree on establishing a (framework for) research cooperation," it said.
In 2012, more than 2,000 scientists called for an international fisheries agreement to protect the waters, where a number of studies have predicted sea ice could disappear completely over the summer in the coming decades.
According to estimates cited by the US-based Pew Research Center, 40 percent of the international part of the Arctic Ocean, also known as the Central Arctic Ocean, was ice-free in September 2012 when the ice melted to a record low.
International environmental group Greenpeace said it "welcomed the move as a small step towards Arctic protection but lamented the failure to make the deal permanent."
"It's clear that most of these countries are motivated by resource extraction, not protection, and see the melting of the ice sheet as an opportunity to fish further north," spokeswoman Sophie Allain said.
The Central Arctic Ocean is surrounded by so-called economic zones belonging to the five countries that signed the agreement. A small part of it is already regulated by the North East Atlantic Fisheries Commission.