Police in the capital are seeking to have the suspect extradited to Norway to stand trial for the attacks, which took place earlier this year and last summer.
A 77-year-old woman in Korsvoll was left fighting for her life after she was attacked and sexually assaulted in February this year.
The suspect has also been charged with the rape of an 87-year-old woman in Etterstad last July.
At the same time police were investigating the two rape cases, the 35-year-old was being held in Østfold for less serious offences.
Police took DNA samples from the man just before his release. He left the country before officials had time to analyze the tests.
“Thanks to technical evidence found at the scenes, we were able to create a DNA profile that turned out to match this man. National and international arrest warrants were put out for him in July,” police inspector Hanne Kristin Rohde told newspaper Aftenposten.