Bishop Bernt Eidsvig told VG newspaper that the priest, who is not Norwegian, was "distraught" over what had happened during "a short period" this summer.
"He admits to having had a sexual relationship with the girl. It lasted a little over a week," he said.
Police in Bergen on Tuesday condemned the church for making the case public before they had had an opportunity to investigate it.
"We would have liked to talk to the victim before the case became known in the media," Rigmor Isehaug of the Hordaland county police, told VG. "We think it is very sad that the church has alerted the media before the police has started its investigation."
The Catholic church in Norway was rocked by scandal in 2010 when it emerged that a Georg Müller, a former Bishop of Trondheim, had sexually abused an altar boy at the start of the 1990s.
The Church had been made aware of the incident, but had sought to hide it from the authorities.