Kristoff held off Peter Sagan in an exciting battle to the finish line, winning the 12th stage in 4 hours, 32 minutes, 11 seconds. The route was the challenging 185.5km Bourg-en-Bresse to Saint Etienne.
It was the Katusha's rider first Tour stage victory and he finished only a half-metre ahead of the Slovakian Sagan. French champion Arnaud Demare was third.
Kristoff was clearly delighted with his maiden win and said after the race to TV2: "This is a terrific feeling. I have dreamt of this since I was a child. I have been close to victory so many times now. It feels a bit unreal. I almost cannot comprehend that I managed it. But it is incredibly great to finally manage it."
"This is such a big ride. This stage victory means a lot. I guess this is my greatest victory."
The Norwegian's win did not change the overall leaderboard however. Astana's Vincenzo Nibali retained the yellow jersey. The rider is two minutes, 23 seconds ahead of Team Sky's Richie Porte.
But tonight Team Katyusha will celebrate their sprint star Alexander Kristoff.