The priest was fired on twice at around 4pm as he was closing the church, and he was being treated on site for life-threatening injuries, the source said.
Witnesses said the church was Greek Orthodox. Another police source said the priest was of Greek nationality, and had been able to tell emergency services as they arrived that he had not recognised his assailant.
The incident came two days after a man shouting "Allahu Akbar!" (God is Greatest) beheaded a woman and killed two other people in a church in Nice. Two weeks ago, a schoolteacher in a Paris suburb was beheaded by an 18-year-old attacker who was apparently incensed by the teacher showing a cartoon of the Prophet Mohammad during a class.
While the motive for the latest attack was not known, government ministers had warned that there could be other Islamist militant attacks. President Emmanuel Macron has deployed thousands of soldiers to protect sites such as places of worship and schools.
Prime Minister Jean Castex, who was visiting Rouen, said he was heading back to Paris to assess the situation.
The Nice attack took place on the day Muslims celebrate the Prophet Mohammad's birthday. Many Muslims around the world have been angered about France's defence of the right to publish cartoons depicting the Prophet.
A third person has been taken into police custody in connection with that attack, a police source said yesterday. The suspected assailant was shot by police and remains in a critical condition in hospital.
Macron took to Arabic-language airwaves yesterday, saying he understood the publication of cartoons depicting the Prophet Mohammad may shock some people but that there was no justification for acts of violence.
In an interview with Al Jazeera released, Macron said his position had been misconstrued: that he never supported publication of cartoons seen as insulting by Muslims, but had defended the right of free expression.
"I understand and I respect the fact that people might be shocked by these caricatures, but I will never accept any justification for acts of violence over these caricatures," Macron said.
The teacher killed on 16 October, Samuel Paty, had showed cartoons in class to prompt discussion of free speech.