He died on Wednesday from heart complications at a hospital in Dar es Salaam, Samia Suluhu Hassan said in an address on state television.
Magufuli has not been seen in public for more than two weeks, and rumours have been circulating about his health.
Opposition politicians said earlier this week that he had contracted Covid-19, but this has not been confirmed.
"It is with deep regret that I inform you that today... we lost our brave leader, the president of the Republic of Tanzania, John Pombe Magufuli," Vice-President Hassan said.
She said there would be 14 days of national mourning and flags would fly at half mast.
Magufuli was declared president on his 56th birthday in October 2015.
He was one of Africa's most prominent coronavirus sceptics, and called for prayers and herbal-infused steam therapy to counter the virus.
Tanzania has not published details of its coronavirus cases since May, and the government has refused to purchase vaccines.
Magufuli was last seen in public on 27 February, but Prime Minister Kassim Majaliwa insisted last week that the president was "healthy and working hard".
On Monday, police said they had arrested four people on suspicion of spreading rumours on social media that the president was ill.
John Magufuli at a glance
Born in Chato, north-west Tanzania, in 1959
Studied chemistry and maths at the University of Dar es Salaam
Worked as a chemistry and maths teacher
First elected as an MP in 1995
Became a cabinet minister in 2000
First elected president in 2015
Won second term in 2020