The 82 year old had been admitted to hospital on 13 August and passed away last night.
Dr Williams' death is the second this week associated to the latest Auckland outbreak of the coronavirus.
In a statement, the Ministry of Health said Dr Williams' death brings the total number of people who have died from the coronavirus in New Zealand to 24.
The Pasifika Medical Association announced his passing, saying he was a well-respected and much loved man.
Dr Williams spent 25 years in the Cook Islands and served as a Cabinet Minister between 1974 and 1978 and again between 1994 1996 before becoming Prime Minister in 1999.
Current Cook Islands Prime Minister Henry Puna said Dr Williams' passing had caused him great sadness.
"Dr Joe was a pioneer on many fronts and a man way beyond his time. He was one of our early breed of home-grown medical officers of health.
"For thousands of our people in New Zealand, his medical practice was where they headed for their primary health care," Puna said.
The prime minister said Dr Williams would be greatly missed.
"E tumu rakau ruperupe teia no te Basileia kua inga.
"On behalf of the Government and people of the Cook Islands I extend our heart-felt condolences to Dr William's wife Jill and family."
A National Memorial Service is to be organised for Dr Williams once his funeral arrangements have been confirmed
As a sign of respect and remembrance all Cook Islands flags on government buildings were being lowered to fly at half-mast.
Dr Williams was born in Aitutaki, went to Northland College, and graduated from Otago Medical School in 1960, later completing a Master in Public Health at the University of Hawaii.
He was involved with the World Health Organisation and in 2016 he received the WHO Award of Appreciation for his role in the elimination of Lymphatic Filariasis. He was also known for his work in the fields of eczema, prostate cancer and diabetes.
Most recently he had been practising medicine in Auckland.
He received the Queen's Service medal in 1974 and was invested with the Companion Queens Service Order in 2011 for services to the Cook Islands community.
He was awarded a Life membership by New Zealand's Pasifika Medical Association in 2004 and was appointed as Patron of the PMA in 2015.
PMA President, Kiki Maoate, said the death of Dr Williams had left the community feeling empty and distraught.
Dr Maoate, who is also a nephew of Dr Williams, said the immediate family was devastated and that feeling of sorrow had spread through the community.
"There will be a sense of emptiness, there will be a sense of deep sorrow as we go through this period but I think at the end of the day we will look an see what he has actually touched is still there and we can carry the good work that he has actually started but he will leave a lot of distraught people for some time."
He said Dr Williams was driven, in particular, by a love for his Cook Island community.
Dr Maoate said the former prime minister was determined to serve his community, even well into his advanced years.
"There are other people communities that he plays a significant role [in] but first and foremost it's about the people for him and that's where his legacy really stands out.
"That's why, I think when you reflect back on what he has developed and processed through the years and achieved, you can actually trace it back to his heritage and his love for the people of the Cook Islands," Dr Maoate said.
Director-General of Health Dr Ashley Bloomfield today said Dr Williams was seen as a leading figure in the Cook Islands medical community and would be sadly missed.
"Our thoughts are with his family and community at this time of loss and grief.
"Today's sad news again reinforces the importance of our shared vigilance against Covid-19, the very serious consequences the virus can carry with it, and the measures we all need to take to stop the spread, break any chain of transmission and prevent deaths."