Federer, 39, has suffered a setback with an existing knee injury.
The Swiss legend won doubles gold at Beijing 2008 and silver in 2012, losing the singles final to Andy Murray.
Konta, 30, has contracted Covid after withdrawing from Wimbledon when a member of her team tested positive and she was identified as a close contact.
Federer, now ranked number nine in the world, had two knee surgeries in 2020.
In the past two years he has seen his Grand Slam tally equalled by Spain's Rafael Nadal at the 2020 French Open and most recently by Serbia's Novak Djokovic - who won a 20th major at Wimbledon 2021.
After losing to Poland's Hubert Hurkacz in the quarter-finals of Wimbledon Federer told reporters he was unsure whether he would play at SW19 again.
Konta pulled out of Wimbledon before it started as she was forced to withdraw and isolate when a member of her team tested positive for Covid.
She says she then developed symptoms and has now tested positive, adding that she has been unable to train for the past two and a half weeks and felt she would not be ready to compete in Tokyo.
"This is a heartbreaking reality for me, as representing Team GB at the Olympic Games in Rio in 2016 is one of my most treasured memories from my career," Konta said on social media.
"I'll be doing my best to get back to full health and fitness and ready to be competing again soon.
"During this time, I'll also be glued to my TV cheering on my fellow Team GB athletes and hope you will all join me."
Konta has only been able to play 15 matches this year because of injury and Covid-related constraints.
Team GB is not able to name a replacement for Konta, as no other British players are ranked highly enough to qualify, so her withdrawal deprives Heather Watson of a doubles partner.
The British number two will be limited to singles unless she and Joe Salisbury make the cut for the mixed doubles. The entry list will be confirmed once the Games are underway.
Dan Evans, Andy Murray and the Wimbledon mixed doubles champion Neal Skupski complete the GB team for the tennis, which starts on 24 July.