Prince Andrew, 59, said he had asked the Queen for permission to withdraw for the "foreseeable future".
He said he deeply sympathised with sex offender Epstein's victims and everyone who "wants some form of closure".
The duke has faced a growing backlash following a BBC interview about his friendship with the US financier.
Companies he has links with, such as BT and Barclays, have joined universities and charities in distancing themselves from him.
For several months the duke had been facing questions over his ties to Epstein, who took his own life in August while awaiting trial on sex-trafficking charges.
Virginia Giuffre, one of Epstein's accusers, claimed she was forced to have sex with the prince three times. The duke has always denied any form of sexual contact or relationship with her.
His latest move, described by Buckingham Palace as "a personal decision", was taken following discussions with the Queen and Prince Charles.