The former Wall Street financier had drawn criticism after calling a reporter to give a profanity-laced tirade against his colleagues.
Mr Trump's chief of staff Reince Priebus and spokesman Sean Spicer both left their posts over his appointment.
The decision was made by Mr Trump's new chief of staff, John Kelly, who was sworn in on Monday.
A three-sentence statement from the White House reads: "Anthony Scaramucci will be leaving his role as White House Communications Director.
