Mr Erdogan had planned to address the rally in the city of Cologne, held to denounce an attempted coup two weeks ago, by video link.
But on Saturday, Germany's Constitutional Court banned the speech from being broadcast.
German media said at least 35,000 people turned out.
An estimated three million people of Turkish origin live in Germany, the majority of whom voted for Mr Erdogan's AKP party in the last Turkish election, according to the Turkish Communities in Germany organisation.