The men - aged 17, 18 and 26 - were detained after a series of pre-dawn raids in the states of Schleswig-Holstein and Lower Saxony on Tuesday.
Police are reported to have seized "extensive material".
But no concrete missions or orders had so far been found, the Federal Public Prosecutor's office said.
In a statement, it said investigations so far suggested the three had come to Germany in November 2015 with the intention of "carrying out a previously determined order [from IS] or to await further instructions".
