He told young people gathered in Krakow "a merciful heart opens up to welcome refugees and migrants", a statement that puts him at odds with Poland's anti-immigrant right-wing government.
It is Pope Francis's first visit to eastern Europe, with Poland the home of the late Pope John Paul II.
On Friday he is visiting the former Nazi death camp of Auschwitz-Birkenau.
There he saw the icon known as Black Madonna, a relic that has been venerated for six centuries.
Later, Church organisers said about 500,000 young people had joined the Pope for an evening rally, where he heard music from around the world.
He urged young Catholics to show compassion, saying a "merciful heart is able to be a place of refuge for those who are without a home or have lost their home; a merciful heart is able to build a home and a family for those forced to emigrate."
Poland refused to take part in an EU deal to take in refugees. Ahead of the visit, the Vatican appeared to tone down criticism of Polish politicians' views of migrants.
Pope Francis is in Poland to celebrate 1050 years since the country adopted Christianity.