Legal challenges failed to prevent the release. The victim's parents were among those detained later at a rally.
The rapist, who cannot be named as he was a minor at the time of the crime, was sentenced to a maximum three years in a reform facility in August 2013.
The rape and the subsequent death of the woman caused global outrage.
Although the convict is now an adult, he was tried as a juvenile and has served the full sentence allowed.
He has been handed over to a charity, where he will remain because of fears over his safety.
Four adult convicts in the case are appealing against death sentences. A fifth died in prison.
The release was opposed by many people, including the parents of the victim.
They were prevented from holding a protest rally in central Delhi earlier on Sunday and when they later joined scores of protesters near India Gate, they were among a number detained by police, who broke up the demonstration.
The convict's identity is being changed and no record of his crime will remain in the public domain, a move that has caused more outrage among many in India.
A legal challenge by politician Subramanian Swamy to stop the release failed.
A court ruled on the case on Friday, saying: "We agree it is a serious issue. But after 20 December, the juvenile cannot be kept at a special home per law."
Late on Saturday, Swati Maliwal, the head of Delhi Commission of Women, filed a petition to the Supreme Court trying to prevent the release.
It will hear the case on Monday, but the release went ahead anyway.
Rape victims cannot be named under Indian law, but last week the victim's mother, Asha Singh, revealed it at a public rally.
After the release on Sunday, she said: "I just want justice, I want a stay on his release."
She said the Delhi Commission of Women had filed its petition too late on Saturday.
"They might have been able to stop the release if they had tried during the day," the Times of India quoted her as saying.