He is scheduled to appear at Wellington District Court later this morning.
Police were called just before 5.30am after the man was seen with the axe.
The man smashed five glass panels at the main entrance to the Parliament buildings.
The entranceway is now blocked off by metal barriers and RNZ reporter Ben Strang told Morning Report an industrial hoover had been used to clean-up the glass debris.
He said the man hadn't actually tried to enter Parliament and they were able to arrest him without any further incident.
The man has been charged with intentional damage and possession of an offensive weapon. The motivation for the attack is unknown.
In a statement, Parliament said the entrance would be closed until further notice, with the rubber doors of Parliament House an alternative entrance in the meantime.
Security screening will be running from there.
Police said the man did not try to enter the Parliament buildings.
The 31-year-old was charged for intentional damage and possession of an offensive weapon and is expected to appear in Wellington District Court this morning.