The shark bit the leg of the surfer, aged between 50 and 60, off Kingscliff, 800km (500 miles) north of Sydney.
Several "heroic" boarders tried to help the man by fighting off the shark but although he was brought to the beach, he died there of his wounds.
This is the third fatal shark attack in Australian waters this year.
The Sydney Morning Herald said that although police gunmen were deployed to find the shark, they were unable to kill it and it left the area after several hours.