Centre Tevita Kuridrani's forceful run and one-armed stretch to touch down on the line gave the Wallabies their second try four minutes from time and Bernard Foley's conversion sealed the win.
Australia, who beat Wales 32-8 in Cardiff last weekend, made light of being a man down in the dramatic final minutes after replacement forward Will Skelton was sin-binned for dangerous play.
Two tries by exciting young centre Huw Jones in his first test start had raised the hopes of Scotland, who lost 35-34 to the Australians in last year's World Cup quarter-final.
"Absolutely heartbroken, went in at halftime with a lead and thought we could hold on," said Jones, whose brace had helped Scotland lead 17-10 at the interval.
"Absolutely gutted. Our discipline was not good enough, too many offside penalties," he told said before adding of his tries that it was an "unbelievable feeling to get over the whitewash."
Australia captain Stephen Moore said the Wallabies had to make a quick adjustment when Skelton went off but kept their discipline well.
"Bit too close for comfort," he said. "Just hung on in, in the second half. Helluva game, just fortunate to come out on the good side of it."
Scotland made a fine start and followed up halfback Greig Laidlaw's early penalty with Jones's first try.
First-five Finn Russell set up the move with a deft chip over the defence and Jones was quick to follow through, pick up a from a favourable bounce and race over.
The Wallabies replied with a try in the 13th minute by centre Reece Hodge.
First-five Foley, whose perfect place kicking display landed 13 points, launched the attack from a ruck then came round behind Israel Folau to take a return pass and set up winger Dane Haylett-Petty who off-loaded to Hodge.
After a Foley penalty had put Australia level, the Scots recovered their lead when lock Richie Gray made the breakthrough inside the Wallaby 22 and passed to Jones who went over.
Scotland would have been deserved winners for their endeavour and thrilling play but only managed to add Gray's try and the conversion to their total in the second half.
It had seemed enough, especially after Skelton was shown the yellow card, but the Wallabies chose to kick their penalties for touch as they sought to overcome a six-point deficit rather than going for three-pointers.
Scotland play Argentina next weekend while Australia visit France.