Coming into Saturday night's match, the Chiefs were unbeaten in the 2020 season, while the Brumbies were coming off a narrow 23-22 loss to the Highlanders at home the week before.
That last minute loss may have spurred the Brumbies on in Hamilton, as it was the visitors who came out firing from the first whistle.
Fullback Tom Banks was the first to score for the Brumbies off a set piece move inside the Chiefs 22.
The last pass from prop James Slipper to Banks appeared to have gone forward as Slipper bobbled the ball while being tackled, but the TMO ruled it was okay and awarded the try to Banks.
Five minutes later, former Warriors centre Solomone Kata scored for the Brumbies in virtually the same spot as Banks.
In a move that was virtually the same as Banks' try, Kata brushed past a scrambling Aaron Cruden on the right wing to score in the corner, giving the visitors a 12-0 lead after 15 minutes.
Five minutes before halftime, the Brumbies scored again, this time off a scrum just five metres from the try line.
Brumbies number eight Pete Samu picked the ball up from the back of the scrum, ran over won defender to score right under the posts and give the Brumbies a 19-0 lead at halftime.
The first half was one to forget for the Chiefs, who gave away countless penalties as they struggled to contain the Brumbies offence, while also producing six handling errors on attack.
The second half didn't get off to a great start for the Chief either with Samu scoring again just two minutes in, this time breaking the line on the 22, then squeezing his way through the two man tackle of Aaron Cruden and Damien McKenzie to dot down under the posts for a second time.
By that point the Brumbies were out to a 26-0 lead and appeared to be cruising towards victory.
That was until the Chiefs, who appeared to be lifeless in the first half, began to find their groove just five minutes later.
Aaron Cruden was the first to score for the hosts, running a beautiful line off a short pass just five metres short of the line to barge his way over the try line.
Then 10 minutes later, his teammate Anton Lienert-Brown did the exact same move in a near identical play off the back of a scrum.
Those two tries cut the Brumbies lead to 26-14 and the Chiefs appeared to be in the mood for more points as their forwards came to life in the final 30 minutes of the match.
However, the comeback was too little too late for the Chiefs as they ran out of time on the clock, losing their first match of the season.
The win was the Brumbies first in Hamilton since 2007, and first win in New Zealand in six years. The result keeps the Brumbies in top spot in the Australian conference, while the Chiefs remain in second on the New Zealand ladder, one point behind the Crusaders.