The Waratahs have conceded more than 100 points in two record losses to start the Super Rugby AU season after a 41-7 thrashing by the Queensland Reds last week and a 61-10 humbling at the ACT Brumbies on Saturday.
New Zealander Penney said his young side were "shattered" and "broken" after a masterclass in ruthless rugby from the nine-try Brumbies at Canberra Stadium.
"I love these boys and I'm doing my best," Penney said. "If other people think I'm not the right person, I'm not going to stand in the way.