Tom Cusack scored Australia’s only try, while South Africa got five-pointers from Seabelo Senatla (two), Roscko Speckman and Kyle Brown.
Australia had beaten South Africa 12-5 earlier on day two to book their spot in the quarters but they couldn’t repeat the dose at Deodoro Stadium.
Australia captain Ed Jenkins told reporters he was “devastated” to bow out after years of hard work.
Gold medal favourites Fiji beat New Zealand 12-7 in a tense opening quarter-final.
Fiji had earlier helped New Zealand — missing three players including Sonny Bill Williams and captain Scott Curry — sneak into the quarters by beating the United States by five points.
“Those quarters you just want to get through them with 12 fit men, which we’ve done,” Fiji coach Ben Ryan told reporters.
“We’ve got one aim and that’s to win gold medal, not silver or bronze.
“Sometimes it’s dangerous to set out goals as a coach, but we’re No 1 in the world, we’re not trying to be arrogant, we’re saying this is what we want.”
Fiji will play surprise packets Japan in the first semi-final on Friday (3.30am AEST).
So I don’t know what’s happened here!
“The way it’s gone so far, I think we’re still in with a big chance.
“We just need a few calls to come our way, maybe a few yellow cards and we could be in for a medal.
“There’s no pressure on us, the pressure’s on the big teams, they are the ones supposed to be winning the medals, not us.”
And Dan Bibby scored a try in golden point extra time as Great Britain beat Argentina 5-0 in an epic third quarter-final.
Britain will play South Africa in the second semi at 4am AEST.
The gold medal match will kick off on Friday at 8am AEST.