Mr Sogavare lost the vote 27 to 23.
Yesterday's vote in Honiara was triggered by a series of defections from Mr Sogavare's coalition government late last month.
The vote followed an arduous debate that stretched over six hours.
Supporters of the motion said MPs had lost confidence in the prime minister because he was not listening to them any more, while development was stalling around the country.
Mr Sogavare's supporters urged MPs to let him finish the final ten months of his four-year term in office.
But the East Honiara MP, Douglas Ete, said time had run out for Mr Sogavare in his third stint as prime minister.
"Ten months is too much time, Mr Speaker, enough time to destroy this nation."
Meanwhile, the parliament is expected to convene later this month to elect a new prime minister. Until then, Mr Sogavare is the acting prime minister.
The opposition grouping hasn't declared a candidate yet - there is understood to be several MPs preparing to put their name forward.