The demonstration in Tel Aviv was attended by nearly all opposition parties, a rare show of unity for Israel's splintered political system. Police did not say how many people attended. A Reuters photographer estimated about 20,000 were present, while organizers put the figure at 80,000.
Benny Gantz, the former Israeli army chief who ran against Netanyahu in the election, addressed crowds at the rally. He said Israel's dream was "falling apart" under Netanyahu's leadership.
In office for the past decade, Netanyahu won a fifth term in April despite an announcement by Israel's attorney general in February that he intended to charge him with fraud and bribery. The prime minister is a suspect in three graft cases.
Netanyahu has denied wrongdoing, calling the allegations a political witch-hunt. The right-wing leader has said that with a renewed public mandate to govern he has no plans to resign, even if charged.
Although the prime minister is under no legal obligation to step down if charged, Netanyahu loyalists in his Likud party have pledged to seek parliamentary immunity from prosecution for him while he is in office.
Expecting legal challenges, they also have been advocating legislation that would annul any Supreme Court ruling rescinding immunity.
Since the election, Netanyahu has not said whether he would seek immunity.
On May 13, Netanyahu said on Twitter that his policy had always been to preserve a strong and independent Supreme Court, but that changes were needed in order to restore balance between Israel's executive, legislative and judiciary branches.
The opposition has described any attempt to shield Netanyahu or put limitations on Israel's highest court as threats to Israeli democracy.
Yair Lapid, one of the leaders of the main opposition party, the centrist Blue and White, said on Saturday the demonstration that Netanyahu was trying to crush the Supreme Court in order to keep out of prison. "He's destroying the country," Lapid said. "We won't let him."
Benny Gantz, the former Israeli army chief who ran against Mr Netanyahu in the election, addressed crowds at the rally. He said Israel's dream was "falling apart" under Mr Netanyahu's leadership.
Netanyahu is trying to form a new coalition with right-wing, ultranationalist and religious parties that would give him control of 65 of the 120 seats in parliament, which has already been sworn in.
Most of the parties expected to join his coalition have expressed support for granting immunity to Netanyahu and limiting the powers of the Supreme Court, branded by some on the right as too liberal and interventionist.
However, Netanyahu has only until Wednesday to produce a government and he has not yet secured a deal with any party. Negotiations came to an impasse this week when the factions failed to agree on a new conscription law for Israel's military.
According to Israeli law, if Netanyahu fails to form a government by May 29 the president can ask another member of the Knesset legislature to try.
No political party has ever won an outright majority in Israel's Knesset, making coalition governments the norm. Coalition talks have often been protracted with deals signed at the very last minute.
Netanyahu is due to attend a pre-trial hearing over the graft charges with the attorney general, set for October.