Ehud Barak, Israel's defence minister, last night delivered an unusually blunt ­warning to his country that a failure to make peace with the Palestinians would leave either a state with no Jewish ­majority or an "apartheid" regime.

His stark language and the South African analogy might have been unthinkable for a senior Israeli figure only a few years ago and is a rare admission of the gravity of the deadlocked peace process.

There have been no formal negotiations between Israel and the Palestinians in more than a year, but Barak was speaking at a rare joint event with the Palestinian prime minister, Salam Fayyad, as part of an annual national security conference in the Israeli city of Herzliya. The pair shook hands and both were warmly applauded.

Barak, a former general and Israel's most decorated soldier, sought to appeal to Israelis on both right and left by saying a peace agreement with the Palestinians was the only way to secure Israel's future as a "Zionist, Jewish, democratic state".

"As long as in this territory west of the Jordan river there is only one political entity called Israel it is going to be either non-Jewish, or non-democratic," Barak said. "If this bloc of millions of ­Palestinians cannot vote, that will be an apartheid state."

He described Israel and the Palestinian territories as the historic "land of Israel" to which Israelis had a right.

"We have to demarcate a border within the land of Israel," he said.

"We have a linkage, we have a right, but the reality of standing on the stage of history in realistic terms requires us to pay attention to ­international constraints." Barak is in a delicate political position. He leads the Labour party, supposedly a centre-left movement, but accepted a position in a rightwing coalition under Binyamin Netanyahu, a decision that split his party.

Though Barak articulates a willingness for peace talks, he represents a government that has defied US and Palestinian calls for a full settlement freeze as a prelude to any negotiations. He was also defence minister during last year's Gaza war in which nearly 1,400 Palestinians and 13 Israelis were killed.

The Herzliya conference has echoed Israeli concerns about growing ­international criticism, particularly in the year since Gaza. Barak himself alluded to the danger that Israel might lose ­legitimacy if no peace deal was forthcoming. "The pendulum of legitimacy is going to move gradually towards the other pole," he said.

He acknowledged that Washington was pushing the two sides towards "proximity talks" but said this was "only an initial stage" before any return to full negotiations.

Fayyad, who has a limited political following among Palestinians, called on Israel to stop settlement building in the occupied territories and to halt military incursions in Palestinian cities as a sign of seriousness about negotiations.

"Things have to begin to happen in order to give the suggestion that this occupation is going to end," he said. "That Palestinian state is supposed to emerge precisely where settlements are expanding." Mahmoud Abbas, the Palestinian president, has refused to start fresh negotiations with Israel unless settlement construction stops, in line with the 2003 US road map. Nearly 500,000 Jewish settlers live in east Jerusalem and the West Bank, even though settlements on occupied land are illegal under international law.

"How confident can we all be that once relaunched that political process is going to be able to deliver that which needs to be delivered, the permanent status issues and the key question of ending the ­occupation?" Fayyad asked.