Surely the heart should give a cheer at the hints and signals that suggest Barack Obama will stand before the world next month, either at the UN general assembly or the G20 in Pittsburgh, and launch his own bid for Middle East peace. We have told ourselves for so long that a solution is possible – that everyone knows the contours of an eventual agreement between Israelis and Palestinians – that the urge is almost overwhelming to believe it is within reach.

After all, here is a president who is internationally admired where his predecessor was reviled; a president apparently alive to the nuances and complexities of this deeply troubled region; above all a president who believes in diplomacy. Surely, if anyone was destined to play the role of Middle East peacemaker, it is Obama. What's more, the moment seems ripe. For once, large swaths of the Arab world share a common interest with Israel: Saudi Arabia and others fear Tehran more than they fear Tel Aviv, and might be prepared to bury their differences with Israel if that brings united, international action against Iran and its nuclear ambitions. Is it too much to hope that Obama is the right man and September 2009 the right time to bring peace to the Middle East?

I fear it might be. The holy land is haunted by the ghosts of men who believed they were uniquely able to succeed where others had failed. Several presidents reckoned they could make the difference – only to fail. What matters is not the special gifts Obama can boast, but the underlying landscape on which he stands. And, in the Middle East, that does not inspire hope.

So it looks encouraging, at first glance, that Israel's Binyamin Netanyahu is in London today for a long session of talks with George Mitchell, the man who brought balm to Northern Ireland and has been tasked by Obama with doing the same for Israelis and Palestinians. At his press conference alongside Gordon Brown yesterday, the Israeli prime minister made a good fist of suggesting he had narrowed the gap between himself and Obama's demand for a complete freeze on settlement building in the lands Israel gained in 1967. Of course there would have to be some building in existing settlements – settlers, explained Bibi, have children who need schools and kindergartens – but there would be no new ones. Using the language of Israel's sternest critics, he insisted: "This is very different from grabbing land ... We're not going to expropriate additional land."

Perhaps he and Mitchell will be able to construct a compromise formula out of that – agreeing, say, to a one-year building freeze with certain exceptions for "natural growth". But the Palestinians say they will not agree to any new talks unless Israel submits to a complete freeze. They may well buckle in the end, especially if Washington insists it still believes in the principle of a complete halt to settlement activity. But such an outcome will represent an inauspicious start to a new peace process. In a staring contest with Israel, Obama will be the one who blinked. He will be exposed as weak, unable to persuade a dependent ally to bend to his will. That represents a loss of face for the man who needs both sides to fear him if he is to get results.

Let's say Obama gets over that hump and talks begin. Do we have grounds to be hopeful then? Veteran analyst and sometime negotiator Hussein Agha is not optimistic: "What exactly is the difference between this and Camp David or Annapolis?" Advocates say the novelty is that this effort will be trilateral, with the US a full participant, while George Bush left the two sides to their own devices. But such a view forgets that Condoleezza Rice travelled to the region 17 times in 10 months. And Israelis and Palestinians hardly lacked for US hand-holding in the Clinton era. And it still didn't work.

Indeed, the Camp David talks of 2000 should give further pause. Israel was represented then by Ehud Barak, who had at least some ideological commitment to peacemaking. He faced in Yasser Arafat a Palestinian leader with the authority to sell any deal to his people. A new initiative would pit Netanyahu, who has made a career out of scepticism towards peace, against Mahmoud Abbas, whose legitimacy among Palestinians is fragile. It is surely a stretch to imagine that these circumstances are more conducive to success than those of nine years ago.

Some will be encouraged by today's Guardian report suggesting that the US envisages a two-year timeline, perhaps even a deadline. But then they will remember the speech Bush made in 2003 promising a Palestinian state by 2005. Or they will recall Bush's promise of statehood by 2008. In the Middle East, deadlines come and go.

There is one significant difference this time: Obama himself. Not his negotiating skills, which are unproven, but his standing in the Arab and Muslim world. That is, unequivocally, better than any of his predecessors (though the settlements stand-off is denting it fast). The problem is that it has come at a price: a Jerusalem Post poll in June found that only 6% of Israelis regarded the Obama administration as pro-Israel. That will surely make it that much harder for the US to persuade Israelis to grant the concessions any agreement will need.

So, not for the first time, an attempt at peace seems to be facing impossible odds. It is a glum thought: Obama's effort taking its place alongside Oslo, Camp David and Annapolis as attempts that failed. What can be done?

The best approach might be to make the problem apparently harder. For the last two decades, advocates of a two-state solution have sought to reduce the scale of the challenge, to confine a peace process to reversing the effects of 1967, ending the occupation that began that year and letting Palestinians rule the lands they lost. But that is to pretend the conflict began in 1967. It did not. It goes back at least to Israel's founding in 1948, if not to the arrival of the first Jewish newcomers, determined to rebuild their ancient homeland, in the last years of the 19th century.

Perhaps successive efforts at peace have failed because they have ducked the core, existential questions, the issues of 1948. Maybe a true peace will only come when it confronts the hardest issues: on one side, the dispossession of the Palestinians, forced to become a refugee nation; and on the other, the urge which led to Israel's creation, the Jews' desire after two millennia in exile to live in a state of their own.

This will require an entirely new approach. Not a reliance solely on mechanistic formulas, land swaps and compensation packages, but a deep, even emotional engagement with the sources of the conflict. It will mean Israelis finally acknowledging the plight of the refugees created by the birth of the state of Israel, and Palestinians finally deciding whether they can accept a Jewish state. That last move will have to be done without jeopardising the place of those Palestinians who live inside Israel as citizens, or asking Palestinians to reject the entire narrative of their recent history. But something like it will have to be done.

Peace has not remained out of reach because the peacemakers did not try hard enough or because the moment was not ripe. Peace may have stayed out of reach because for too long we refused to confront the true causes of this war.