The august ranks of those who form the conventional wisdom in American politics are as one: Barack Obama's Democrats are going to take a hammering in next Tuesday's midterm elections. One of the few elements of the Obama record not blamed is also, paradoxically, one of those areas that need to change on 3 November. It is the administration's handling of the peace process between Israelis and Palestinians.

Consensus is famously rare in these matters, but there's a striking degree of agreement both among the parties themselves and seasoned Washington hands that things have gone badly wrong. A president who was inaugurated amid great hopes – generally and for the Middle East in particular – has seen his stock tumble in the intervening 21 months. Obama won great credit for making Israeli-Palestinian peace a priority on his very first day in office, declaring it a national security interest of the US, and for appointing the elder statesman of the Northern Ireland effort, George Mitchell, as his special envoy. Since then there have been crockery-throwing rows with the Israelis, and a nice speech in Cairo, but little tangible progress. The most Obama has to show for his labours is an opening round of direct talks between the two sides, currently on hold.

Ask those involved what explains the journey from hope to disappointment, and they all start in the same place: the US administration's early demand that Israel cease all settlement building in the occupied West Bank before any talks could begin. Make no mistake, Palestinians and plenty of others would like nothing more than just such an outcome; their objection was not to the substance of the demand but to the politics of it.

Even the Palestinian president, Mahmoud Abbas, would tell visitors to his Ramallah compound the ultimatum was a mistake. Negotiations had taken place before without a settlement freeze, but once the precondition had been made Abbas had to insist on it too. As Abbas would put it: "This has put me up a tree and I have no ladder down."

Others say the demand itself was perfectly reasonable; the problem was that the US did not stick to it. Rather than hold firm, it found itself haggling, eventually compromising on a partial, temporary freeze. Put simply, America blinked. The result was a loss of face from which the administration has not recovered.

That first phase – of calibrated confrontation with Israel – reached a climax in March, when Vice-President Joe Biden visited the country only to be greeted by the announcement of more house-building in east Jerusalem. Neither he nor Obama kept their fury to themselves.

The problem with the initial approach, says Daniel Levy of the Washington thinktank the New America Foundation, is that Obama had not yet "established his bona fides with the Israeli people". If Obama wanted to speak over the heads of the Israeli leadership, as Bill Clinton had done before him, he needed first to reassure Israelis that he was not the "Barack Hussein Obama" of hostile mythology. But he made no such move. He didn't do so much as a single set-piece interview with the Israeli media. Instead, the vacuum was filled with what Levy believes was a "concerted campaign to undermine Obama with the Israeli public", one backed by a Binyamin Netanyahu determined not to repeat the experience he underwent in the 1990s, when a US president more popular with Israelis than himself was able to push him to the peace table.

Either way, after the Biden standoff the White House changed course, concluding that confrontation was bearing little fruit. What's followed has been a classic Washington over-correction. As Mitchell's prominence has waned, the veteran US mediator Dennis Ross has gained influence – and Ross's approach, says one non-admirer, is "you do whatever is the maximum the Israelis will do". The advantage of such a strategy is that it's realistic, understanding that, as a defender puts it: "Ultimately, if you want to move forward, you're going to have to work with the government of Israel, not against it." The downside is that it fails to push the Israelis to move.

Which brings us to the round of talks that began in September and has been on ice since Israel allowed its partial settlement freeze to expire. Here the protagonists are scathing. "The Americans are desperate," says a Palestinian adviser. "They have no idea what they're doing or what they're doing it for."

The criticism starts with the basis on which the talks are conducted. There are no terms of reference, no letter to the parties as there was before the Madrid conference of 1991, no parameters laid out by the Americans as there was before Camp David in 2000. The two sides are just meant to get on with it.

What's more, adds the Palestinian insider, Washington is "frittering away its best cards". Anxious to keep Netanyahu at the table, the US has dangled before him a string of "inducements", ranging from the right to purchase F35 fighter planes – which no one else in the world has – to tolerance of a continued Israeli military presence along the Jordan valley, perhaps for up to 30 years after any peace deal, thereby boxing in a future Palestinian state on both its eastern and western borders.

Others note the behind-the-scenes efforts Washington continually makes, "working the phones" to foreign capitals to shield Israel from severe international criticism or isolation, including smoothing the path for membership of the OECD earlier this year. "What's it all for?" asks one observer, incredulous that the US is apparently offering all these goodies in return for nothing more than a temporary extension of Israel's partial settlement freeze, perhaps for 60 days. More alarming, one involved close up says that Washington is offering what it cannot deliver: hinting at concessions to the Palestinians, for instance promising a state broadly on the 1967 lines, that Israel has not yet agreed to. It does indeed carry the whiff of desperation, suggesting an administration doing somersaults to avoid another public failure.

There is a case for the defence. It gives credit to Obama for sticking with an issue that his predecessor long preferred to ignore and that almost always drains rather than enhances presidential prestige. It notes that Israel, whether through the initial stick or the later carrots, has moved a bit: some say the very act of freezing settlements has forced Israel to admit they are a problem that cannot be sustained indefinitely. And it points out the undeniable truth that the lack of success is hardly the exclusive fault of the US: the parties themselves are weak, with Abbas ruling over only one half of a divided people, while Netanyahu – even if he is sincere about wanting to do a deal, as he insists – is constrained by coalition partners of the dogmatic right.

Nevertheless, the post-midterm reckoning should include an honest analysis of what's gone wrong in one of the president's key foreign policy priorities, and a plan for what to do next. That could include setting out an Obama plan for the region and compelling the parties to negotiate around it. It could mean finding a new path between the ill-considered pressure on Israel of 2009 and the unreciprocated generosity towards it of recent months.

Whatever course Obama chooses – and he will deserve admiration for not simply washing his hands of the entire mess – he does need to change course. To borrow a phrase from his secretary of state, it's time to hit the reset button.