How 1,600 confidential Palestinian records of negotiations with Israel from 1999 to 2010 came to be leaked to al-Jazeera

The revelations from the heart of the Israel-Palestine peace process are the product of the biggest documentary leak in the history of the Middle East conflict, and the most comprehensive exposure of the inside story of a decade of failed negotiations.

The 1,600 confidential records of hundreds of meetings between Palestinian, Israeli and US leaders, as well as emails and secret proposals, were leaked to the Qatar-based satellite TV channel al-Jazeera and shared exclusively with the Guardian. They cover the period from the runup to the ill-fated Camp David negotiations under US president Bill Clinton in 2000, to private discussions last year involving senior officials and politicians in the Obama administration.

The earliest document in the cache is a memo from September 1999 about Palestinian negotiating strategy. It suggests heeding the advice of the Rolling Stones: "You can't always get what you want, but if you try sometimes you might find you can get what you need." The final one, from last September, is a Palestinian Authority (PA) message to the Egyptian government about access to the Gaza Strip.

The Palestine papers have emerged at a time when a whole era of Israeli-Palestinian negotiations, starting with the Madrid conference in 1991, appear to have run into the sand, opening up the prospect of a new phase of the conflict and potentially another war.

In particular, they cover the most recent negotiations, before and after George Bush's Annapolis conference in late 2007 – when substantive offers were made by both sides until the process broke down over Israel's refusal to freeze West Bank settlement activity.

The bulk of the documents are records, contemporaneous notes and sections of verbatim transcripts of meetings drawn up by officials of the Palestinian negotiation support unit (NSU), which has been the main technical and legal backup for the Palestinian side in the negotiations.

The unit has been heavily funded by the British government. Other documents originate from inside the PA's extensive US- and British-sponsored security apparatus.

The Israelis, Americans and others kept their own records, which may differ in their accounts of the same meetings. But the Palestinian documents were made and held confidentially, rather than for overt or public use, and significantly reveal large gaps between the private and stated positions of Palestinian and, in fewer cases, Israeli leaders.

The documents – almost all of which are in English, which was the language used by both sides in negotiations – were leaked over a period of months from several sources to al-Jazeera. The bulk of them have been independently authenticated for the Guardian by former participants in the talks and by diplomatic and intelligence sources.

The NSU – formally part of the Palestine Liberation Organisation (PLO) – is based in the West Bank town of Ramallah under the chief Palestinian negotiator, Saeb Erekat. It has drawn heavily on the expertise of Palestinian-American and other western-trained diaspora Palestinian lawyers for technical support in negotiations.

In the case of one-to-one talks between Israeli and Palestinian leaders – especially between Mahmoud Abbas and the then Israeli prime minister Ehud Olmert – NSU officials were not present, but reports on the outcome of the encounters were often given later to the unit and records made.

After the breakdown of the Camp David talks, which Clinton and Israeli leaders blamed on Yasser Arafat and a lack of technical Palestinian preparation, Palestinian leaders went to great lengths to ensure that the fullest records and supporting documents were drawn up for later talks. Among NSU staff, the Arab-American lawyer Zeinah Salahi drew up many of the meeting records, while others were made by the French-Palestinian lawyer Ziyad Clot, author of a book about the negotiations, Il n'y aura pas d'Etat Palestinien (There will be no Palestinian state).

The role of the NSU in the negotiations has caused tensions among West Bank-based Palestinian leaders and officials, and widespread resentment about the salaries paid to its most senior managers, notably Adam Smith International's Andrew Kuhn, who stepped down from running the unit last year.

But as the negotiations have increasingly been seen to have failed, and the Ramallah-based PA leadership has come to be regarded by many Palestinians as illegitimate or unrepresentative, discontent among NSU staff has grown and significant numbers have left. There has also been widespread discontent in the organisation at the scale and nature of concessions made in the talks.

Among NSU staff cited in the documents, Salahi now works for the US embassy in Cairo, Clot has returned to France and Rami Dajani works for Tony Blair in his role as the Middle East quartet's envoy. Kuhn is working elsewhere for Adam Smith International, including on projects in Afghanistan.

In response to the leaks, PA and PLO leaders such as Saeb Erekat can be expected to point out that one of the core principles of the negotiations is that "nothing is agreed until everything is agreed". As such they are not necessarily committed to provisional positions that in the event failed to secure a settlement – though Erekat made clear to US officials in January 2010 that the same offers remained on the table.

Critics are likely to argue that concessions – such as accepting the annexation of Israeli settlements in occupied East Jerusalem – are simply pocketed by the Israeli side, and risk being treated as a starting point in any future talks.

Some Fatah leaders are likely to accuse al-Jazeera of having an anti-PA agenda by publishing the leaked documents, which they believe will benefit their Hamas rivals, backed by Iran — as shown in critical comments about the TV station in the documents themselves.

Relations between al-Jazeera, the most widely watched TV channel in the Middle East, and the PA leadership have often been strained after it has run reports regarded by the administration as hostile – as is the case with regimes throughout the region.

The documents have been redacted to remove details such as email addresses, phone numbers or other information that could identify those who leaked them.