As President Obama prepares to visit Israel and Palestine later this month, he's facing a set of political and diplomatic impasses that seem virtually insurmountable. Israel and the Palestinians have never been further apart on final-status issues since formal negotiations began more than 20 years ago. Hamas and Fatah appear hopelessly deadlocked on everything from elections and a possible "unity" government to the bigger picture issues of the nature of Palestinian society, relations with the West, to what national liberation would look like and how it should be achieved.

Within those groups there's also significant factional infighting and a sense that domestic political power and national policy -- including the very concept of a two-state solution -- are at stake. Recent strikes and demonstrations against the Palestinian government about economic grievances and violent protests against Israeli treatment of prisoners have also erupted in the region.

 Among Israelis there is a marked lack of consensus about social policies, settlements, the feasibility of the two-state solution, and need for, or even desirability, of a peace agreement with the Palestinians. And while Israel continues to enjoy a special relationship with the United States, strains between Obama and Israel's prime minister Benjamin Netanyahu, strains that began over the settlement-freeze issue, haven't really abated. Washington's relationship with the Palestinian leadership has also been under serious pressure over both American disapproval of Palestinian diplomatic moves at the United Nations and Palestinian frustration with the American inability to shift Israel on settlements.

Under these circumstances, and with a significant recent deterioration of the political, security, and economic situation on the ground, a major strategic advance in the immediate future remains an ambitious agenda.

The antagonisms between and within the parties are shaped by their regional context, in particular the upheavals in several Arab countries, especially Egypt and Syria. The Arab political scene is generally characterized by a contest between three major trends: traditional regimes and their state structures, rising Islamist movements, and fledgling new opposition groups centered on universal values and the rights of citizens.

Arab monarchies have their own unique dynamics. They have the benefit of being integrated into the international economy and global security system, advantages need not only to preserve but enhance as they negotiate their internal issues. Their challenge is maintaining hard-won stability by developing a consensus between government and opposition on how to ensure the consent of the governed.

In the rest of the Arab world, a number of long-standing regimes -- ranging from war-torn Syria, to unstable Sudan and Yemen, and a wary Algeria -- continue to face demands for change, and, like the rest of the region, they're confronted by two rising and opposing trends: The first, and best organized, are the Islamist groups, both Sunni and Shiite. They claim authenticity based on religious and cultural uniqueness and foster dreams of recuperating cultural and doctrinal "purity." The other trend is the fledgling groups informed by universal, pluralistic and democratic values and deeply connected to developments throughout the world.

In a sense, the Islamist trend looks to the future in terms of a return to a largely imaginary past. The nascent pluralists, on the other hand, seek to build a future for a Middle East that's integrated into a broader, rapidly developing global economy and society. The Islamists essentially point the Arabs in the direction of Iran or Sudan, while the universalists and pluralists look towards the West.

Islamists, claiming deep roots in history, tradition, and religion, have the advantage of years of organizing and opposition, and are unburdened by the record of corruption associated with former dictatorships. On the other hand, the pluralistic, universalist groups suffer from a double-whammy of disadvantage from the old authoritarian regimes. They were more ruthlessly suppressed than the Islamists, who could always operate under the cloak and within the structures of religious institutions. Worse, their rhetoric about the values of modernity, democracy, equality, women's rights, citizenship and pluralism was systematically distorted and discredited by authoritarian regimes that hypocritically hijacked this language -- perhaps most shamelessly of all in Damascus.

All of these trends are in play among the Palestinians. But Palestinians also have to contend with a distinctive challenge not faced by other Arabs: the occupation. Living under occupation means that others, particularly Israel and the donor community, can set real limits in deciding what the Palestinians can and cannot do. In most ways, this is a profound disadvantage for Palestine. But it does also mean that Palestinians can construct their state from scratch. Notwithstanding recent UN and other international recognitions, there's no existing sovereign Palestinian state on the ground, although there is indeed a virtual and aspirational one.

The issue of Palestinian political legitimacy can only be resolved through free and fair elections. This means not just voting, but preparing the political basis for elections by enabling a contest of ideas, values, and choices that allows for the formation of parties and brings in new leaders.

At the same time, a serious improvement in living conditions on the ground would restore belief in the possibility of peace and prosperity through political means. The vital lesson of the past two years is that the international community must shield the progress made in living conditions, good governance, and reform from the vagaries of the political and diplomatic processes.

It may well be that for Israelis and Palestinians to achieve what they both require at an existential level, which is also a vital American national security interest -- a negotiated peace agreement allowing two states to live side-by-side in peace, security, and dignity -- broader regional questions, particularly around political culture and governance, need to be resolved. The West and the rest of the international community has a vested interest in helping ensure that the outcome of the transformation in the Middle East is eventually defined by a generalized acceptance of the values of freedom, pluralism, and respect for the rights and responsibilities of citizenship.

Such an outcome would require and produce an independent Palestinian state, genuine acceptance with political and economic normalization of Israel within the region, a final resolution of the broader Arab-Israeli conflict, and the birth of a new, stable, and prosperous Middle East integrated with the rest of the international community.

The real battle in the Middle East is between those who wish to see the region in general, and the Arab world in particular, modernize along the lines of universal values, and those who would impose their own versions of intolerance and authoritarianism. Which is to say: Values and ideas matter. The future of Palestine, Israel and the rest of the Middle East will be determined more by an ongoing contest of ideas -- just as it was in Europe in the decades after World War II -- than by the exigencies of the present moment.