The Economist on Predicting behavior and Forecasting choices: Interview by Nate Silver of FiveThirtyEight.com

Nate Silver has gained his reputation as an accurate forecaster.  Silver uses sophisticated statistical tools such as Logit Model to describe and predict choices.  Silver’s website, FiveThirtyEight, is a must for any policy person interesting in predicting choice.  The Economist did an interesting interview with Nate Silver.

Here is the abstract: “NATE SILVER has been called a “prodigy”, a “wunderkind”, and a “genius”. Using his unique methodology, he was able to correctly predict every state but Indiana in the 2008 presidential election. He also got every Senate race right. That is why we here at The Economist find ourselves visiting his website, FiveThirtyEight, several times a day. This week Democracy in America talked with Mr Silver about polling, predictions and politics.”

Here is the link to the interview: Seven questions for Nate Silver

Karl Rove reviews the 2008 U.S. Presidential and Congressional elections

Karl Rove reviews the data and makes interesting observations about the 2008 Presidential elections.  The complete article can be seen in The Wall Street Journal, and here is an abstract —

“Political races are about candidates and issues. But election results, in the end, are about numbers. Wa So now that the dust is settling on the 2008 presidential race, what do the numbers tell us?

First, the predicted huge turnout surge didn’t happen. The final tally is likely to show that fewer than 128.5 million people voted. That’s up marginally from 122 million in 2004. But 17 million more people voted in 2004 than in 2000 (three times the change from 2004 to 2008).

Second, a substantial victory was won by modest improvement in the Democratic share of the vote. Barack Obama received 2.1 points more in the popular vote than President Bush received in 2004, 3.1 points more than Vice President Al Gore in 2000, and 4.6 points more than John Kerry in 2004. In raw numbers, the latest tally shows that Mr. Obama received 66.1 million votes, about 7.1 million more than Mr. Kerry.

One of the most important shifts was Hispanic support for Democrats. John McCain got the votes of 32% of Hispanic voters. That’s down from the 44% Mr. Bush won four years ago. If this trend continues, the GOP will find it difficult to regain the majority.

In a sign Mr. Obama’s victory may have been more personal than partisan or philosophical, Democrats picked up just 10 state senate seats (out of 1,971) and 94 state house seats (out of 5,411). By comparison, when Ronald Reagan beat Jimmy Carter in 1980, Republicans picked up 112 state senate seats (out of 1,981) and 190 state house seats (out of 5,501).”

Mark Halperi: Ten interesting observations on the U.S. Presidential election

Mark Halperin recites ten interesting observations about the 2008 U.S. Presidential elections.  Visit Time.com for the complete article.  Here is an abstract —

“3. Wrong Track Sky-High

“By Election Day, the national mood was so sour that fundamental change seemed the most rational choice.”

4. Outsiders In

“Experience is an overrated commodity in presidential politics, but typically at least one of the two major parties nominates someone with long-standing ties to Washington and backing from the party establishment.”

6. An African American but Not Just an African American

“…not only was race not Obama’s signature dimension by any measure, but — with the exception of the Rev. Jeremiah Wright controversy — it was barely an issue at all.”

7. Best. Reality Show. Ever.

“Politics is show business for ugly people, the old joke goes, but the 2008 campaign was just plain show business, with a cast of fascinating if not always camera-ready players.”

8. Internet Fund-Raising Comes of Age

“In September, when Obama collected a stunning $150 million in 30 days, almost 75% of the haul arrived via the Web.”

10. An October Surprise (in September)

“The drumbeat of bad economic news never let up through Election Day, drowning out any other message Republicans tried to deliver and blunting the impact of character attacks against Obama.””

Marc Ambincer’s reflections on why Obama won and McCain lost

Marc Ambinder’s (The Atlantic) thoughtful analyses on the why Barack Obama won and John McCain lost the U.S. presidential elections can be found in his blog, http://marcambinder.theatlantic.com/

Here is part of that analysis —

“– Obama is a once-in-a-generation candidate, a brilliant communicator in an age of communication. Cool and consistent under pressure. He grew over the course of two years into a candidate voters believed was ready to be president. The right candidate at the right moment. The most un-Bush of any of the Democratic candidates.

— The financial crisis, and the candidates’ response to it.  Probably the crucial moment for both campaigns. The voters saw the two men react to an unexpected crisis. Voters seemed to prefer Obama’s steadiness to McCain’s suspended campaign. McCain’s sudden decision was 180 degrees from what he had been saying a week before (“fundamentals of our economy are strong”).

— Sarah Palin. Polling shows that she drove some voters away from Sen. McCain and to Barack Obama. Voters judged her to be too inexperienced to be president. Also, instead of appealing to independents, she became a polarizing figure.  ALSO — her persona highlighted McCain’s age and health since she could have taken over. ALSO — her selection killed the “inexperience” argument against Obama.

— Message, message, message. Obama branded himself as “Change” two years ago, McCain tried Maverick, Reformer, Country First, Steady Hand At The Wheel, Tax Cutter, and even flirted with “Real American” by the end, and none of them were consistent.”

Concession Remarks by Atal Behari Vajpayee in May 2004, and John McCain in November 2008

Many Indians have compared the then Indian Prime Minister Atal Behari Vajpayee’s concession remarks in 2004 to John McCain’s touching concession remarks on November 4th evening.  In May 2004, Atal Behari Vajpayee and his party Bharatiya Janata Party lost in India’s parliamentary elections — that electoral outcome was a stunning event as no pre-elections poll had even hinted this.  At that point, Prime Minister Vajpayee relinquished power with ease and grace, and without any efforts to cobble together any unholy and unseemly electoral coalition partnership.

Upon review, I do find a common sense of generosity and purposefulness in Prime Minister Atal Behari Vajpayee’s remarks and John McCain’s remarks.  Thanks to P.K. Ramakrishnan, and Bibrama Sinha who pointed me to this.  And here are the remarks by the then Prime Minister Atal Behari Vajpayee as reported in The Tribune on May 13th, 2004 —

“Mr Atal Bihari Vajpayee, who this evening submitted his resignation to the President, Dr A.P.J. Abdul Kalam, said his party and alliance might have lost, but India had won.

“India has shown its resolve and capability to overcome the challenge of cross-border terrorism. India has also embarked on a historic peace process with our neighbour. It will remain my lifelong wish to see
that we herald a new chapter of peace, cooperation and friendship in  South Asia, with the People’s Republic of China and with other nations in the world,” Mr Vajpayee said in his address to the nation.

The outgoing Prime Minister began his address by saying that he accepted the verdict given by the people. He said when he assumed office, the nation was faced with the challenge of stability, good
governance and development.

“It is for you — and history — to judge what we achieved during this period,” he said and added that the country was stronger and more prosperous than when the reigns of office were put in his hands.”

Christopher Buckley endorses Barack Obama

Christopher Buckley, the son of William Buckley — the founder and leader of conservative movement in the United States — has endorsed Barack Obama.  Here is a part of Christopher Buckley’s opinion and rationale as it appeared in The Daily Beast —

“I’ve read Obama’s books, and they are first-rate. He is that rara avis, the politician who writes his own books. Imagine. He is also a lefty. I am not. I am a small-government conservative who clings tenaciously and old-fashionedly to the idea that one ought to have balanced budgets. On abortion, gay marriage, et al, I’m libertarian. I believe with my sage and epigrammatic friend P.J. O’Rourke that a government big enough to give you everything you want is also big enough to take it all away.

But having a first-class temperament and a first-class intellect, President Obama will (I pray, secularly) surely understand that traditional left-politics aren’t going to get us out of this pit we’ve dug for ourselves. If he raises taxes and throws up tariff walls and opens the coffers of the DNC to bribe-money from the special interest groups against whom he has (somewhat disingenuously) railed during the campaign trail, then he will almost certainly reap a whirlwind that will make Katrina look like a balmy summer zephyr.

Obama has in him—I think, despite his sometimes airy-fairy “We are the people we have been waiting for” silly rhetoric—the potential to be a good, perhaps even great leader. He is, it seems clear enough, what the historical moment seems to be calling for.

So, I wish him all the best. We are all in this together. Necessity is the mother of bipartisanship. And so, for the first time in my life, I’ll be pulling the Democratic lever in November. As the saying goes, God save the United States of America.”

Fierce partisan politics new? No, think 1800 U.S. Presidential elections

Robert Novak writes eloquently about the origins of partisan politics.  This opinion appeared in The American Spectator —

“AS THE ELECTION of 1800 approached, Federalists commonly stigmatized the rival Republicans (forerunners of today’s Democrats) as “Jacobins” to associate them with the French Revolution, and Republicans called Federalists “monarchists” to associate them with England. Thomas Jefferson, the Republican vice president, publicly accused his 1800 presidential rival John Adams, the Federalist president, of “political heresies.” The outspoken First Lady Abigail Adams wondered whether God would protect America if it elected Jefferson, “who makes no pretension to the belief of an all wise and supreme governor of the world.”

Republican journalists were thrown in prison under terms of Adams’s Sedition Act. Connecticut publisher Charles Holt was jailed and his newspaper shut down for much of the 1800 campaign. Pennsylvania printer Thomas Cooper was jailed for six months.

Jefferson hired the scandalmonger James Thomson Callender, who had uncovered sexual and financial improprieties by Federalist Secretary of the Treasury Alexander Hamilton, to print “vitriolic assaults” on Adams. (Callender, because of a dispute over payments due him, later turned on Jefferson during his presidency and exposed his fathering of a son by the slave Sally Hemings. In 1803, Callender’s body was found floating in the James River.)

For the 21st-century reader, however, even these personal smears are eclipsed in the pattern of deceit and duplicity practiced by our Founding Fathers during a 16-month battle over selecting federal electors that amounted to a long primary season. Larson reports that Hamilton, the leader of the High Federalists, “told friends as this campaign got underway that he could no longer support” the re-election of Federalist John Adams. That began Hamilton’s efforts to elect instead Adams’s vice presidential running mate, Gen. Charles Cotesworth Pinckney. “I will never more be responsible,” said Hamilton in a letter to the High Federalist House Speaker Theodore Sedgwick, “for [Adams] by my direct support even though the consequence should be the election of Jefferson.”

Hamilton was taking advantage of a constitutional flaw (corrected after this election). Each member of the Electoral College had two votes, but could not designate which vote was for president and which for vice president. Thus, Hamilton was trying to get enough Federalist electors to “drop” Adams so that Pinckney would end up with more votes and be elected president.

Aaron Burr, Hamilton’s bitter rival in New York politics and Jefferson’s running mate, was playing similar tricks on the Republican side. Called by Jefferson “a crooked gun…whose aim or stroke you could never be sure of,” Burr was manipulating the Electoral College. He misrepresented the northern electors as voting for Jefferson and Pinckney, so that southern electors would not have to drop Burr’s name and could vote for both Jefferson and Burr and still elect Jefferson president.

Just as Burr had calculated, he and Jefferson tied, each with 73 electoral votes, sending the election to the House of Representatives. Once in the House, Burr hoped to win over enough Federalist Congressmen to be elected president. Federalist Speaker Sedgwick called Burr “a profligate without character and without property — bankrupt in both.” Nevertheless, Sedgwick preferred Burr to Jefferson on religious grounds: “He is not under the direction of Virginia Jacobins. He is not a declared infidel.”

ALEXANDER HAMILTON was one Federalist who entertained no such mitigating views, calling Burr “the most unfit man in the U.S. for the office of president. Disgrace abroad, ruin at home are the probable fruits of his elevation.” (This passion four years later produced a bloody duel that destroyed Burr’s political prospects and ended Hamilton’s life.). Hamilton made those comments to 33-year-old Federalist Congressman James Bayard, who controlled Delaware’s vote as the state’s only House member and had suggested Burr “is willing to consider the Federalists as his friends and to accept the office of President as his gift.”

That was the line taken by other Federalists in the House. The six states whose House delegations were dominated by Federalists all voted for Burr. Jefferson carried eight states, with the remaining two states split and therefore not voting. So Jefferson was one short of the nine states needed for election.

The deadlock lasted for four days and 33 ballots, and endangered the peaceful transfer of power. When Federalists began to talk of an “interim” appointed president (who would be a Federalist), Jefferson threatened Adams with “resistance by force and incalculable consequences.”

Young Congressman Bayard had tried to pull a handful of Republican Congressmen to swing three states for Burr, but failed. He then decided to abandon Burr, handing Delaware and the presidency to Jefferson. Did he do so to avert a constitutional crisis? In truth, Jefferson agreed to retain two federal port collectors sponsored by Bayard. “You are safe,” Bayard wrote to one of them.

The federal union had averted an early crisis, but not everybody was happy. Abigail Adams wrote to her husband: “‘What an inconsistency,’ said a lady to me today, ‘the bells of Christ Church ringing peals of rejoicing for an infidel president!'” We do have a long tradition of finding it easier to be partisan than conciliatory.”

Anna Quindlen on Race, Affirmative Action, and McCain and Obama

Anna Quindlen (in Newsweek) writes thoughtfully and interestingly on the role of race and affirmative action in American life and particularly in the political world.   Here is part of that article —

“The fallacy at the heart of most discussions of affirmative action is twofold: that it replaced a true meritocracy, and that it means promoting the second-rate. The meritocracy theory requires us to believe that for decades no women and no people of color were as qualified as white men, who essentially had every field locked up. Belief in the ascendancy of the second-rate requires us to demean the qualifications of countless writers, jurists, doctors, academics and other professionals who gained entry and then performed superlatively. Part of the tacit deal for most of them was not that they be as good as their lackluster white male counterparts, but as good as the best of them.

“As good as the best of them” might well have been Barack Obama’s slogan as he rose to be editor of the Harvard Law Review, faculty member at the University of Chicago Law School, state representative and U.S. senator. It is easy to see all the ways in which his race could have played a part in his rapid rise, but not necessarily in the way his opponents might suggest. Being an outsider probably taught him how to work well in two worlds, the world of those who take their place of primacy for granted and the world of those who have no such place, or who have to fight for it twice as hard.

Much has been made by Senator McCain’s supporters of his history as a survivor of a Viet Cong prison camp, of the broken bones and psychological onslaughts that he withstood for five long years. They argue that such an experience builds character. They should also take note of the challenges faced by a black man in America, challenges that have built Senator Obama’s character. These may be harder to quantify than imprisonment and torture, but they are onerous in a different and inescapable way.

The McCain forces have accused the Democratic candidate of injecting race into the campaign. That’s silly. The man is black. His candidacy is indivisible from that fact, given the history and pathology of this country. When Senator Obama said that he did not look like the guys on our currency and that his opponents were likely to portray him as Other, he was stating the obvious. Perhaps he was also pointing out that, despite efforts to maintain the status quo by generations of conservatives, this remains a nation so progressive that an American from a group once held as personal property could become president. The suggestion of something untoward was pandering to stereotypes and fear. Senator McCain was playing the Caucasian card.

Does that sound offensive? I suppose it is, just as offensive as styling the race card a pejorative. But any black man or woman in America has heard worse. If people make assumptions about you simply on the basis of your appearance all your life, assumptions ranging from criminality to sloth to unearned opportunity, it can make you bitter and hard and cynical. That none of those things is part of the Obama character means that he has turned his particular version of the race card into an ace and is using it to play with the full deck. That is not a deficit. It is an advantage.”

John McCain’s odds of winning the U.S. Presidency are small? Is the national mood too dim?

Clive Crook (clive.crook@gmail.com) of The Financial Times opines that john McCain’s odds of winning the U.S. Presidential elections are relatively small.  Here is the opinion —

“…Alan Abramowitz, a politics scholar at Emory University, has shown that summer head-to-head polls convey almost no information about the forthcoming election. (Subsequent head-to-head polls are not much better.) Instead, he has a simple “electoral barometer” that weighs together the approval rating of the incumbent president, the economy’s economic growth rate and whether the president’s party has controlled the White House for two terms (the “time for a change” factor). This laughably simple metric has correctly forecast the winner of the popular vote in 14 out of 15 postwar presidential elections.

The only exception is 1968, when the barometer (calibrated to range between +100 and –100) gave Hubert Humphrey a wafer-thin advantage of +2; he lost, with a popular vote deficit of less than 1 percentage point. The barometer not only picks winners but pretty accurately points to winning margins, too. In 1980, Jimmy Carter had the biggest postwar negative reading (–66); Ronald Reagan beat him by nearly 10 percentage points.

President George W. Bush’s net approval rating (favourable minus unfavourable) is currently –40; the economy grew at a 1 per cent annual rate in the first quarter; and Republicans have had two terms in the White House. Plugging the numbers into Mr Abramowitz’s formula gives the Republican candidate a score of –60, about as bad as it gets: second only to Mr Carter’s in the annals of doomed postwar candidacies. The barometer says Mr Obama is going to waltz to victory.

The unsettling thing about this way of predicting the outcome, of course, is that it does not matter whether the Democratic candidate is Mr Obama or Hillary Clinton – or Joe Biden or Dennis Kucinich, for that matter. The Republicans’ choice of Mr McCain was equally beside the point. On the merits, one candidate may be much better than another – a separate and endlessly interesting question. When it comes to predicting the result, the barometer says that as long as the incumbent is not running, it makes no difference.

Are there special factors that could throw the calculation off? No doubt, and this year one above all cries out. Mr Obama would be the first black president, a possibility the barometer has not yet had to contemplate. Who knows what difference his colour will make, whether it will help him on balance or hurt him. History suggests neither; that the choice of candidates, their strengths and weaknesses and the way they fight their campaigns, matters less to the outcome than one might suppose and infinitely less than the political commentariat is honour-bound to maintain.”

Is the electorate composition shifting in favor of Democratic party?

Rhodes Cook, a well-known, political analyst states that there is a perceptible shift in the composition of the electorate that appears to help the Democratic party at the margin. Here is the analysis —

“…. this overall trend–Democrats up, Republicans down–is also mirrored in many of the states that already have been identified as battlegrounds for 2008. And with only a comparative handful of votes needed to swing key states such as Iowa and Nevada the Democrats’ way, the latest registration numbers can only fuel the party’s considerable optimism.

There are a variety of reasons why the Democrats are gaining new voters, starting with demographic change. In fast-growing Nevada, for instance, a 4,400-vote registration advantage for the Republicans in November 2004 has been transformed into an imposing registration edge of more than 55,000 for the Democrats. That represents nearly three times Bush’s margin of victory in Nevada four years ago.

In states that voted near the end of the primary calendar this spring, the spirited Democratic contest between Obama and Hillary Clinton brought tens of thousands of new voters into the party’s ranks. In Pennsylvania, there was a Democratic registration surge of more than 300,000 in the six months between November 2007 and the April presidential primary. In Oregon, Democratic registrations increased by more than 100,000 between last November and the May primary. In both states, which went narrowly for Democrat John Kerry in 2004, the number of registered Republicans conspicuously dropped during the same per.

And there are states where Democratic gains seem related less to the excitement of the recent Clinton-Obama contest than to the general nature of the times – in which President Bush enjoys little support beyond the GOP base and the appeal of the Republican “brand” is questioned even by party loyalists.

Case in point: Iowa. At the time of the state’s precinct caucuses in early January, the number of registered Democrats across the state was essentially unchanged from the 2004 election. But in the months since the caucuses, Democratic registrations have surged by nearly 70,000, while the Republicans have gained barely 7,000 voters – all this in a state that Bush carried in 2004 by barely 10,000 votes.

The same Democratic registration trend is evident on an even larger scale in California. Since the state’s presidential primary in early February, Democratic registrations have mushroomed – growing by more than 300,000, while Republican registrations have increased by just 15,000. The disparity underscores why California’s 55 electoral votes should be safely found once again in the Democratic column this fall.

To make matters worse for the Republicans, they continue to follow the path to obscurity in much of the Northeast, an area of the country where just a half century ago the GOP’s then-large moderate wing was rooted. Nowadays, barely one quarter of all registered voters in New York are Republicans. In Massachusetts and Rhode Island, the GOP share now is down to 12 percent or less.

The saving grace for Republicans is that this does not appear to be a “base” election like the two won by George W. Bush. In 2000 and particularly 2004, both parties emphasized registering and turning out their own voters. This time, independents will be extremely important – a group that comprises roughly a quarter of the voters in party registration states. McCain’s longtime appeal to independents gives him an opportunity to offset losses caused by a shrinking GOP base.

Voter registration tallies will be updated on a regular basis between now and the election as both parties and their allies become fully engaged in registering new voters. But in a campaign that is already uphill for McCain and the Republicans, this is another important area where they will be playing catch up.”