What Today's Buggy Article Is Up To
In this, the 5th buggy article in a strung-out series on Professor Robert Pape's nationalist theory of suicide terrorism, our focus narrows and attaches itself determinedly, with single-minded thrusts, on the statistical effort by Pape to test his nationalist theory of suicide terrorism by means of a logit regression model.
Pape himself makes various claims for the data-base that he himself gathers, codes, and organizes and for the estimated effects of the logit models that he then specifies to analyze the data-set. It's on pages 96 and 97 of Dying to Win that these claims first appear:
To test my theory, Pape writes, I employ a methodology that combines the features of focused-comparison and statistical-correlative analysis using the universe of foreign occupations, 1980-2003. Correlative analysis of this universe [by means of a logit model] enhances confidence that my theory can predict future events [of suicide terrorism] by showing that the patterns predicted by the theory actually occur over a large class of cases. Detailed analysis of historical cases enhances confidence that the conclusions found in the larger universe are not spurious --- that is, that my theory accurately identifies the causal dynamics that determine outcomes.h
Note Pape's Words Carefully
The correlative analysis of what he identifies later as a logistic regression model using logit analysis --- the two terms are roughly interchangeable --- will do two things: it will enhance confidence . . .
That his nationalist theory of suicide terrorism has accurately identified the causal dynamics of suicide terrorism that occurred during all foreign [military occupations] between 1980 and the end of 2003 --- which turn out to be 58 such cases in all, or so he later insists. Note in passing that these 58 cases are then coded and organized into data that correspond, as we'll soon see soon, to the dependent and independent variables in his logit model(s) and become its sample selection that the model than is run on.
And that, in turn --- thanks to these accurately identified causal dynamics --- his theory can predict suicide terrorist attacks.
The Buggy Take Here
As we'll see, Pape is flat-footedly wrong. His data-set and the estimated logit model's reported outcomes --- to the extent Pape provides any information at all about them (p. 99 and fn. 43, p. 294) --- do not enhance our confidence whatsoever. They do the exact opposite, and for reasons set out at length in today's substantive argument.
How That Buggy Argument Will Unfold in Three Parts.
In Part One, We'll summarize anew Pape's nationalist theory of suicide terrorism --- especially its alleged causal links --- and cap the summary by reproducing the schematic diagram that appears on p. 96 in Dying to Win. Even if you've read the previous buggy article that performed both of these tasks, the comments here will likely help jog your memory again.
Enter Part Two. We'll examine Pape's logit model in depth there. In particular, we'll try to reproduce how Pape likely specified it, assuming he followed a proper statistical procedure, including its variants. We'll also look at his very scantily reported regression outcomes. Remember, with such stingy information, some conjecture is inevitable --- but most likely, sound conjecture.
Part Three is the key to the buggy argument, a matter of analysis and criticism --- mainly the latter.
Two More Preliminary Comments To Keep in Mind:
1. If you have no understanding of linear regression modeling and how logistic regression differs from it, be sure to read the previous buggy article. It sets out a series of fairly straightforward, easy-to-grasp comments about the nature of linear regression, how logistic regression differs from it, and why Pape had to use the latter to test his theory of suicide terrorism: the use of a binary qualitative variable as the dependent or output variable --- what he3's trying to account for, given a set of independent or explanatory variables that he specifies in his logit model. (Remember, logistic regression and the logit model (or logit analysis or transformation) are roughly but not exactly interchangeable terms).
2) More to the point, the buggy analysis that will unfold here cannot avoid using a fair amount of statistical terminology. You may have already suspected this. Despite prof bug's best efforts, there's no help for it. To avoid the technical terms, it would be necessary to expand the article several-fold in length order to clarify each point effectively.
Even so, prof bug urges you not to give up if some of the criticisms he makes of Pape's statistical work elude your understanding. Keep reading on. The main problems in any case derive from fairly easy-to-understand matters, specifically:
* The defects in Pape's data-set, which is his sample selection for the logit model he specifies, are numerous, and it will be easy to grasp them.
* Pape claims for what his logit model will do --- enhance our confidence in his theory of suicide terrorism, give us confidence about its predictive value for the future, and so on --- are not at all justified by his statistical work, starting with the data-set itself and with all the problems that hound his model's specification, interpretation, and reported statistical tests. The hollow nature of these claims will easy to grasp as the buggy argument unfolds.
*Finally, we'll end by some explicit and more general problems that beset regression modeling, linear or otherwise, in the social sciences . . . particularly claims made by the slew about how such modeling establishes causal links and predictive powers of the various theories that social sciences claim to be testing.
PAPE'S THEORY OF SUICIDE TERRORISM SUMMARIZED ANEW
Chapter Six Takes Center Stage
It's in chapter six of Pape's book (pp. 79-101) that a full-fledged theory of suicide-terrorism emerges for the first time.
To Pape's credit, his "nationalist theory of suicide-terrorism is set out clearly there and discussed in useful detail. It consists of four component parts --- a set of independent or explanatory variables if you prefer, all intended to clarify the circumstances that will likely "cause" suicide attacks to be launched. Pape refers to them as "causal" influences. It's those four that form the basis of his logistic regression exercise, a logit model intended to "test" his theory's postulated "causal" pathways that lead to suicide terrorism.
The latter pathways, as you'll see, are juggled around and summarized by Pape himself in a schematic diagram that is found on p. 96 and that will be reproduced at the end of this buggy part.
The Four Independent ("Causal") Variables Sketched In
Sketched in is the key term here.. The four independent (estimating) variables are set out in a fast, top-skimming manner, at any rate for now. The aim is to give you a rough working idea of each variable, followed by the schematic diagram; nothing more . . . not for today anyway. It's only in the next buggy article, remember, that each of these alleged causal links and pathways will be delved into at length.
As for the use of quotes around causal, they're there purposefully. Pape's theory isn't a scientific theory in a strong sense we'll see, rather more like a wiring diagram with arrows drawn in different pathways to signify hoped for causal pathways among the variables. And very few regression models, linear or non-linear, cannot test causally any theory except a few in the natural sciences.
As we'll also see, even the most formalized economic regression-models running on strictly quantitative data that can be randomly sampled have never been tested accurately for any causal links --- even, believe it or not, neo-classical demand theory, the core of microeconomic theory. For a good 75 years now, econometricians have been trying to test the various empirical variables in demand theory --- the symmetry of (compensated) Slutsky matrix, the homogeneity of degree zero of individual and aggregate demand functions, and Walrus Law (adding up or Engle aggregation), and guess what? Almost all tested statistical models have produced results that contradict the theory of demand. (See the discussion of this on pages 96-98 in D. Wade Hands, Reflections Without Rules; Economic Methodology and Contemporary Science Theory
[Cambridge, 2001). The quoted terms are from pp. 96-97)
So Where Are We?
All these points will be clarified later on in Part Three. For the moment, fasten your attention on the independent variables that comprise Pape's nationalist theory of suicide terrorism.
1) There's an alien military occupation on a territory of another people --- a national or ethnic community that resents it --- by a democratic country. Pape says that he can find no cases where suicide terrorism has been used against non-democratic occupiers.
Note in passing that this particular variable doesn't enter into Pape's logit model per se. It's not even an exogenous variable, operating from the outside on the logistic regression estimates of the outcome variable and the coefficients of the estimating variables. Instead, by definition, Pape limits his data-collection and coding to only democratic occupying countries . . . wrongly so, as we now know and as will be clarified in part three.
2) Sooner or later, a noticeable religious conflict between the occupying power and the occupied population has to develop that aggravates the locals' fears and resentments of the occupier and leads to demonizing its society and civilians. In turn, the demonizing will celebrate national martyrdom by armed rebels against the evil occupier, justifying the use, if need be, of suicide terrorism . . . the latter seen by more and more of the occupied people as a last desperate resort at coercing the alien democratic country to withdraw its military forces.
Note: though prof bug has striven hard to refrain from critical comments of Pape's theory until the next buggy article in this series, he can't, alas, fully adhere to this self-denying ordinance here.
More specifically, if Pape is right, the fact that one religion --- Islam, which constitutes most of the suicide terror campaigns between 1980 and 2003 even in his flawed and understated data-set that appears on p. 15 of his book, and 7 of 9 such suicide-terrorist campaigns that he himself codes in appendix two (pp. 265-67) --- has a 1300 year-tradition of armed martyrdom suffered in jihad against infidels, with the suicide attackers instantly entering Paradise the second after they're killed, where they can now enjoy worldly pleasures for eternity without guilt, has nothing to do per se with such demonizing or justification of suicide attacks. Nor does it have anything to do with Islamist terror groups carrying out 90% of the suicide attacks between 1980 and 2003, at any rate when Pape whitewashed data-set is corrected(A table of all the suicide attacks by radical Islamist groups --- mainly against Muslim countries --- that Pape omits or conceals appeared in two earlier buggy articles: the one immediately preceding this one, and the 2nd one in this series on Dying to Win.
Then, too --- if there isn't something specific to Islam compared to, say, Christinaity or Judaism --- why is that Palestinian suicide terrorism against Israelis has been going on now for 11 years by now, yet not one Christian Palestinian has engaged in the more than 500 suicide terrorist attacks launched from the West Bank and Gaza? Christians comprise about 10% of the total Palestinian population. If it's nationalist fervor that is the driving force of suicide terrorism, why then wouldn't Christian suicide terrorists account for about 10% of the total? Yet far from that being the case, there's not one suicide attack that originated out of the Christian minority.
(Pape does show, revealingly, that 3 of the 41 sucide attackers operating in Lebanon in the 1980s --- almost all under the auspices of Hezbollah and with the attack directed at French, American, and Israeli forces --- were Christian. Revealing, too --- after supplying this informative statistic on p. 205 --- he provides a radically misleading pie chart that shows Christains comprising 71 % of the 41 bombers. Usually, 3/41 equals 7.3%, not 10 times that number.
Since Christians in Lebanon were about 50% of the country's total population, the odds that a Christian would participate in a suicide attack were roughly 1/6th of the odds that a Muslim would participate. Still, the most illuminating point is that there is no record of a Christian suicide terrorist in the Palestinian community since 1994, the year that Hamas --- richly financed by Saudi oil-money to further Wahhabi Islamism (not that Pape tells us this) --- began its suicide operations.)
All this, understand, before 2004.
If you add in the suicide terrorist attacks carried out by Islamist radicals since then --- including such target countries as Spain, Britain, Turkey, Egypt, Tunisia, Morocco, Indonesia, the Philippines, Iraq, Israel, and Russia among others --- you'll get hundreds more, not to mention the hundreds of attempted suicide attacks that have been thwarted by European, American, Russian, Arab, and Israeli security forces. Note that the buggy table referred to a moment ago will be reproduced later in this article, when we examine Pape's coded data-set itself)
Nor is that all.
Pape's view turns all the more fanciful if you consider the tremendous number of other murderous actions carried out by fervent, militantly enraged sections of Muslim populations against secular, Christian, Buddhist, animist, Hindu, or even Muslims designated by fundamentalist extremists as apostates --- world wide, mind you. None of this, apparently, has anything to do with the fact that 7 of the 9 suicide terrorist groups active between 1980 and 2003 that he himself identifies and codes to use as his data-set (sample selection) for his logit model were and are led and manned by Islamist extremists . . . not to forget, as two earlier buggy articles showed, that Pape decided to overlook another 9 suicide terrorist groups active in this period, all of them Islamic. You want clear evidence of the connection? Well, consider this: a table compiled daily by ReligionofPeace.com
shows that --- since 9/11fs massacres in New York and Washington D.C. --- there have been more than 2800 Muslim terrorist attacks around the world as of August 28, 2005.
To repeat: in the four years since 9/11, Muslim-inspired terrorism has resulted in 2800 different attacks against overwhelmingly civilian targets.
As if that weren't bad enough, a radical Islamist terror group in Algeria --- the Salafist Group for Preaching and Combat (GSPC) --- just announced (November 4th, 2005) that France is the Islam's enemy number one, the enemy of our religion, the enemy of our community and has urged jihad and Islamic martyrdom to attack it in various ways, including by suicide terrorism. Yet Pape is found in chapter 7 of his book claiming that Spain was attacked by terrorists allied with or sympathetic to al Qaeda --- with other NATO European allies threatened with similar attacks --- strictly for nationalist reasons, as punishment for participation in the Iraqi war. But then didn't France once occupy sacred Algerian territory, even if it withdrew and Algeria was given its independence in 1963? And if the terrorist groups, according to Pape, define some alien country as an occupier --- al Qaeda, for instance, seeing the US occupying Saudi Arabia between 1991 and 2003 --- that is what counts after all, isn't it? And if bin Laden refers to Southern Spain --- Andalusia --- as what was once sacred Islamic territory seized violently by Christians in the 14th and 15th centuries (even though Muslims seized all of Spain violently in the 8th century), then shouldn't Spain be considered occupied militarily by an alien democratic regime today . . . fully deserving to be subject to suicide bombings?
Presumably, too --- by the same logic --- if the Isalmo-fascist government of Iran (Shia in its religious extremism) openly declares it would like to see Israel wiped off the face of the earth, doesn't that also fit Pape's model of suicide-terrorism . . . probably, if Tehran's clerical fascists could get their way without retaliation, by future nuclear attacks? Actually, Pape says nothing about state-sponsored terrorism in the Islamic world --- for instance, Iran's sponsoring and financing Hezbollah from the early 1980s on, and later Islamic Jihad, and for that matter through its Syrian proxy (another dictatorship ruled by a Shi-ite group, albeit a small Alawite one); but then neither does he let it be known that all three terrorist groups have openly committed themselves to Israel's extinction too. Then, too, Saudi Arabia's lavish financing of extremist Sunni Islam, including some generous donations to al Qaeda by members of the Royal Family, seems irrelevant to Pape's concerns as well.
Oh well, that's scholarship on the frontier of nationalist explanations of suicide terrorism world-wide since 1980.
Really, in the end, does anyone besides Pape, quite a few gullible left-wingers everywhere, and several vocal apologists for Islamist extremism --- the latter only a minority of Islamfs 1.2 billion people --- believe that his nationalist theory is accurate about the clearly dominant role of Islamist extremism in the active terrorist campaigns world-wide . . . whether involving suicidal terrorism or not?
Note: In the NY Times
of November 18, 2005, an astounding article spelled out in chilling detail the fanatical, kill-crazy nature of Islamist extremists living in Italy . . . the ringleader alleged by Italian investigators to be the mastermind of the Madrid bombings in March 2003. The vicious hatred for all infidels, especially Christians and Jews, along with the glorication of gore, torture, and mayhem as their just due, comes through with alarming conviction that has nothing to do with any alleged occupation by the US or NATO allies of Iraq or any country --- rather, to do with a marked psychopathology that makes the cannibalistic Hannibal Lecter, played with rippling talent by Anthony Hopkins in the Silence of the Lambs
, look half-way benign. You'd have to probe the stench of Nazi leaders to find an apt comparison:
MILAN - Playing an Internet video one evening last year, an Egyptian radical living in Milan reveled as the head of an American, Nicholas Berg, was sawed off by his Iraqi captors.
"Go to hell, enemy of God!" shouted the man, Rabei Osman Sayed Ahmed, as Mr. Berg's screams were broadcast. "Kill him! Kill him! Yes, like that! Cut his throat properly. Cut his head off! If I had been there, I would have burned him to make him already feel what hell was like. Cut off his head! God is great! God is great!"
Yahia Ragheh, the Egyptian would-be suicide bomber sitting by Mr. Ahmed's side, clearly felt uncomfortable.
"Isn't it a sin?" he asked.
"Who said that?" Mr. Ahmed shot back. "It is never a sin!" He added: "We hope that even their parents will come to the same end. Dogs, all of them, all of them. You simply need to be convinced when you make the decision."
Unconvinced, Mr. Ragheh replied: "I think that it is a sin. I simply think it is a sin."
The blunt exchange is contained in an 182-page official Italian police report that has not been made public, but is widely available in court circles and frames the judicial case against the two men. "The Madrid attack was my project, and those who died as martyrs were my dearest friends," Mr. Ahmed boasted in one intercepted conversation. . . .
The sites are filled not only with calls for the destruction of Israel but also raw anti-Semitism. In one question-and-answer session with a Saudi sheik who is asked what suicide operations against Jews are allowed under Islamic law, the sheik responds that Jews are "vile and despicable beings, full of defects and wickedness." God, he added, "has ordered us to wage war against them."
But what the heck, there must be some territory somewhere being occupied by a democratic government oppressing Italian Muslims to inspire such bloodthirsty longing for slaughter and destruction of all Islam's enemies, right Doctor Pape? And he must be, on your logic, a community-minded altruist --- nothing more --- who's willing to slaughter infidels everywhere (oops, citizens of the military occupier) even as he makes, sooner or later, the ultimate sacrifice of his own life, right again?
3) Enter Pape's third independent variable.
Before suicide terrorism is resorted to as a final desperate effort at national liberation, the growing nationalist ardor and rippling hatred of the occupier has to spawn an armed rebellion --- whether non-suicide terrorist attacks or urban or non-urban guerrilla warfare --- carried out by national resistance-movements against the foreign country's armed forces and civilians. There haven't been any suicide terrorist groups that have emerged between 1980 and 2003 without a prior national rebellion already in progress, and they start suicide attacks only after these other forms of armed rebellion have failed . . . or so it seems (Pape isn't clear on this point).
4) And finally --- as his statistical testing of the theory reveals at the very end of chapter 6 --- there's a lack of noticeable concessions by the occupying power to the occupied people's desires for national self-determination, in which case suicide-terrorist groups will very likely materialize. If, though, concessions are offered that the locals judge as ensuring local autonomy or holding out a prospect of it in the future --- even though the local autonomy will fall short at times of full-fledged national independence and sovereignty --- suicide terrorism will either not occur or fizzle out.
Oops: Almost Forgot That a 5th Variable Sneaks In the Pape Theory
Specifically, at the very end of chapter 6, Pape adds a 5th independent or estimating variable to his theory --- or more accurately, to his logit model: timely concessions offered by democratic governments that work to forestall or end suicide terrorist attacks. No need to say anything more here on top of what we said earlier in this article The ad hoc nature of Papefs salvation-variable and all the murk and confusion that hovers around it --- Pape devoting an entire four substantive sentences to the topic --- look so contrived and embarrassing that Pape preferred, it seems, to rush on at the end of chapter 6 and add some hedging points about his theory and logit testing of it rather than dwell on what would be too plainly like excuse-making and rationalizations for the mediocre statistical results that he reported earlier on p. 99 in a 2x2 classification table.
A Schematic Diagram of the Pape Theory
On p. 96 of his book, Pape provides us with a diagram of his full-fledged theoryfs causal pathway. He allows for an alternative pathway running in the opposite direction as a control. What follows is taken directly from that page.
Pape's Model of Suicide-terrorism
1) Solid arrows represent the theory proposed in this book.
2) The dashed arrow --- running from rebellion to nationalism --- represents a casual path
that sometimes influences the production of national identity;
but that plays little role in determining when suicide-terrorism campaigns occur.
3) The dotted arrow represents a causal path that al-Qaeda and perhaps other terrorist organizations
hoped will occur, but that has not done so.
Needless to add, it's the causal pathways postulated by Pape's theory running from left to right that his logit model is intended to test statistically.
WHY PAPE USES A LOGIT MODEL FOR HIS TESTING.
Note: Several visitors to the buggy site have indicated that they wanted to see more clarification of non-linear logistic regression --- and the logit model it uses --- than they found in the previous buggy article.
Well, maybe understandably so. The long Part Two in that article dealt largely with the basics of linear regression. and though there were some comments about logistic regression, there weren't enough, apparently, to illuminate its basics. And so this part of today's article will concentrate much more on clarifying logistic regression, at any rate to the extent it's possible using fairly limited technical terms. It will do so, please observe, by moving back and forth between the general nature of logit models and Pape's use of them for testing purposes.
Observe something else.
By the time prof bug was finished setting out these clarifying comments --- using Pape's models as one point of reference, along with examples and interpretative remarks about logistic regression in general linked to other online sites --- it turned out to be a longer endeavor than he had anticipated. One reason: it took a few hours to run google searches to find some illuminating online articles at the basic level, look through them, decide what was relevant or not, and so on . . . following which prof bug decided after all the effort to throw in lots of comments taken from those articles.
The result: instead of adding the lengthy Part Three that probes deeply into Pape's logit models and analyzing all the problems that seem to beset them --- especially as tests of his nationalist theory of suicide terrorism --- it seems wiser to end today's article and continue with the analysis in the next article.
In the meantime, prof bug hopes that it won't take hours again to find some simple, easily understood comments and examples that illuminate a few technical issues that his criticisms of Pape's models entail . . . such as the need to jackknife his classification reports; the need to bootstrap the data-set; a diagram of the delta-p method for estimating the change in probability on the Y dependent variable --- in Pape's case, the probability that suicide terrorism will occur the next time a random variable is drawn from his data-set --- that occurs when one of his independent variables undergoes a unit change; the marginal probability change, which is the partial derivative taken along the probability slope etc. (Don't worry about these technical terms.
They don't figure in Part Two today, but they will in Part Three when it continues in the next buggy article.)
I. Enter Logit (Logistic) Regression
For his statistical test, Pape uses a logit (non-linear) regression model --- in reality, most likely four different models --- and runs it on a data-set that he himself assembled, coded, and classified.
Pape's aim is to use his logit models to estimate those factors --- mirrored in its independent or "predictor" variables by his set of independent variables --- that influence the behavior of suicide terrorism: in particular, when it's likely to occur or not. And as we'll clarify later, the final logit model he reports on has four predictor variables: b1
X (national rebellion), b2
Z (religious conflict), b3
XZ (national rebellion and religious conflict working in tandem --- known as an interaction variable), and b4
X (concessions offered by the military occupier to nationalist insurgents that prevents suicide terrorism from occurring). As with any regression equation, too, there is a constant term (a) on the same side of his various logit models' independent or predictor (or estimating) variables.
Technical Note: Some logistic regression specialists refer to the independent estimating or explanatory variables as "predictors." There's nothing wrong with the term provided you remember that it hasn't anything to do with the forecasting of future events --- rather, the ability to predict accurately whenever (to return to Pape's case) his logit models analyze his data-set or sample selection, one data-piece at a time, and suicide terrorism turns up as opposed to not turning up
Several points now follow logically, starting with . . .
(1) The Reasons Pape Couldn't Use Linear Regression
The answer: linear regression --- which we analyzed in the previous buggy article --- can't calculate accurately the relationship between a set of independent estimating variables (B1X1 + B2X2 + B3X3 . . .+BkXk), plus a (a constant term), and the resulting mean-values of the Y dependent or output variable when the latter is a binary qualitative variable of the sort Pape is concerned with estimating ---suicide terrorism occurs (set to a value of 1) or doesn't occur (set to a value of 0).
If Pape were to run his data-set or sample selection of 58 individual cases of military occupation between 1980 and 2003 --- which we'll analyze soon --- the resulting relationship that would emerge between the mean values of Y and the coefficient values of the predictor X variables wouldn't and couldn't be a straight-line. Rather, thanks to the qualitative binary nature of the output variable Pape is concerned with, the resulting relationship between the estimated distribution (relationship) between the Y outcome variable and the predictor X variables will be S-shaped . . . and not just S-shaped, but compressed by logit modeling between the values of 0 and 1.
The following diagram brings out the difference between the two kinds of resulting regression functions, linear and logistic.
Some Clarification About th Diagram Might Help:
(i.) Note that the logistic curve is compressed between 0 and 1 . . . which is equivalent to saying that the independent X variables of a logistic regression are seeking to estimate a rolling mean along the Y axis in probability terms. Probability terms, of course, are expressed between 0 and 1, and hence the compression.
(ii.) What the logit model does for logistic regression is three things:
1. It first transforms the probabilities in a logistic regression into odds: the odds formula is:
Odds = P/(1-P).
Hence if the probability of an event occurring --- say, the probability that suicide terrorism in Pape's logit models will occur, which is determined when his specified independent estimating variables are run on his data-set --- turns out to be hypothetically .67, then the odds are easily calculated as .67/1-.67 = .67/.33 = 2. (Strictly speaking, .67 probability or 2.0 odds would be the calculated mean along the non-linear S curve after all the data-points were analyzed by his model) This 2.0 odds value would be determined, you understand, by the independent variables and constant term on the right-side of his reported 2nd logit model:
i. nationalist rebellion, coded as a dummy variable (1 = rebellion occurs; 0 = if not);
ii. religious differences between the military occupier and the local national, likewise a dummy variable (1 = religious differences; 0 = no differences).
iii. and the two influences working in tandem --- (an interaction term of the two dummy variables just mentioned that is more complicated to estimate and interpret, and so we'll ignore for the time-being).
Assuming then that Pape's reported logit model passed certain statistical tests and his nationalist theory of suicide terrorism was a scientific theory, not just a wiring diagram of hoped for causal pathways --- both heroic assumptions --- we could "very loosely" claim that the odds are 2:1 that suicide terrorism will occur when we find a country in Middle East where there is a nationalist rebellion occurring against an alien democratic military occupier whose religion differs from the local nationals. But note swiftly: we could say this only "very loosely" --- nothing more. To be more accurate, we'd have to say something like the following:
"Out of the next 100 cases where a military occupation coincided with religious differences between the occupier and the occupied, suicide terrorism would likely occur in the aggregate a good 67 times."
The reason for the rephrased claim? In plain language, there's is no way to move from forecasting the aggregate or cumulative frequency of any regression equation, linear or non-linear, to forecasting a particular outcome or event. That's one reason economic forecasting, with all its sophisticated models of different sorts and use of strict quantitative data selected randomly, has such a mediocre record.
Even that rephrased claim wouldn't be fully accurate. No way. The only accurate thing that could be said with confidence after any statistical testing of a theory --- even a good scientific theory, which Pape's isn't --- would be the following statement (to stay with Pape's logit model):
"If Pape's sample selection of 58 cases were a fully random sample from a large number of cases between 1980 and 2003 --- say, 1000 of them --- and if we were then to draw a new random sample from the population of 1000 past cases, there'd be a "good chance" that suicide terrorism would occur 67 times."
As the term "good chance" makes clear, you couldn't even say that there'd be a total chance suicide terrorism would have occurred 67 times in that new random sample. Why?
Because that's the nature of statistical testing --- in this case regression analysis. Any inferences from the estimated "mean value" of Pape's sample selection of 58 cases back to the actual value of the mean of the large population --- remember, which hypothetically contains 1000 independent data-points or observations --- would be strictly "conditional" on that sample "mean" and on the estimated coefficients in his logit model that emerged from that sample. In the upshot, were Pape or anyone else to run the logit model again several dozen times using a new random sample each time, then he wouldn't very likely get the same results at all except by accident . . . though they might be close to one another.
The point being made here can be put more precisely in technical terms.
A stochastic or random variable (x) drawn from any sample selection --- yes, even a sample that is selected in fully probability terms from a large, naturally generated population of observations --- is never likely except by random chance to be independent of the error term in a regression equation. If some statistical theorists were to claim that they don't expect a stochastic x to be independent of e, then we can rephrase the claim in slightly watered-down terms: forget whether or not the relationship between a random variable x and the error variable in a regression equation isn't fully independent. All the same, it's very unlikely except by random chance that the conditional means of the errors around any regression slope will be zero.
For those who understand the previous paragraph but are doubtful about it or who just want a fuller clarification, see Richard Berk, Regression Analysis: A Constructive Critique (Sage Series in Advanced Quantitative Techniques 11; Sage, 2004), especially pages 69-74. A professor at UCLA, Berk is of the most prestigious statistical theorists of the last generation and is in the departments of both sociology and statistics there. The quoted words above are his, found on p. 72 of this illuminating book.
Keep in mind a key point about these last few paragraphs. They're purely hypothetical, making the best case for Pape's logit models imaginable. As Part Three will show, his actual models are a mess.
For one thing, his sample selection isn't drawn randomly from a large, naturally generated population (universe) of cases. It's a non-probability selection equal to his universe or population of cases, and that population is non-natural: its data are coded, assembled, and classified by Pape himself, and the population (which is equivalent to the sample selection run on Pape's logit models) is a small one, totaling 58 cases in all. For another thing, his self-selected sample selection solves one of the biggest problems for his nationalist theory by definition: it excludes any suicide attacks launched by Islamist terrorists against non-democratic governments in the Muslim world . . . the number of groups launching these almost as large (10) as the 14 different suicide terrorist groups in his sample selection. Then, to top it off, there are all sorts of technical problems with his logit models: very likely serious specification errors of the models, interpretation errors, and errors in statistical testing of the reported 2nd and 3rd models . . . all matters dealt with, to repeat, in Part Three of this or the next buggy article.
In summary, then --- to return to our main concern here --- the first thing logit analysis does is changes probabilities into odds.
2. The next thing a logit model or analysis does is transforms the odds into logged values
It does so by using a base of natural logarithms . . . compared to the conventional base of 10 that ‘re likely familiar with. We'll explain natural logarithms in a moment. For the time being, focus on the main point here: the logit model first transforms probabilities into odds, and this transformation removes the ceiling that probability analysis imposes --- the value "1" as its maximum. In turn, a natural log transformation of odds removes the bottom or floor of g0h on the probability scale as well.
To get a clear idea of this, look at the following relationship between odds and probabilities across a wide range:
Odds Compared To Probabilities
i. When odds change in the mid-range, .1 to 1 and 1 to 10, that is also the range where probabilities will also change most noticeably: from 9.0% to 50% and again from 50% to 90.9%.
ii. By contrast, outside that mid-range of odds --- .1 to 10 --- probabilities change much less, especially on the up side (increases in positive odds). Hence an increase of odds from 100:1 to 1000:1 increases the probability by about 8/10th of 1.0% (0.8902%). And if the odds increase than 10-fold to 10000:1, the probability itself increases by about 1/100th of 1% (0.0899%)
All of which brings us to natural logarithms.
The symbol ln --- which the logit model uses and which we'll set out algebraically in a moment --- stands for the natural log that uses the base e equal to 2.71828 rather than the base 10 of conventional logarithms. Why use natural logarithms? Well, as it happens, the natural log has certain mathematical advantages for analytical purposes in much of the sciences, in computer programming, and in certain social sciences . . . above all, the derivative of the natural log ex is equal to 1/x, while the derivative of log10x is more complicated: (1/x)log10 e.
If you have a calculator, the ln button will transform any number into its natural log. Oppositely, to exponentiate --- which means to find the antilog of a logged number or value (or its original pre-logged value) --- you enter that number or value and press the button labeled "ex . . . possibly (as with prof bug's bugged-out Sharp calculator) by entering the number, then pressing the = button, and then the ex. Note that the function of exponentiating --- returning a natural logged number (or value) back to its original value --- is used extensively by researchers once the logit model has finished estimating the coefficients of the constant and independent variables as logged odds to the base e.
That's because it's much easier to interpret and make sense of odds as opposed to log odds, , a point that will be clarified later.
3. Thanks to the fact that most of the probability changes occur in the mid-range of odds changes, the logit model --- which we'll express algebraically in a moment --- is able to concentrate on roughly mid-range odds and hence mid-range probabilities.
You can get a better idea of how the logit transformation is able to focus on mid-range odds and hence mid-range probabilities, where the relationship between the binary qualitative dependent variable --- in the Pape model, whether suicide terrorism occurs or not --- and the independent influencing or estimating variables is no longer non-linear but instead linear:
A Different View:
Probabilities and Transformations Into 1) Odds, 2) Log OddsSource: click here
Essentially, as the table shows, the logit model is able to concentrate in the middle range of probabilities, by limiting its analysis to a range in logged odds of -2.20 and +2.20 . . . equivalent to 0.10 and 0.90 in probability terms. The linear or straight-line result is brought out in the following diagram, which compares this range along the relevant part of the original non-linear logistic s-curve with the same range of an entirely normal (linear) regression function. For further illumination, compare this diagram with the original diagram set out earlier. (Note that the Y axis, where the qualitative dependent variable is measured, uses a probability scale between 0.0 and 1.0, and the X axis is the independent estimators --- "a" or the constant (intercept) term that figures in both linear and non-linear regression and the bX independent variable measured in log odds)
How The Logit Model Is Essentially Linear In Nature
Source: Click here
In summary, then, as the two tables and the latter diagram should make clear, what started out as a non-linear s-curved logistic regression function --- compressed between the range of 0.0 and 1.0 on a probability scale --- can be treated after the logit transformation as a linear equation that uses a scale of logged odds. To return to the Pape's model's hypothetical coefficient, we could now say the logged odds that suicide terrorism will occur --- given a one unit increase in nationalist rebellion while all the other independent variables are held constant --- will increase from 1.3 to 1.5. (Remember, this is strictly a hypothetical estimation, not an actual value reported by Pape . . . and in logged odds.)
If these three points still stump you, move on and look at the fairly simple algebra of the logit model that follows, along with the buggy comments and some quoted comments from logistic regression specialists..
(2) The Logit Model Expressed Algebraically
What follows is a restatement of all these points about the logit model's linear form, but in algebraic form. By the time you finish reading this and the the next few sub-sections, you should have a good working idea of what logistic regression amounts to when it uses a logit transformation or model.
Note that the equations and interpretations here are taken from a very illuminating online article by John Whitehead , mentioned in the last buggy article, that sets out --- far more clearly than any article prof bug has seen except for one mentioned in a few moments--- logistic regression basics. The math is kept to a bare minimum, and the article is a gem of concise and nicely illustrated writing. It does presuppose that you know the basics of linear regression. )
The "logit" model, Whitehead notes, solves these problems:
ln[p/(1-p)] = a + BX + e
[p/(1-p)] = exp(a + BX + e)
- ln is the natural logarithm, logexp, where exp=2.71828c
- p is the probability that the event Y occurs, p(Y=1)
- p/(1-p) is the "odds ratio"
- ln[p/(1-p)] is the log odds ratio, or "logit"
- all other components of the model are the same as linear regression.
"The logistic regression model is simply a non-linear transformation of the linear regression. The "logistic" distribution is an S-shaped distribution function which is similar to the standard-normal distribution (which results in a probit regression model) but easier to work with in most applications (the probabilities are easier to calculate). The logit distribution constrains the estimated probabilities to lie between 0 and 1. "
For instance, the estimated probability is:
p = 1/[1 + exp(-a - BX)]
With this functional form:
- if you let a + BX =0, then p = .50
- as -a - BX gets really big, p approaches 1
- as -a - BX)gets really small, p approaches 0.
3. A Trio of Buggy Clarifications:
(i.) Prof. Whitehead says that:
ln[p/(1-p)] is the odds ratio or "logit.
No, not really. It expresses instead the logged odds of the Y or outcome variable, not an odds ratio. An odds ratio is the ratio of two odds --- or, the same thing, the ratio of two probability ratios. As ln[p/1-p)] indicates, there is only one probability ratio here: the ratio of the probability that the outcome variable occurs --- in Pape's model, the probability that suicide terrorism occurs --- compared to its non-occurrence in the denominator. Whitehead's initial logit equation at the very top of his commentary,
ln[p/(1-p)] = a + BX + e
expresses the coefficient effects of gah the constant and of the independent variable BX in logged odd terms, along with the value of the error term (e).
But note quickly. The second logit equation that Whitehead immediately sets down afterward does express the odds ratios of the coefficients (parameters) of the constant term and the independent variables.
The reason: as you can see, that equation is an "exponentiated" version of the logit estimates . . . which means that it expresses the antilog of the "un-logged" odds on the left side of the equation and the various "un-logged" coefficients of the constant and independent variables on the right side, along with an error term or residual. These "un-logged" coefficients or values are not just easier to make sense of than logged values, but --- on the right side -- can be explicitly regarded as odds ratios for the independent estimating variables. (We'll illustrate this later at the end of Part Two when we specify Pape's logit models and see what first the logged odds estimates of the coefficients look like and how --- once they are then exponentiated (the original un-logged values or numbers are calculated and replace the logged versions) --- they express odds-ratios.
(ii.) The reference by Prof. Whitehead to the logit distribution constraining the estimated probabilities between 0 and 1 isn't accurate as it stands.
Before the logit transformation of a non-linear regression equation, you do get the s-curved cumulative distribution function or regression function diagrammed earlier it's constrained between the values of 0 and 1. That's because the estimating X variables of the binary qualitative variable (Y) are expressed in probability terms; and by definition, probabilities have to be expressed in the range between 0 and 1. Not odds though. And remember, the logit is a model that transforms the probabilities of a non-linear equation into first odds, then logged odds. In turn, as we know, logged odds aren't limited the way probabilities are between the values of 0 and 1.
On the contrary, they can run to negative infinity in one direction and positive infinity in the other. It's precisely this logged odds transformation that constitutes the logit:
ln[p/(1-p)] = a + BX + e
(ii-b) To put it differently, it's the logit on the left side that become the transformed dependent variable whose mean-value the logit estimators or independent variables on the right side seek to calculate . . . which is essentially a rolling (mean) average that comes to an overall mean-value only when the entire data-set or sample-selection is calculated. In short, the logit model is a link function. The constant term a and the various bXi estimating variables aren't directly estimating Y along the s-curve, but rather the logit link itself. If it were directly estimating Y in probability terms, the effect of any one independent bXi variable on Y's value would establish a non-linear relationship; and the effect --- or impact --- on Y in probabilities would vary according to the level of P.
By contrast, the logit link uses logged odds, and when the logit equation is run on a data-base (sample selection) and finishes by estimating the various overall coefficients (values) of b1X, b2, b3X, . . . bkX on the right side, the values are expressed in logged odds. If you then exponentiate and "un-log" the terms on both sides --- the Y and X variables (alone with the constant term) --- you have a relationship between Y and each X that is linear. (The relationship is liner in logged odd terms too, but logged odds are hard to make sense of.)
To get a clearer idea, note that the logit transformation of a logistic regression produces a logit curve --- a geometric representation of the logit link --- look at the following diagram:
"Note that the natural log is zero when X is 1. When X is larger than one, the log curves up slowly. When X is less than one, the natural log is less than zero, and decreases rapidly as X approaches zero. When P = .50, the odds are .50/.50 or 1, and ln(1) =0. If P is greater than .50, ln(P/(1-P) is positive; if P is less than .50, ln(odds) is negative. [A number taken to a negative power is one divided by that number, e.g. e-10 = 1/e10. A logarithm is an exponent from a given base, for example ln(e10) = 10.]"
This diagram, along with the commentary, is taken from what seems to prof bug --- after spending almost a full day looking for good introductory surveys of logistic regression and the logit model --- to be the best single introduction online that he found. It can't be too highly recommended for those of you who want to learn more about them: short and extremely well written, it's focused on first a simple regression that --- after some good exposition --- uses a case-study from medical science with a hypothetical data base for 20 heart-attack victims. The anonymous author explains it the case this way:
"Suppose that we are working with some doctors on heart attack patients. The dependent variable is whether the patient has had a second heart attack within 1 year (yes = 1). We have two independent variables, one is whether the patient completed a treatment consistent of anger control practices (yes=1). The other independent variable is a score on a trait anxiety scale (a higher score means more anxious)."
(iii.) Something else is relevant here .
As we also saw a minute or so ago, the logit model can be treated in a linear fashion because --- thanks to the logit transformation --- the odds are calculated largely in the mid-range of probability estimates while stretching the extreme ends of the non-linear s-curve and largely ignoring the possibility that a random variable with an extremely high value in probability terms will emerge out of a sample selection. It can largely ignore that possibility because the odds that a random variable whose probability is 99% (0.99) will turn up are roughly 1 in a 100. Even the odds that a probability with a value of 90% will turn up are only 1 in 10.
The analysis can be taken a step further. Whereas probabilities don't increase or decrease in a linear fashion --- remember, as the table above showed, a one unit change in odds leads to a much greater % change in probabilities near the mid-point of 50% than at 10% or 90% (0.1 or 0.10) --- odds change in a linear manner across the whole range.
Here is how these matters expressed by Fred C. Pampel in what is the best theoretical introduction to logistic regression --- which assumes that you know roughly the basics as found in the otherwise impressive Whitehead survey. (Pampel, Logistic Regression: A Primer (Sage, 2000), pp. 14 and 15.
It helps to view the logit transformation as linearizing the inherent nonlinear relationship between X and the probability of Y. We would expect the same change in X to have a smaller impact on the probability of Y near the floor or ceiling --- [of the sort of s-curve diagrammed above, prof bug] --- than near the midpoint.
"Because the logit expands or stretches the probabilities of Y at extreme values relative to the values near the midpoint, the same change in X comes to have similar effects throughout the range of the logit transformation of the probability of X. Without a floor or ceiling --- [because it transforms probabilities into logged odds, and odds can run on toward positive infinity unlike probabilities; prof bug] --- the logit can relate linearly to changes in X. One can now compute a linear relationship between X and the logit transformation. The logit transformation straightens out the nonlinear relationship between X and the original probabilities [of the non-logit transformed logistic regression.
"Conversely, the linear relationship between X and the logit implies a nonlinear relationship between X and the original probabilities [of the non-transformed logistic regression equation] . . ."
Earlier, we noted why the logit transformation (logit model) --- by using logged odds, not probabilities --- is able to do what Pampel just said very cogently: it "expands or stretches" the probability estimates at the end points of the logistic regression s-curve, while focusing on the mid-point ranges of logged odds and probability equivalents. The result is that the logit model has transformed the original logistic regression from a non-linear relationship between Y and the X variables on the right side of the equation into a linear equation.
4. But Note:
The key thing that Whitehead's exposition shows, though, still stands clearly: the logit transformation of a logistic regression does two things 1) it expresses the probability ratio that the dependent variable will be equal to "1" (or suicide terrorism occurring in the Pape model) as odds, not probabilities; and 2) expresses the odds on both sides of the equation as "logged odds" to the base e. In the upshot, the transformed logistic regression equation --- originally a non-linear equation --- can be treated as if it were like a linear equation. It does so because the logit --- ln(p / 1-p) itself --- becomes the dependent or output variable the independent variables on the right side of the logit-transformed equation estimates.
And, as we now know, the resulting estimates will be expressed as natural logged odds.
5. Finally, If You Believe You'd Like More Clarification of Logistic Regression, Then Try These Sources:
(i.) Those working at the introductory stage of regression equations should start with a surprisingly good little book on linear regression that prof bug wishes he could have had hold of when he started work in econometrics decades ago: Leo H. Kahane, Regression Basics (Sage, 2001). To the extent there's any book on statistics that's fun to read while still being illuminating, this is it.
(ii.) Then, once you think you've got a grip on linear regression, work your way through the anonymous article online we used a minute or two ago: the best single introduction online that he found.
(iii.) If you've gone that far, then try the John Whitehead article John Whitehead . Unlike the article just mentioned, it digs deeper into explaining some of the theory behind logit models --- including how they use maximum likelihood estimation, goodness-of-fit tests, and tests on the individual coefficients, and so on.
(iv.) If you work your way through these two articles, then tackle the short book --- theoretically challenging, but full of clear commentary and good common sense --- by Fred C. Pampel, Logistic Regression: A Primer (Sage, 2000).
(v.) Along the way, you ought to get access to a software program like SAS or SPSS or Stata or whatever that has some hypothetical examples in its logistic regression section that you can actually run. In doing so, follow a manual or the exercises set out by the program. For a very easy-to-follow book for SPSS users, click here.
(vi.) Those who already have a good grasp of the basics of logistic regression and logit modeling will probably find this widely used and highly regarded book as the best guide and reference: David W. Hosmer and Stanley Lemeshow, Applied Logistic Regression (Wiley, 2nd ed.) It's also not too long as statistical books go (about 330 pages). The analysis is all the more interesting because it uses case-studies in the health sciences, though at times --- as happens to theoretical statisticians --- the exposition cuts loose from earth and starts wandering into high-flying abstractions. Still, practically all the issues that enter into logistic regression are treated intelligently in this work, and those that aren't will most likely be of interest to professional statisticians, not applied researchers.
What's more, if you think you already know this book well, you probably have a better grasp of logistic regression than prof bug himself. (Working your way through some theoretical analyses put out by professional statisticians, you wonder who really reads them from start to finish other than other professional statisticians, and these specialists are notorious --- if they ever do applied work like most economists or political scientists or psychologists or health-scientists --- for climbing down from the cloud-covered penthouses of statistical buildings and descending into the brightly lit basements along with the rest of us. Or as a folk story has it, economists spend their lives groping around in a dark room looking for a non-existent black cat; econometricians are people who claim they've found that cat.
II. Back To Pape's Logit Models
Against this background, it's fairly easy to conjecture what Pape's logit models looked like --- which means the 2nd and 4th models, the only ones his scant information about them on p. 99 and in fn. 43 on p. 294 refer to.
The Only Logit Model Reported On Is Likely Pape's 2nd:
The report of this model's results, found on p. 99 in Dying to Win, comes in the form of a 2 x 2 classification scheme . . . which reflects the number of cases that led to suicide terrorism or didn't, as originally estimated or "predicted" by Pape's 2nd model and the actual numbers of these that conformed to his "predictions" or not. (No need right now to set out he classification scheme. That will be done in Part Three, along with how Pape misinterprets the results . . . or the number of successful "predictions" divided by the number that went astray. On top of that, quite apart from the misinterpretation, he makes a claim about how the model has enhanced confidence in the power of his nationalist theory of suicide terrorism to "predict" future cases of suicide terrorism. Wrong! All those results do --- leaving aside all the other problems of the logit model --- is confirm that Pape accurately classified the data in his also sample-selection . . . a sample that he himself created, coded, assembled, and classified to begin with; nothing more.)
Back to what we can infer about Pape's model from these reported results.
Remember, we're left speculating as to what Pape's four logit models looked like . . . including this second one. We have no choice. In decades of wading through hundreds of statistical studies, prof bug has never seen a book by a serious researcher who uses a statistical test for an important purpose and then does what Pape does: report it in such scant detail --- not even, as it happens, in a one or two page appendix tucked away at the end of his book --- that you haven't the foggiest idea what his statistical work amounted to. The very stingy details are referred to briefly on p. 99 and in a short fn. 43 on p. 294, and that's all. Then too, as we noted earlier, the fact that the book is aimed at a large non-scholarly community doesn't excuse the paucity of detail; there are, after all, three long appendixes at the end of Dying to Win that set out various data-sets Pape used in the substantive chapters of the book.
What can you conclude other than that Pape --- probably because of what certain readers of his ms. told him --- ended up less than enthusiastic about the logit modeling . . . and maybe even embarrassed?
Keeping All This In Mind, We Can Conjecture . . .
that Pape's 2nd logit model looked like this.
Y = ln[p/(1-p)] = a + b1X + b2Z + b3XZ
* ln[p/(1-p)] = the logit = the logged odds that suicide terrorism (its value equal to "1") will occur if someone were to pick a random case from Pape's 58 data sample and observe whether in fact it will occur or not the next time.
*a = the constant or intercept, whose final value Y would take takes when the X estimating variables all equal zero
*b1X = nationalist rebellion
*b2Z = religious differences between the occupier and the occupied people
*b3XZ = an interaction term for nationalist rebellion working in tandem with religious differences.
Note two things about the latter interaction variable.
1. There's nothing wrong with specifying an interaction term in a regression equation. It's even highly recommended for Pape to have created one if, in his baseline or 1st logit model, he found out that the influence (or "effect") of "nationalist rebellion" on suicide terrorism differed depending on the magnitude of the value that "religious differences assumed". In statistical jargon, the "focal" independent variable B1X is mediated by the "moderator" variable B2Z.
2. However . . . the interaction term almost certainly caused Pape to misinterpret its coefficient, and for a reason to be clarified in a few seconds.
Fasten For the Moment On This Buggy Claim That . . .
Pape very likely misinterpreted the interaction term's estimated coefficient or value. What then?
Well, if that variable's coefficient was misinterpreted by Pape, then, in turn, the interdependence of all the logit models' independent variables would entail that Pape's two other independent variables --- the non-interaction ones --- were misinterpreted too. These misinterpreted coefficients would further mean that Pape's reported p-values for each of his variable's statistical significance --- whether each coefficient wasn't due to random chance or other distortions at some level of probability (with a 0.05 likelihood of being due to that, or 0.01 or less) --- would be worthless too. Yes, worthless. And as it happens, those p-values are the only thing Pape actually reports on besides the classified results in a 2x2 table. In short, quite apart from all the problems that beset Pape's data-set for the logit models --- which we'll comment on in a few seconds --- the reported results or "effects" of his logistic regressions are probably worthless.
But, to repeat, all this is for latter.
Now For The Reason That Pape Very Likely Misinterpreted The Interaction Term
The specific mathematical reasons will be set out in Part Three. Of that, you can be sure.
It's enough to note here that a review of the 13 most prestigious economic journals found that 72 articles between 1980 and 1999 used an interaction term, and guess what? All 72 interpreted the interaction term accurately, and so all the estimated coefficients in their models --- logit, probit, and tobit --- turned out to be wrong, as did their statistical tests. (See Edward C. Norton) If 72 economics articles out of 72 incorrectly used an interaction term in their authors' non-linear regression models, what would you be willing to wager that Robert Pape, a political scientist, avoided their mistakes?
A Few Others Points To Note About This Likely 2nd Logit Model Pape Specified
(i.) Remember, the estimated coefficients of the independent or predictor variables in Pape's 2nd model would be set out in logged odd terms.
Suppose, for instance, it turned out that the coefficient for nationalist rebellion was 1.39. That means that when all the other independent variables are held constant, a one-unit increase in b1 would increase the logged odds that would suicide terrorism would occur by 39%, given Pape's sample selection of 58 cases.
Still, as we've noted, logged odds are hard to make sense of. They can be much more easily understood if we exponentiate his hypothetical example --- which means to find the antilog (the original un-logged number or value) --- and transform them into ordinary odds. In our example, the coefficient of b1 would now be expressed as 4.0. (You can try this on your calculator: enter 1.39 and hit your button that reads ex. It will report 4.0, the antilog.) Recall now that the exponentiated coefficient --- the coefficient of the antilog --- expresses not just an odds but an odds-ratio. It then becomes easy to interpret what the 4.0 b1 signifies for this hypothetical case: specifically, the odds that suicide terrorism will be resisted violently by nationalist rebels are 4.0 times greater than when nationalist rebellion doesn't occur.
A clarification for statistical cognoscenti intrudes here. The hypothetic coefficient "4.0" is a point-estimate. Though it's always a good idea for a researcher to report the tested confidence intervals around a point-estimate arrived at in linear regression, it's doubly important to do this for non-linear logistic regression. Especially for large coefficients, the confidence intervals --- or margin of error around the point-estimate --- can range very widely . . . often by 20 or 30 intervals.
Did Pape estimate and test confidence intervals around his logit models' effects or coefficients? If he did, he certainly didn't report on them. For that matter, he didn't report anything about his coefficients whatsoever other than to claim --- almost certainly wrongly, given the interaction term in his models --- that they passed an (unknown) test for their statistical significance.
(ii.) Suppose someone would be interested in transforming these odds back into probabilities, the original measure used in logistic regression before logit analysis. Well, there are ready-made formulas for doing this just as we've seen that probabilities can be transferred into odds. Here's the easiest one to make sense of:
Pi = ea + biXi 1 + ea + biXi
This looks more formidable than it is. What the equation requires you to do is take the "logged" values of "a" the constant variable and of any "b" coefficient --- say, nationalist rebellion in our hypothetical case, which is 1.39 (the natural log of the odds effect of that variable) --- and add them together in the numerator and then do the same in the denominator. Then exponentiate the resulting sums in the numerator and denominator . . . which means, recall, finding the antilog or un-logged value, not forgetting to add "1" to the antilog in the denominator. You now have un-logged sums on top and bottom, and all you have to do is divide the numerator by the denominator --- and voila! you are back to probabilities! It's the existence of "1" in the denominator, of course, that ensures that the resulting probability will always be expressed as a value between 0 and 1. (There's actually a simpler formula, but it requires you to remember to take the "negative" antilog in the denominator, and even prof bug has forgotten to do this on occasion, leaving him baffled as to why his SPSS software had suddenly gone haywire and calculated the clearly wrong probability.)
But wait! Once you're back to probabilities . . .
you're no longer on the linear logit link and are instead back to the non-linear relationship between the X independent variables and the qualitative binary Y outcome variable. This creates a problem. Any logistic software like SPSS, it's true, can easily calculate probabilities using the formula just set out, but where exactly along he changing slope of the s-curve logistic regression function should the probabilities be taken? The curved slope is always changing, sometimes rapidly, sometimes slowly --- but always changing. So where exactly?
The dilemma can be expressed mathematically.
In non-linear regression, the changing slope of the regression function is calculated as its partial derivative, the formula for which is the following:
@P/@Xk = bk* P*(1-P)
Actually, this is a piece of cake to interpret: the partial derivative --- or marginal or instantaneous effect of a changing point along the s-curve --- is arrived at by multiplying the logistic regression coefficient by the probability at that point and 1 minus the probability or P*(1-P). (Note that "@" is as close as Prof Bug could get to reproducing the symbol for the partial derivative without some convoluted HTML formatting.)
As you can see though, the partial derivative (marginal effect) varies directly with the value of P*(1-P). As Fred C. Pampel notes --- he's the author of that terrific theoretical survey we mentioned earlier, Logistic Regression: A Primer --- the effect of bX (in terms of the logit logged odds) will translate into different probabilities depending on the level of P itself . . . which means whereever along the s-curve regression line that marginal change is calculated. "The effect will be at its maximum when P equals .5 since .5*.5 = .25; .6 *.4 = .24; .7*.3 = .21, and so on . . . " (p. 25).
So where, then, should P be calculated? Needless to add, this sets off a debate among statistical theorists. Some say the partial derivative or marginal probability should be taken at the "mean" of the s-curve; others say that you should calculate several points --- which ones? --- then find the average; others come up with partial derivatives of the umpteenth order using complex information matrixes that take six pages to express and look like something that a lunatic mathematician using Hal-I --- the paranoid computer in Stanley Kubrick's Space Odyssey film --- would produce in his padded cell at 3:00 A.M. when he's not on his medicine and had just taken another swig of Draino.
Except when a researcher has a solid rationale, it's almost always better to interpret odds and odds-ratios in logit models, and test them statistically using odds, rather than translate these linear coefficients into non-linear probability terms.
As you'll see in Part Three, the informative article by Edward C. Norton, Hua Wang, and Chunrong Ai have written about interaction terms in logit (and probit) models have a very solid rationale to deal with probabilities, not odds and odds ratios. As they argue convincingly, the interaction terms are invariably misinterpreted --- whether the terms are for two interacting dummy variables as found in Pape's 2nd and 4th logit models, or for quantitative variables, or some mixture. Odds ratios just won't calculate properly in such logit models. Instead, you have to use probabilities --- but with a formula that would require Pape to calculate the double discrete differences (with respect to bZ) of the single derivative (with respect to bX) . . . where bZ stands for religious difference (the moderator variable) and bX stands for the focal variable, nationalist rebellion.
For those interested and can't restrain themselves until Part Three, here's the formula that Norton and his colleagues calculate for the interaction of two such dummy variables is:
The Stata Journal, 2004, 4(2): pp. 103-116, and is very illuminating if you have some decent mathematical background.) Just recall what we said earlier: the chances that Pape correctly interpreted his interaction term in his reported 2nd and briefly mentioned 4th logit models are next to zero . . . in which case, so we'll see in Part Three, his other coefficients are dubious, his use of the z-statistic (the Wald test) for their statistical significance, and so on are not just dubious but unreliable.
(iii.) Pape refers, briefly and with abrupt suddenness at the bottom of p. 99, to a final logit model that he specified and ran on his data set . . . his 4th (so we can conjecture).
It adds a 4th independent variable to the 2nd model for "concessions" . . . offered to nationalist rebels by the government of the military occupier that prevents suicide terrorism from occurring. It's this salvation variable that saves his reported 2nd logit model from mediocrity. . . or so we'll see; but it looks like it was pulled out of a hat like a magician's rabbit and has almost as little to do with his nationalist theory of suicide terrorism as the rabbit would.
In fact, before p. 99, about the only references to concessions in Dying to Win are how 13 past suicide terrorist campaigns that ended by 2003 had elicited in about half the cases certain gains from the military occupying governments --- more specifically in 7 of those campaigns. Pape then tells us on the bottom of p. 64 that these gains were reflected in "significant" policy changes by the target states that were in line with the suicide terrorists' goals. The next couple of pages relate how the success on this score has contributed mightily to the burst of suicide terrorist groups around the world.
But note quickly three things:
1. Nothing is said by Pape about what the timely concessions made to ongoing suicide terrorist groups look like --- their nature, their magnitude, and any variation across different suicide terrorist cases. It's all left very mysterious, this salvation variable.
2. More curiously, the "significant" policy changes that Pape refers to on p. 64 --- those that either prevent or end suicide terrorism --- turn out, in his own words, not to be significant at all for the governments that offered them. At most, as he grants, suicide terrorism has been ineffective in coercing occupying governments no matter how many suicide attacks have been launched against any one of them. If governments have decided to offer "timely" concessions, they have been not compromised basic national security or national wealth.
3. The most curious point remains to be set out. If "concessions" are so important to Pape's logit modeling as a test of his nationalist theory of suicide terrorism, why is there at most one very brief paragraph about them on p. 94 in chapter 6 before they are suddenly conjured up as a 4th and conclusive independent variable? The role of concessions doesn't appear anywhere in the talk about causal pathways for 31 pages until it's thrown in, suddenly, on p. 94. What's odder, two pages later Pape diagrams the causal pathways in his nationalist theory that the logit modeling will test --- and yet the role of governmental concessions doesn't figure at all in that diagram. If it's so important, why can't we find it as a key causal influence in that diagram?
Can one infer, then, the paragraph about concessions on p. 94 was stuck in there as an afterthought by Pape once he found that his 2nd reported logit model was in effect mediocre when it came to "enhancing confidence" in his theory's causal pathways?
In the end, what else could we infer?
So what are we left after looking at the few lines devoted on p. 99 to citing how "concessions" as a 4th independent variable improves his 2nd logit model's classificatory outcomes in the key cell of interest in his 2x2 classification table: a jump upward from a mediocre 50% of "correctly predicted outcomes" to 100%?
Not much, if anything. We have no idea why, say, the Provisional IRA --- whose terrorist activities against Britain extend back a century in time --- never resorted to suicide terrorism despite a noticeable lack of any concessions until the last couple of years by London; and those concessions, as we know, are of a minimal nature. Essentially, the IRA was defeated by counter-terrorism and public disgust in Northern Ireland with violence. Neither do we have any idea why the Indochinese Communists who fought first the Japanese in WWII, then the French between 1945 and 1954, and then the South Koreans and the US military between 1957 and 1973 and then the Khmer Rouge in Cambodia never resorted to suicide terrorism despite any lack of concessions by the governments in question here until the Communists achieved full victory in open warfare with the defeat of the Japanese in 1945, of the French 1954, and again in 1975 when the US didn't come to the rescue of South Vietnam and its ineffectual military forces were overrun from the North. We are totally uninformed, similarly, why Mao's guerrilla activities between the late 1920s and the late 1940s didn't entail suicide terrorist attacks either, or why the Boer resistors to the British in the late 1890s did resort to them, or why the Filipino guerrillas who opposed the US occupation at the start of the 20th century and then, during WWII, opposed the Japanese occupiers didn't either.
On the other hand, if you tried generalizing Pape's model back before 1980, you have no idea why the Japanese in World War II resorted to suicide terrorist attacks from the air, on the ground, and by tiny submarines against US forces, but why --- oppositely --- China's leaders, whose country was invaded and occupied by the Japanese in 1936, never used it at all.
(iv.) In the end, to continue this analysis briefly, the whole categorical term of "concessions" that salvages Pape's final logit model is so coarse and ambiguous that, essentially, we're left with a tautology: in certain cases of guerrilla warfare or non-suicidal terrorism, democratic governments being challenged violently by disgruntled ethnic minorities --- or at least certain groups acting in their name --- were able to settle most of the conflicts by negotiations. Which, after all, is what you'd expect to happen in democratic countries most of the time, no?
So what can we conclude other than this: the causes of suicide terrorism aren't explained fully or accurately by Pape's nationalist theory of its occurrence. Just the opposite, the causes seem to have deep roots in specific cultures, including specific religious beliefs and tenets that are culture-based. Religion, of course, may not be the only motivator. Japanese Kamikazes --- not just using planes, but submarines and waves of banzai charges --- were motivated by factors specific to Japanese culture, partly nationalist it's true, but also emperor worship and a bushido tradition of militarism centuries old that constituted something of a combined nationalist and religious world-view. By contrast, the countries Japan attack and occupied brutally --- 30 million people killed by Japanese forces directly or indirectly (starvation of prisoners, massacres of civilians and the like) in Asia --- did launch guerrilla activities, including a few terrorist attacks, but never suicide terrorism.
And yet the Chinese, the Filipinos, the Vietnamese, the Cambodians, the Indians, the Indonesians, the Malaysians, the Koreans, and the Burmese --- all brutalized by Japan and occupied by it in WWII, losing 30 million dead, not to mention the tens of millions scarred, wounded, raped, and what have you --- never resorted to suicide terrorism at all. Yet they were all, no exceptions, different in religion from the Japanese occupiers.
Instead, only the Japanese resorted to suicide attacks --- and they were never occupied at all except for Okinawa in the late spring of 1945 until Tokyo surrendered in the late summer of that year to American and allied forces. Why? Where does Pape's nationalist theory of suicide terrorism and the role of religious differences throw light on these contrasts?
(v.) If, then, certain national or ethnic cultures -- including those influenced heavily by particular religions --- are most likely to generate suicide terrorism, which ones would they be? To answer, let's jump ahead from WWII to 1980 --- the year Pape picks to start his investigations.
In particular, focus on the the concrete numbers in Pape's data-set . . . the sample selection (equivalent to its "population") that he uses for his logit modeling. The data are nicely laid out by Pape in Appendix II on pp. 265-67. T
There were, recall, 58 cases of "military occupations" that Pape discovered between 1980 and 2003 by "democratic governments" only. Of these 58 occupations, 23 entailed some violence by rebels --- the levels of which, naturally, aren't clarified by Pape either: did 10% of the oppressed local nationals or ethnic minorities join the rebels? Or were there only 1/100th of 1.0% How many casualties on both sides were there, and so on? Never mind. Pass on. Of the 23 cases of occupation that generated "nationalist rebellion", only 9 led to suicide terrorism --- and here's the key: if you then examine the 9 cases, 7 of them were carried out by Muslim terrorist groups. The two exceptions are the secular Hindu Tamil Tigers (LTTE) in Sri Lanka battling the Buddhist-dominated government, and the Sikh BKI --- which launched a grand total of 1 suicide attack against the Indian government.
Pause now and ponder the numbers here: 7 out of 9 suicide terrorist groups active between 1980 and 2003 were Muslim in nature. Of the two remaining terrorist groups, one --- the Sikh BKI --- launched exactly one suicide attack in the 23 years Pape examines. [The other, the Tamil Tigers' LTTE, launched hundreds of such attacks, Pape wrongly interprets it to be a Hindu "secularist" group. In reality, the most knowledgeable observers of the LTTE regard it as indistinguishable from a religious cult in its recruitment, indoctrination, kidnapping of hundreds and maybe thousands of very young boys to rear as fervent recruits, use of rituals and symbols, and the views of martyrdom and death are "deeply rooted in the in the lifestyles and religious practices of Tamils in India and Lanka," while the active terrorist members are not just Hindus but attract support from "a significant Christian minority within the Sri Lankan Tamil population" (See, for these quoted words and a very cogent up-to-date analysis --- drawing on anthropological and history studies --- Michael Roberts, Tamil Tiger "Martyrs": Regenerating Divine Potency? Studies in Conflict and Terrorism, (Nov-Dec 2005), pp. 493-514.)]]
But to repeat: the key fact is that 7 out of the 9 terrorist groups active in suicide attacks between 1980 and 2003 were Muslim.
Now note quickly. We'll see in a moment or two that these numbers --- 7 out of 9 suicide terrorist groups were Muslim --- are themselves wildly underestimated. They figure in Pape's data-set for the logit models, and like much of his crucial data are simply very wrong. Yet even if they weren't, Pape assures from p. 4 of his book onwards that no specific religion has anything to do with suicidal terrorism. Come to that, religion per se has nothing to do as a primary motivating influence behind such terrorism. To the extent that religion plays a role at all, it is secondary and indirect --- it aggravates nationalist rebellions against hated democratic military occupiers, at any rate when the religion of the occupier differs from that of the local nationalists.
You rub your eyes: is this man being serious?
(vi.) And it gets worse for his nationalist theory of suicide terrorism, believe me. As a motivating power in suicide terrorism since 1980, the overwhelming nature of radical Islamist ideology can be brought out in a different way.
To see how, go back to the start of chapter one in Dying to Win where Table 1 shows a revealing fact: in simple language, a good 8 of the 10 suicide terrorist campaigns that involved the same terrorist groups between 1980 and 2003 happened to be Muslim. Even while the Sikh BTL is included in that group of 10, Pape ignored or consigned to statistical oblivion a good 9 other Islamist terrorist groups that carried out a total of 14 different suicide campaigns between 1980 and 2003. Two earlier buggy articles in this series on Pape's work, recall, documented those campaigns; the links are found at the very start of today's buggy article. Besides these 9 other Muslim suicide terrorist groups and the 8 in Pape's table, no other groups can be found that were active between 1980 and 2003)
So where are we now?
In Pape la-la land, that's where. Specifically, his Table 1 in Chapter One of his book turns out to be totally unreliable when it comes to the numbers of suicide terrorist groups active since 1980, just as is their overwhelming Islamist nature. In particular, of the 19 different suicide terrorist groups at work between 1980 and the end of 2003, 17 of them --- or 89% of the total --- happened to be Muslim in religion. Seen from this angle, Pape's original data-set --- which serves as the springboard for his book's entire argument about nationalism and military occupation as the major causes of suicide terrorism --- is simply wrong, and by a long chalk.
If map-making were this unreliable, you'd board a plane in New York that you thought was bound for London, and the pilot would land you in a fog around Disneyland, California 6 hours later, all the while screaming into his radio set for Mickey Mouse, Crazy Cat, and Goofy to get off the runway. And by extension, the data-set that's laid out in chapter 6 by Pape and is used as the sample selection for his logit modeling is no less flawed.
It's all odd, no? --- all this hugging and puffing by Pape about data, sample selections, causal pathways tested by statistics, and one bungle or distortion after another.
(vii.) What To Do? What To Do? That Is the Question
More generally, whenever a researcher encountered a data-set of such lopsided nature as this, he or she, it would seem, had an obligation to go back to the basics, look at the overwhelming number of Muslim terrorist groups in the data-pool, and decide whether the two non-Muslim suicide terrorist groups, the Hindu LTTE and the Sikh BKI, weren't outliers. There are even formal diagnostic tests that a statistical researcher can use to detect an outlier in a sample selection --- not that the decision here can be reduced to technicalities alone. As always, including in statistical work, it's up to the judgment of the researcher himself or herself to decide what to do.
And Pape's judgments here and elsewhere are thoroughly unreliable, creating a nationalist theory of suicide terrorism that is entangled in a self-made thicket of data distortions, theoretical dreams, and statistical illusions.
Note: As we'll see when this buggy series moves on and looks at Pape's chapter 7 --- where he examines al Qaeda in depth and shows how, in a fairyland view, its motives and behavior fit his nationalist theory of suicide terrorism to the tee --- a new data-set he conjures up for statistical testing is no less shot through with howlers.
These wild distortions appear Table 13 on p. 111, and in particular its second part.
Pape sets out there more magical statistics that seek to show how al Qaeda suicide bombers --- those 67 whose identities were known before the end of 2003 anyway --- come disproportionately from countries like Morocco that have no strong radical Islamist movements. Huh? Of those 18 bombers who hailed from such moderate national circumstances, 12 comes from Morocco alone. No radical fundamentalist movement of note there? Pape's source for his ignorance of the Moroccan political and cultural landscape derives, as his footnotes show, from a 1993 book that was already outdated when it appeared. In reality, Morocco has a mass Salafist movement of 4 million people, the biggest in the Middle East. For that matter, Pape happens to be wrong about the two countries where the remaining 6 bombers grew up too.
Chapter 7's wonderland stuff abounds elsewhere.
To prove that Islamist fundamentalism isn't extremist or violent --- rather, just opposed to Western globalization and yearning to return to the dreamland days of purified Islam and the rule of Muslim law --- Pape, for instance, cites two sources: one, an introductory chapter in an anthology, was written by a notorious post-modernist adherent of Foucault's views of the West; the other is a recent book by an Islamic scholar --- said by Pape to be "widely respected" --- who explicitly says the West is "evil" unlike Islam. Nor, as the murky cloud-chasing goes on, is anything said by Pape about the rabid anti-Semitism that abounds in radical Islamist circles --- easily documented even in public opinion polls (as we'll see), and easily shown to be riddled with crackpot Nazi stuff; nor anything either about the paranoid views in these same circles of why Islam is backward, poor, weak, and almost everywhere governed by tyrants. It's the all the fault of Jews, Israel, the United States, and its lackeys, you see . . . and of course hundreds of millions of misled Muslims world-wide who have strayed from purified Islam.
All this escapes Pape. Reading chapter 7, you'd never know that the shared mind-sets in al Qaeda and its affiliated terrorist groups --- including imitators in Europe and the Middle East --- crackle with vicious, hot-wire urges toward revenge and sadistically spiteful punishment of infidels, Muslim apostates and traitors, and arrogant Jews and Americans . . . complete with eternal damnation that will follow their earthly comeuppance even as the faithful create a new Islamic caliphate world-wide and continue the struggle against the evil ones until their purified Islam reigns supreme.
But then how could any of this be part of their world-views?
In Pape's theoretical la-la land, these suicide terrorist movements are filled with altruistic types motivated overwhelmingly by selfless desires to rid their lands of democratic military occupiers.
(viii) As it happens, finally, there's a 3rd model that Pape alludes to in the short fn 43 on page 294: he specified, he says, another logit model that substituted "linguistic differences" for "religious differences" as a check on the estimated effects.
Presumably, we can conjecture that he also then used this alternative variable in place of "religious differences" in the interaction variable and included that interaction variable --- linguistic differences acting in tandem with nationalist rebellion --- in his model. Is the conjecture sound? Who know? All we get from Pape is a passing statement that though the new logit model with the new independent variable more efficiently "predicted" when suicide terrorism would occur, he decided to scrap that logit model. The reason? There wasn't, he says, enough variation in the model's estimated coefficients and outcome values . . . those outcomes along the Y axis too constant to have much explanatory power.
Statistically speaking, if that were the case, he had two choices: he could do what he did, drop linguistic differences from his logit work . . . the course he chose. Or, alternatively, he could have re-specified the model entirely to see if there were more variation in the estimates with linguistic difference retained as one of the explanatory variables . . . most likely as an interaction variable along with nationalist rebellion. Whether he tried that or not isn't reported (as little else is either about his statistical work), and we are also left in the dark with linguistic difference was run in tandem with any of the other independent variables. If so, possibly there would have been more variation in the estimated effects.
NOTE: Given the length of this article, Part Three --- which will examine the numerous problems that beset Pape's logit modeling --- will appear in the next buggy article.