This, the 9th article in a long-running series on Robert Pape's Dying to Win , continues its recent scrutiny of his efforts to test his theory's causal pathways by use of logistic regression. Aside from a few introductory comments that follow, the article is wholly concerned with analyzing all the bugs, errors, glitches, blunders, and rippling misinterpretations that envelope Pape's statistical work for his stated purpose . . . or, come to that, any purpose under the sun except those hatched and admired by the denizens of the funny farm.
The Necessary Background
Recall that the 4th, 5th, and 6th buggy articles in this series set out the basics of linear regression and of non-linear logistic regression, and the use of logit modeling or analysis that enables a researcher like Pape to estimate the coefficients of his independent variables and monitor the behavior of his dependent variable's outcomes --- whether suicide terrorism occurs or not in each of the 58 cases in his data-set or sample selection –-- as a linear regression in logged odd terms.
No need to say anything more about these technical basics. If you find that you're unable to make sense of today's buggy analysis, you'd be well advised to look over those earlier articles again.
Pape's Disastrously Small Sample Size
The 6th and 7th buggy articles also set out the severely flawed nature of Pape's data-set, both substantively and for its itty-bitty sample size . . . too small for the reliable use of maximum likelihood estimation, the normal and most effective way logit modeling estimates the coefficients of the independent variable and the behavior of the outcome or dependent variable.
Summary and a Pointer to Parts Two and Three Today
We'll say a little more today about the huge problems that Pape apparently was unaware of caused by using such a small data-set for logit analysis --- or maybe, come to think of it, that he just side-stepped in case one of his 20 expert scholarly chums ever put him wise. From several angles, these problems torpedo any effective logistic regression run on his data-set whether viewed as . . .
- The minimum size data-set Pape needed for Maximum Likelihood Estimation, MLE, which entails "asymptotic" assumptions --- which means that the samples are large enough to "assume" (not prove) that MLE will produce unbiased coefficients of the estimators (independent variables) as well as an error term that assumes or approximates a normal distribution. Pape's set, as we've seen in earlier buggy articles and will see again today, is simply too small to meet this minimal requirement. And note carefully that asymptotic assumptions are precisely those, assumptions and nothing more: we will return to this critical point in Part One
- Or the number of variables he used for reporting any results in a 2x2 classification table for "predictive success,"
- Or the number of "events" needed on the smallest of his binary dependent variable of a qualitative sort . . which for Pape is Y = 1, with only 9 events of suicide terrorism possible not just in his sample of 58 cases, but in the population from which is sample is drawn (one and the same).
As Parts Two and Three will explain carefully, Hosmer and Lemeshow --- the authors of the best book on applied logistic regression, and by far --- insist that a minimum of 10 events on the smaller outcome of the categorical (qualitative) variable is needed for each of the predictors or estimators on the right-side of the logit model . . . which means that Pape couldn't even generate a null model with only an intercept variable accurately. By this measure, Pape's logistic regression plops into fatuity again. In particular, on p. 99, his reported logit model has at least 4 estimators or independent variables, and so he would need a minimal number of 40 suicide terrorist events to produce anything close to reliable estimation of the variables' coefficients or parameters. As it is, recall, there are only 9 such suicide terrorist events in his entire population! (Other logistic regression theorists, by the way, require even more events for proper logit modeling by means of minimal likelihood estimation or MLE than do Hosmer and Lemeshow.)
And It Gets Worse
As we'll see, all these problems that entangle Pape's logit models are compounded by other howlers --- such as an inaccurate interpretation of his interaction term and a zero-cell defect that any logistic regression researcher should easily have caught and corrected. A zero-cell defect, which is innocently shown in Pape's reported logit mode, will "play havoc with the estimation routines." (See J.S. Cramer, Logit Models From Economics and Other Fields Cambridge University Press, 2003, p. 46).
Then, too, in logit modeling, the use of case-study data frequently creates a problem of "endogenous sample selection" --- sometimes called "state-dependent sampling" --- which means that the sample values of an X estimator or independent variable "are not independent of the values taken by Y." Unless corrected, as J.S. Cramer observes on p. 39, such interdependence in the data between X estimators and the Y outcome-variable will "do serious damage to maximum likelihood estimation" --- the universal way that logit modeling estimates the parameters and other effects of logistic regression.
Whether out of innocence, incompetence, or fudging, Pape has sidestepped all these crackling problems of estimation and interpretation that hound his reported logit model on p. 99 of Dying to Win . . . the resulting statistical work a horror-show exemplar, when you get down to it, of everything that's wrong with reflexive, cookbook statistical regression --- software driven and mechanically carried out, with little or no understanding of what efficient logistic regression entails. To compound the ignorance, Pape then serves up some misleading puff-claims that herald his logit model's success.
All these and other technical howlers, almost telephone-book in size, that infest Pape's statistical work are carefully explained in parts one, two, and three of today's buggy article.
All of Which Brings Us To the Greatest Blunder of All: Pape's Data-Set Fantasies
The 7th and 8th buggy article also delved deeply into the even more serious flaws and howlers that make his data-set totally unreliable in substantive terms . . . like virtually all the major data-sets and charts in his book, about 25 in all that turn out to be largely make-believe stuff like the Sea-Serpents thought to be genuine by ancient peoples. Part One will touch on these fantasized data-sets later on today.
Recall Some of
. . . the most blatant deficiencies set out in the last buggy article. Specifically,
1. Click here for Pape's table 1 on p. 15 of Dying to Win that is the first installment of a lengthy snow-job that blankets out the towering near-monopoly of radical Islamist groups in suicide terrorism after 1980.
(i) Note prof bug's favorite gem in this bleached-out table, with its Hall-of-Mirrors display and its cover-up stuff: case 18, where Professor Pape confesses that he's unaware of the "religion" of the Iraqi Kabooming rebels. Hmmm, who might they be . . . Slews of Seventh Day Adventists who took a wrong turn on the Hollywood freeway one night in early 2003 and ended up hours later full of road-rage in Baghdad? A gaggle of Gone-Haywire Quakers, tired of their aggression-blocked, turn-the-other-cheek pacifism? Nested swarms of New Age Spiritualists telepathetically transported to Iraq and hearing Vangelis music inform them to kill US soldiers and their allies . . . oh, and of course a few more than 10,000 Iraqi school kids, shoppers, cafe patrons, pedestrians, and people at prayer?
Isn't it touching to see how modestly Professor Pape confesses to his ignorance here . . . this despite having 16 research assistants at his disposal and 20 expert scholarly chums as readers of his ms.? Reminds you of poor Alice at the Mad Hatter tea party, no less puzzled than these 37 researchers taken together, the poor things, when the Mad Hatter set out a brand new riddle:
"Why is a raven like a writing-desk?"
`Have you guessed the riddle yet?' the Hatter said, turning to Alice again.
`No, I give it up,' Alice replied: `What's the answer?'
`I haven't the slightest idea,' said the Hatter.
`Nor I,' said the March Hare.
"Nor us either" replied without dissent, even if much more recent,
A Sly-Boots Professor and his 36 Simple-Soul Assistants,
Each a totally stumped Innocent,
Left in the dark, hopelessly Ignorant.
The Riddle of the Rebels' religion,
An Iraqi puzzle with no solution.
Yes, no doubt about it: this Pape-puzzlement is so touching that prof bug promises in the very next buggy article to help our self-confessed ignoramus-colleague and his 36 adjutant-sidemen work through their distressing failure of enlightment on this score . . . starting, maybe, with some more heartfelt tips as to the religion of the "Iraqi" rebels such as Abu Musab al-Zarqawi, the head of al Qaeda's Kaboomings in the country who, at last report --- found obviously to be unreliable by Professor Pape, hawkeyed as ever in his sixth-sense soothsayer's way --- was regarded as a Jordanian.
Not to worry though.
When you're briskly busy sweating 5th-dimension non-linear regression 20 hours a day, you don't want to be distracted by trivial details of this sort, do you? Iraqi rebels are all Iraqi, right? And their religion is a real brain-scratching puzzler, just about the biggest scientific dilemma of the 21st century, wouldn't you say? --- beyond even the ability of agile mental giants like the March Hare and the Mad Hatter to solve, not to forget poor perplexed Alice who had tumbled recently down the dark, mile-long rabbit-hole into another world and ended up in their company at a tea-party . . . maybe, who knows? immediately after reading some of Dying to Win in early ms.-form and stumbling around in amazed disbelief. In which case, when you ponder it, she's lucky that she hadn't ended up in a padded-cell for chronic, helplessly incurable brain-blown schizos.
(ii.) Wait though!
One further tip that comes right to mind by way of helping our bewildered colleague is worth mentioning just now: possibly the Kabooming Iraqi rebels are a Meshuga Mob of Manic Malibu Sun-Worshippers, who had been vacationing at a very chic Club Med in the middle of the Iraqi desert (delicious fried lizards for breakfast, yummy sautéed lizard-gizzards with organic turnips and dandelions for dinner, and lots of cheap hash for snacks, ha ha, the kind you don't really eat) when the war with the Saddamite regime began in March 2003; and so--- good and pissed off, and very understandably so, I mean, who could blame them now that their sunshine was occasionally blotted out ever afterwards by swarms of American attack helicopters flying overhead" --- they decided in unison as a raucous Rolling Stone song played loudly through their iPods (all up-to-date, the latest model just out with up-to-five-hours more ear-splitting battery life) that they had had enough, You don't piss on sun-tanners, you imperialist nasties, you! and pell-mell, not a dissenting vote, agreed to convert their beloved Sun-Lotion bottles ("Waterproof Coppertone, 45 SPF Rated") into Molotov cocktails that they've been slinging around in Iraqi cities ever since.
Vigilante vengeance at its best, right?
Right, provided we add that their vengeance is altruistic in motive, just as Professor Pape says it is repeatedly in Dying to Win. Their Kaboomings, we have to further infer from Pape-logic, fervently supported by oppressed sun-tanners world-wide, unable any longer to enjoy lengthy, low-cost sojourns at that formerly chic Iraqi Club-Med in the desert --- the joint now turned into a gay-bar run with zest by al Qaeda terrorists, all briskly keen to demonstrate, for everyone with 20-20 vision, how rollickingly diverse Islamist fundamentalism happens to be around the world. Yes, very very diverse, and exactly as Professor Pape assures us it is between pages 105 and 110 in his no- bullshitting, straight-from-the-horse's-mouth account . . . which, alas, manages once more to crash into fantasy-land fatuity, AKA data-table 13, the whole thing a whitewash job of breath-taking dimensions and unrelieved eye-popping error.
So click here for the rest of this startling, breathtaking analysis of the Manical Mobs of Mass-Murdering Malibu Sun-Tanners --- the possible hard-core Kaboomers in Iraq that Professor Pape, his 16 research assistants, and his 20 scholarly pals are seemingly in the dark about (how sad! how sad!) --- tucked away neatly in an Appendix at the very end of today's buggy article. Click now or read later, depending on your preference --- but note, you are duly warned: if it's late at night when you're reading this and eerie noises are sounding in the inky night-shadows outside your residence, you'd do well to wait until the morning . . . the high-pounding tension calculated to keep you in awake in a damp sweat until then anyway.
2. For a corrected buggy table that shows how Pape omitted 20 cases of suicide terrorism between 1980 and the start of February 2004, click here .
3. Poor Professor Pape can't even divide properly . . . or check to see if his research assistants could.
On p. 205, there's an extraordinary pie-chart that pretends to show the ideology or religious background of 38 known Hezbollah suicide-terrorists in the 1980s. It shows that 71% of them were Christians, yet the one paragraph that sets out the data here, just above it, finds that only 3 of the 38 were Christians. Usually, in earth-bound mathematics, 3 / 38 equals 7.9%, not 71%, but what the heck, when you're busy adding bleach most of the time to your analysis of Islamic terrorism, you're probably too busy to do 2nd grade mathematics properly. You are left wondering whether any of the 20 "expert" scholars who Pape acknowledges at the book's end as readers of his manuscript were sober or even sane when they looked it over.
4. As for an even worse botch-job of data-analysis and the extravagantly misleading claims that Pape postulates on its basis, see this table about al Qaeda suicide bombers and Pape's eye-popping inability to ever check his sources . . . no doubt brought to him by the broom-and-shovel graduate research assistants, all 16 of them acknowledged in Dying to Win as indispensable to his analytical catastrophe. click here
Which brings us to
5. The specific data-set that Pape contrives in chapter 6 for running on his logit models.
Based on 58 cases he coded that involve democratic governments militarily occupying either foreign territory or their own regional territories where restive ethnic minorities were active, it derives in part from the earlier error-riddled data-sets that hide the overwhelming dominance of Islamic terrorist groups in the 23 years after 1980 that Pape focused on, and it's just as full of howlers . . . beginning with his decision to look only at democratic occupiers, then followed by the use of crude categories for classifying his coded data. Needless to add, the resulting data-set is as markedly misleading as the other major data-sets in his book. In particular, as the data-sets corrected by prof bug showed, most targeted countries were not democratic military occupiers but Islamic ones, none of which were democratic when attacked by radical Islamist suicide terrorists except for Turkey . . . a point that we'll briefly clarify once more in a moment or two
No need to say more about Professor Pape's Fairyland Data-Sets at this point, most of which will no doubt have a place-of-honor one day in the Valhalla of Whitewash and Pishposh. We'll place them in storage, labeled clearly buggy article #11 --- the article after the next installment in this series --- while we move directly to Part One and more technical statistical matters that underscore just how rollickingly cruddy Pape's use of logistic regression turns out to be.