Intelligence failure?

Spencer Ackerman argues that attempted suicide bombing of Flight 253 on Christmas Day didn’t necessarily represent an intelligence failure. I think the key part of his post is:

The intelligence community is drinking from a fire hose of data, a lot of it much more specific than what was acquired on Abdulmutallab. If policymakers decide that these thin reeds will be the standard for stopping someone from entering the United States, then they need to change the process to enshrine that in the no-fly system. But it will make it much harder for people who aren’t threatening to enter, a move that will ripple out to effect diplomacy, security relationships (good luck entering the U.S. for a military-to-military contact program if, say, you’re a member of the Sunni Awakening in Iraq, since you had contacts with known extremists), international business and trade, and so on.

As someone whose day job involves analyzing lots of data, I think I agree with most of this. Synthesizing all the related pieces of information – a warning from his father that Abdulmutallab might be dangerous and in Yemen, rumors that a Nigerian might be part of an Al Quaeda plan, generalized threats from Yemen – seems quite hard. If heading off terror attacks requires drawing the conclusion that a specific individual is likely to attempt an attack out of hundreds of thousands (or millions?) pieces of small information like that, it’s probably hopeless; what you’ll get is a lot of noise and very little signal. The complexity involved seems huge and predictive ability seems low.

On the other hand, I believe simple things can work. In the case of Abdulmutallab, a simple thing would have been to take action based on the warning from his father, ignoring all the other factors. And the action that was apparently taken was to put his name in a list of half a million people, but that wasn’t the smaller no-fly list (~4K people) nor the “selectee” list (~14K people).

Matt Yglesias writes that, because actual terrorists and terrorist incidents are so rare, there will be lots of false positives for any methods for identifying potential terrorists. That seems right. That the level of evidence against Abdulmutallab put him in the top half million of suspected terrorists, but not the top eighteen thousand, confirms for me both that the false positive rate is very high (it’s unlikely that there are half a million active terrorists our there) and that we can’t predict very well (Abdulmutallab clearly should have been in a higher scrutiny category).

One particularly worrisome class of false positive is, of course, false reporting. I’m sure for every legitimate case of a warning that so-and-so is a terrorist, there are hundreds or thousands of cases of self-interested accusations. (False reporting could be considered the spam problem in intelligence.)

Rather than inherently ruling out doing anything, though, I think that the large false positive rate means that the consequences of a false positive should be fairly minimal – an interrogation, a more intensive physical search, perhaps an investigator making a phone call or two to understand the reason for travel – instead of denying boarding or shipping a suspect off to Bagram. No question that extra screening at the airport would be inconvenient, unpleasant, and intimidating. And no question that it could be expensive to implement more screening.

I think the relevant policy question is “for our false positive rate, what is the appropriate action to take?” Right now, it appears that there’s very little middle ground between no-fly and “just another passenger,” which appears to keep the no-fly list small. (Yet, it’s famous for having lots of false positives.) Given poor predictive ability of any of these lists, I don’t think that makes sense. Instead, a much more widespread system of heightened scrutiny would seem more likely to prevent terrorist attacks, but still not affect more than a small fraction of travelers.

“Polled” on Net Neutrality

I just participated in a phone poll from some outfit (Western Wats) calling with caller ID saying 801-823-2023. Occasionally, I’ll do these things out of curiosity about what they’re asking, but this one really offended me by how blatantly the questions were directed to a particular result (and how clumsily done that was).

The “poll” was clearly commissioned by carriers opposed to net neutrality. It started with a set of questions to gauge how engaged I was in politics and technology: Do I read news sites online? Do I post comments on blogs? It then moved on to questions about broadband internet policy: Should the government “regulate the internet”? Does Congress have more important things to do than regulate the internet? Should internet service providers ensure “routine internet usage” isn’t disrupted by “large file transfers”? (Is YouTube routine? How about Netflix-via-TiVo? Amazon’s MP3 downloads? Just to name three routine things I’ve done in the past 24 hours…) The last set of questions were looking for agreement with fairly confusing premises, all of which were along the lines that net neutrality would undermine all these good things the internet can do. For example, do I agree that we shouldn’t regulate the internet if/because doing so would prevent empowering the poor to use the internet? (No, I don’t agree.) At the end, parsing the questions, I felt as if I was continually being asked “Have you stopped beating your wife?”

I have no problem with carriers opposed to net neutrality polling to figure out where their message resonates. But this “poll” crossed an ethical line, giving questions with no good answer for people who disagree with their point of view. Perhaps most polling is of this stripe, but I’ve responded to a fair number of phone polls and none of the previous ones was this crass in driving towards a specific result.

Newspapers just don’t get the web, part 7542

I spent a little time playing The New York Times’s Times Reader 2.0 this evening and it’s pretty nice. It gives what appears to be a full copy of the day’s Times in an easy to browse format. Cut and paste works. And doing the crossword puzzle on it was fun. If I were planning a plane flight, I’d definitely use this for offline access to a newspaper.

(This was also the first Adobe Air application I’ve used. I was quite impressed with how smooth the Air experience is and how zippy and close to native the application feels.)

One flaw: the search is based on substrings, not full words, which makes it feel very low precision. Searching for [star trek] in today’s paper showed results about “states to start” and “Representative Pete Stark.” But there are so few documents in a given day’s paper that search probably isn’t a very big issue.

But, showing a newspaper’s typical cluelessness about the web, the Times Reader doesn’t provide a way to get a URL for the article you’re reading. That’s inane. This is 2009. They want people to link to their articles. They want people to tweet them, to share them, to post them on Facebook. The Times knows this: they have share buttons on all articles on their website. The Reader even has a way to send a link to an article to an email address. And they link to “Times Topics” pages from inside the articles, so it’s clear they know how to embed URLs properly. But as far as I can tell, there is no way to just click a button and go to the article in a web browser, so that I can just share it. Instead, if I’m reading something I want to pass on, I’ll need to search for it again on the web to find a URL.

Do they just not want to participate in the conversation?

The things we carried (personal electronics edition)

For a recent family vacation, we had with us

  • two iPhones
  • two iPod nanos
  • a speaker/docking station for the iPods
  • a MacBook
  • a Kindle
  • a Panasonic LX3
  • a Flip Mino
  • and a Nintendo DSi

At one point on the outbound flight, Susan was reading the Times her Kindle, Matthew was playing on his DS, Allie was watching The Little Mermaid on Susan’s iPhone, and I was catching up on work email on my laptop. All the devices got used quite heavily on the trip.

I compare this to traveling a little more decade ago, when Susan I were on the road for a few months straight and had nothing closer to any of these than a compact 35mm camera. (We did use a lot of payphones and internet cafes, though, and picked up the International Herald Tribune when possible.)

(Updated to include a few minor things.)

The Irrationality of Flexible Spending Accounts

Ezra Klein has a good post up today on the problems of giving employers, but not individuals, a tax exemption on health insurance. This is clearly central to the problems of healthcare financing in the US, but, given how things are, it’s not the sort of policy that can be changed by itself – doing so without another mechanism to pay for insurance would end up making many more people uninsured.

What has always seemed to me to be bad policy, but not so intimately tied to the rest of our economy, is the existence of Flexible Spending Accounts. The gist of an FSA is that an employee of a firm which offers such a plan can set aside a fixed amount of their salary to pay for health care or dependent care; that portion of their salary is tax exempt, but must be used in the space of a little bit more than a year or it is forfeited.

I can’t see any public policy purpose here. Why is the tax exempt status of my medical or child care spending dependent on my employer offering such a plan? Why do I need to play “Let’s Make a Deal” to guess the closest dollar amount without going under to the amount I will spend on health or child care in order to exempt it from taxes? How does setting aside “use it or lose it” money help in any way to reign in health care costs?

I do participate in these plans. For child care, it’s easy to figure out in advance how much we’ll spend in a year on preschool. For health care, we overestimate how much we’ll spend and, towards the end of the year, use the leftover money to pick up a few pairs of glasses, since optometry is covered as medical. (Call me cynical; I’ll take advantage of a tax break even if I think it’s bad policy.)

But who benefits from FSAs versus a policy which says “The first N dollars of health or child care spending per year is tax exempt”? I can see how the companies which offer these plans to employers benefit. I can see how such things benefit optometrists or other providers where people can spend their money before losing it. I can even see how employers benefit, since they receive the money their employees forfeit due to the “use it or lose it” issue, though I don’t think many employers are actively seeking that revenue.

But the benefit to the public? To individual employees of companies offering the plans? To people who don’t have access to such plans? There are much more straightforward, efficient, and fair ways to provide a tax exemption for medical expenses.

Lawrence Lessig and Why I’m Going on Strike

Lawrence Lessig spoke at Google this week on his and Joe Trippi’s Change Congress organization. In particular, he made a convincing pitch for the Strike 4 Change initiative, which asks people to make a pledge:

“I’m pledging not to donate to any federal candidate unless they support legislation making congressional elections citizen-funded, not special-interest funded.”

Many of my politically smartest friends believe that campaign finance changes are essential to making government work. I’ve never been in that camp – I’m very skeptical that anything can remove the influence of money on government, at any level – but I’m coming around to the belief that we need to try to plug holes in the system and hope to make progress while the “moneyed interests” are figuring out how to route around the new rules. A few observations seem to have tipped the scales for me:

  • Regulatory and legislative capture by established, often declining, industries appear to me to often be the biggest roadblocks to progress, even within those industries. (Think: cars, music, finance, or health care for starters.)
  • The common and brazen movement between government officials and lobbying firms is a form of institutionalized corruption.
  • The amount of time elected officials need to spend raising money for their next races is shocking and necessarily distorts everything else they do.

Lessig makes these points, and more, very effectively.

For the last several years, I’ve been giving money to a lot of campaigns, almost always Democrats challenging Republicans or contesting open seats. I feel good about this (even though an economist would probably find the utility of these contributions as low as the utility of voting). At first I was negative on the idea of the donor strike, because I feel it’s one side giving up a possible powerful edge unilaterally. What I came to realize, while listening to Lessig, is that this acts as a useful filter – I don’t want to give money to someone who’s wants the status quo in politics to continue; there are certainly plenty of possible candidates to give money to and this gives me a way to nudge on this important issue.

Let’s hope Lessig’s optimism about being able to pass the Durbin-Specter bill is justified; my cynical fear is a filibuster that “supporters” of the bill don’t try too hard to override.

I suspect the Google talk will be available online soon, but it doesn’t appear to be yet. This talk appears to be similar:


Allie, our three year old, just told me she’d do something “soonly,” obviously generalizing from an observation that she wanted a word that ended in “-ly” for that role. Matthew, our son, didn’t make the same types of errors at that age. He’s nearly eight now and a very good reader; I’m not sure if he sees the distinction between adjectives and adverbs today and it doesn’t seem to interfere with his speaking, reading, or writing.

Allie also refers to her school, the Eureka Learning Center, as “My-reka;” for Matthew, it was always “Eureka.”

I’m reminded of Steven Pinker’s Words and Rules, which posits that there are two different mechanisms in the brain for modeling language, represented in the title as “Words,” for memorizing vocabulary and irregular forms, and “Rules,” for grammar and declension/conjugation of regular forms. Allie’s language development seems to be very rule-centric.

Both our kids are great with language. But it seems like they’re wired a bit differently from each other.

Interviewing someone is a skill

We saw Paul Auster interviewed tonight as part of the City Arts & Lectures 826 Valencia Program. Auster is one of my favorite authors — very few books are as good as his Leviathan and others, from the sparse New York Trilogy to the rollicking The Book of Illusions, are also among my favorite novels. He manages to be thoughtful and entertaining at the same time and his books are all very well-crafted. I saw him in a similar interview about a decade ago and remember it as a great evening.

Not tonight. Auster was, as before, great to listen to. But the interviewer, who I won’t name, was terrible. She pitched him pointless questions with short answers (favorite type of scotch? you grew up in New Jersey?) and gave him very few opportunities to talk at length, except where she wanted him to tell stories he’s already written. And we really didn’t need to spend five minutes talking about Auster’s relationship with his assistant.

Interviewing well is hard. The point, I think, is to get the subject to express as much information as possible in a short time and to keep it focused on things that are interesting. It’s not about the interviewer, it’s about the interviewee. If you’re interviewing someone, get your ego out of the way and find ways to make your subject say things that the audience didn’t know.

Over the years, I’ve seen some wonderful interviews with City Arts & Lectures and some real clunkers. But never before has the interviewer been so bad that I was looking forward to the audience’s questions; they were a mixed bag, but almost all were better than the official questions.

New York, July 2008: A vacation in Links

(I blogged a little about our trip, posted a couple of FriendFeed and Facebook messages, but wanted a little bit more. At the same time, it seemed like too much effort to actually write details about everything and, with us back for three weeks, I doubt I’ll get to that. Instead, I decided to linkify my notes.)

Virgin America.

The view from our window:

Sarah & Brian’s wedding. David Stark Design. Prince George Ballroom.


American Museum of Natural History: astrophysics camp for Matthew.

South Pacific. Fireworks with the Philharmonic.

Jazz at Lincoln Center, Dizzy’s Club. Marcus Roberts Trio.

MoMA. Home Delivery: Fabricating the Modern Dwelling. Buster Keaton’s One Week. Cafe 2.


August, Osage County.

The Met. Jeff Koons on the Roof. Superheroes: Fashion and Fantasy. Petrie Café.

John’s Pizza. (Bleeker Street, of course.) Cones.

Children’s Museum of Manhattan.

Waterfalls. Grimaldi’s. Brooklyn Ice Cream Factory.

Katz’s Deli. Russ and Daughters.

Tang Pavillion.

Staten Island Ferry. Playing the Building.

Thunderstorms on the way home:

(Image from Flight Wait.)

Obama, live

We attended a Barack Obama fundraiser in San Francisco this evening. It was my first time at a full-on, campaign-run political fundraiser. I can’t say I like the overall experience; my preference is usually to just give money quietly online. But I thought that hearing Obama live would be worth it and it was.

Obama is a great speaker and not just in comparison to Nancy Pelosi, Steve Westley, and John Roos who preceded him tonight. And not just in comparison to John McCain or George Bush. What’s appealing to me is how much intelligence he was able to convey. Tonight, he spoke without any obvious notes or teleprompter and appeared to be speaking off the cuff. He would pause at times, even stop in the middle of sentences, listen for audience reactions, and think about where to go next. He clearly had a bunch of packaged riffs that, once he decided what topic he wanted to talk about, he could rely on for phrasing — he does those set pieces extremely well. But the interludes, the improvisation, the weaving it all together was at least as compelling.

The emphasis of the speech will, I suspect, show up in his convention speech: if people believe that he will make what he promises happen, people will vote for him. A strong message about creating confidence in a government that can execute competently and effectively. Clear and measurable platform-style positions for health care (insure all children, cost effectiveness) and education (college in exchange for national service). Less on foreign policy than I had expected, but forthright statements about a war that shouldn’t have been fought and restoring alliances. The biggest applause line may have been about shutting down Guantanamo and restoring Habeus Corpus; perhaps that’s a function of the audience, but it’s a significant sign of how disappointed everyone is with what the Bush presidency has done.

This was not a big speech for Obama. He was speaking to a very friendly audience, whose main goal, I think, was looking for reassurance that this election was not going to be 2004 (or 1988) all over again; I came in cautiously optimistic and left the same way. But I also left feeling that he’d be someone you could have an intelligent conversation with on almost any topic. And, more importantly, someone I can trust to lead this country thoughtfully and with the right goals in mind.