Thinking About Trade-Offs: Handicap Acccessible

Are laws that mandate wheelchair ramps and specially designated handicap parking unconditionally good?

Well, you know what I think on the subject--nothing is unconditionally good.

This is the sort of thing that gives economists their bad reputation. How can you say that there are downsides to making life easier for people with disabilities? Aren't their lives hard enough already?

I'll start by making the obvious point that making things handicap accessible is not costless. It costs something to make the ramp, or the doors that open at the push of a button.

I'll follow up with the point that the number of people that these measures will help, in any specific instance, is fundamentally uncertain. I know that, for instance, when a friend of mine was involved in building a theater, they had to make the actors' shower usable for handicapped individuals. Even if they were to never cast a single handicapped actor in their entire existence, they still had to bear the additional cost of having the special shower. So instead of helping people with disabilities, in this instance, you would simply be hurting a small theater still trying to become a financially sustainable operation.

I don't think there is anything wrong with aiming to make the lives of people with disabilities easier. What I do think is wrong is a one-size-fits-all approach that doesn't take into account different circumstances, and how the costs will have greater impacts in particular cases and the benefits will be smaller in others.

These one-size-fits-all tactics are a symptom of a broader inability on the part of many policymakers to face the basic fact that all of life is trade-offs.

Language is a Public Good

In general when economists talk about a public good what they're interested in is the free rider problem. Since no one can be excluded from the benefits of a public good, everyone has an incentive to put nothing into the maintenance of that good and free ride on the efforts of others. Economists look at this from the perspective of public finance and of group action.

I think that the concept of public goods is useful beyond the traditional bounds of economics, and into evolutionary biology. For instance, language can be thought of as a public good. It is my belief that language is taught and maintained through a system of signaling to people when they are using a word in a way that it is not commonly understood in the community.

If my hypothesis is correct, and everyone puts effort into teaching everyone else the proper use of the local language, then we would have to reconcile the free rider problem of economics with the fact that people are in fact investing in making communication within the community less difficulty. I think that isn't difficult, however; economics employs methodological individualism, that is, the individual person is the unit of interest. Modern evolutionary biology has an even small unit--it looks at the gene as the focus of evolutionary pressures. So while genes may have a "free rider problem" when it comes to developing phenotypes that benefit all of the genes involves, the free rider problem across individual organisms may sometimes be overcome when they have enough genes in common.

This is a very simplistic analysis of course; the free rider problem still does exist across individual humans in many circumstances--that's why economists have any notion of it at all. But I think it's interesting to think about when evolutionary pressures may actually help overcome this at the level of the individual, and why that isn't enough in the circumstances of interest to most economists.

Reject the Universal and Embrace the Conventional

I just recently found this post at This Field is Required discussing how Kant's moral philosophy is insufficient to provide guidance even on the simple matter of whether or not you should let someone cut in line. The post is a followup on an earlier one that dealt more briefly with the matter as well as two other philosophies that have trouble with the line-cutting scenario.

The followup came in response to Jacob Levy's argument that Kant's philosophy actually could rise to the challenge. PJ takes this apart pretty thoroughly, so for a very knowledgeable explanation on how Kantianism does not measure up I recommend reading the whole post for yourself.

The difficulty of generating rules of behavior for this very simple task illustrates, to me, the impotence of rationalist, universalist moral philosophy. The focus on Kant in particular is useful here because I consider Kant to be the most relentless in pursuing rational philosophy of the Platonic sort to its logical extreme. He argued that moral duty was something that could not involve personal advantage in any way, and that an action was only justified if it could be turned into a universal principle.

I think morality is completely different from the way Kant and all the less extreme rationalists envisioned it. I don't think it involves generating rules of action from well reasoned philosophical principles, and I definitely don't think it involves universal rules for almost any of it.

Morality is in the first place a feeling that we have, individually, about whether or not an action taken by someone was wrong. This feeling is then taken and shaped by local conventions, a subject that the Vulgar Moralist has written about quite recently.

Here's my half-baked theory on how this works: over thousands of years people have encountered countless specific situations that have evoked particular feelings from our moral sentiments. As a way to minimize the amount of information people need to make decisions for a lot of the situations we find ourselves in, conventions of behavior for different sets of circumstances emerge and are passed on down the generations. These conventions are not encompassing--there will always be circumstances that are so new or so rare that tradition leaves us with few tools to respond to other than personal discretion. And conventions are dynamic; changing alongside the kinds of situations that people face regularly, as well as with some unpredictable variation from generation to generation.

Let's go back to the line-cutting example. The line itself is a convention. It is a long established mechanism for rationing when more people want something than, for whatever reason, can be given it at the same time. If people were to just stand in a formless crowd, it would be much more difficult to get the same job done. So before we even ask the question of how we know it's wrong to cut in line, we should acknowledge that the question itself has been framed by things we take for granted.

What happens when someone cuts in line? It feels wrong. It feels wrong because we all know you're not supposed to do it. We know it not because we've read Kant's Fundamental Principles of the Metaphysics of Morals, or because we were taught Bentham's Introduction to the Principles of Morals and Legislation. We know it because we've learned it over the course of our life; learned it from our peers the same way we learned the language we speak.

Belief is not rational, and neither is morality. If it was, we'd be dead, because we'd never figure out what to do in time. Fortunately, we have nonrational processes to guide us and give us context. By the time we get to the point of making a decision, convention has stripped away so many erroneous choices that even our fallible reason is capable of making a choice.

Being a Scholar When You Can't be a Scientist

It is my basic belief that there are only a limited number of things in this world that we can talk about with any kind of scientific rigor. The number of subjects in which our conjectures can be tested is less than the total number of subjects we can theorize about.

Still, that doesn't mean that we should have nothing to say, in those other subjects. I do not believe that just because you can't say something scientifically, you shouldn't say anything at all. After all, my main interests are history, philosophy, and economics--subjects in which almost every aspect falls well out of the realm of comfortable scientific testability.

When you cannot be a good scientist then you should work very hard to be a good scholar.

What does it mean to be a good scholar?

First, it means admitting that you aren't making testable assertions, that your claims do not have the force of science behind them. This humble honesty should be the starting point for any self-respecting scholar.

That aside, I think there are three practical aspects to good scholarly work: transparency of method, extensive sources, and clarity of presentation.

"Transparency" goes beyond simple truthfulness and into breaking down the strengths and weaknesses of your approach as far as you are aware of them. Whether it's in the body of the work, in footnotes or endnotes, this is the hardest and most important part. The hardest, because we always want to see how brilliant and effective our methods are and rarely even want to consider the fact that we necessarily had to make some trade-offs. Even when we believe in principle that any tactic we take is going to have shortcomings, it's difficult to get motivated to poke holes in your own analysis. It is important, however, as finding the downside to your approach is just as valuable a contribution as finding its strengths.

Sources are very important. You can't test your arguments scientifically so you need some other leg to stand on. It's not about having a large quantity of sources per se, but having a lot of different kinds of sources. If you can get a few people who took very different approaches but arrived at similar or the same conclusion, you're better off than if you just get a thousand people running some variation of the same regression agreeing with one another.

To get back to transparency, I also think there is great value in finding sources that express an alternative point of view that you think make a good case, which you can respond to.

Finally, write in real English, for crying out loud. History, philosophy and especially economics are plagued with jargon and opaque, bad writing. Part of transparency is making your case clearly. Part of integrity is making a genuinely strong case without hiding behind impenetrable prose with a vague air of authority.

Being a scholar means acknowledging that you're not a physicist and the subjects you're interested in will probably never reach that level of rigor. It means making the most of what you've got.

And sometimes something that is untestable today becomes testable tomorrow. Especially if you put your work in writing, in a form that can outlast you. The great scholars in history are the ones that had their work vindicated generations after it was completed. Scholarship is a worthy pursuit. It just merits a little more humility than a lot of scholars have brought to it.

Five Years

Five years?! That's half a decade! What kind of obsessive compulsive nut keeps on blogging for that long?

It's been a long journey from the day the 19 year old decided to pay homage to his favorite Greek philosophers and to the blogfather without consideration for how pretentious the resulting name sounded. Embarrassingly, said 19 year old didn't even know what a "pundit" was, nor did he bother to look it up before starting his new blog.

This kid was quite interested in politics, and wrote about it a lot. He was also interested in philosophy, though frankly the stuff he wrote was quite bad. Some of the first semi-decent stuff he wrote that could be called philosophy had to do with the ethics of discussion, something which would start from modest thoughts and find its conclusion in a post he's still proud of to this day.

He really grew interested in blogs because of the Dan Rather memo scandal, and so became very involved in debates about mainstream news outlets vs. blogs. An increasing number of posts were dedicated to journalism, its problems, why it isn't particularly special and why bloggers in net could do more than any news outlet ever could.

Let's fast forward a little here. I'd like to think I've come a long way since being that 19 year old. The interest in blogs grew into an interest in new media generally, which ultimately lead me to start an entirely new blog to focus on internet-related topics. Meanwhile, an increasing interest in economics led me to the economics MA program at GMU. Sophistpundit has always been about whatever happened to be on my mind, so it's probably not too surprising that I've written a lot of posts on economics over the last year and a half. Some of these are already drawing a surprising amount of search traffic.

Of course with both blogs, and all the other social media I'm on, the best part is all the people I connect with. People like Stephen, Missy, and Charles, and more who I would never have known about if not for the internet.

The mortality rate for blogs in a year, much less five, is pretty high. This has been so fulfilling for me, however, that I don't see myself going anywhere. In fact, I'm looking forward to another 20 years of blogging, at least.

Manski Bounds

The technique that Charles Manski has pioneered in a number of journal articles and explained in at least two books is usually explained in the language of set theory. Here I'm going to try my best to do it justice in just plain old English.

I've explained the nature of identification problems. I find it intuitively easiest to explain in terms of the survey who has nonresponders that introduce nonrandomness into the data. Manski's insight is that this does not make the data totally worthless; we can actually get some information from it that is scientifically rigorous, it just might not be as much information as we'd like.

Let's consider the case of a binary survey, where people are simply asked to say whether they are voting for candidate A or B. In order to preserve randomness., we would need to know:

(The percentage of responders that said they would vote for A) multiplied by (the proportion of those called who responded) plus (the percentage of nonresponders who would have said they would vote for A, had they responded) multiplied by (the proportion of those called who were nonresponders).

Let's call the categories in parenthesis 1, 2, 3, and 4, respectively. A real world survey will tell us 1, 2, and 4--that is, the answers given by responders, the proportion that responded, and the proportion that did not respond. It does not tell us 3.

But we still know some things even without knowing 3. Manski's approach is to provide bounds, or to show the range of what the survey outcome could possibly be for all possible values of 3.

So one bound would be the case where all nonresponders would have said they were going to vote for A, and the other bound would be the case where none of the nonresponders said they would vote for A, but instead would vote for B.

The width of these bounds--that is, the difference between the two extreme possible survey outcomes--is determined by 4, the proportion of those called who were nonresponders.

So if only 1% of the people called are nonresponders, then the difference between what the survey results would look like if they had all said they'd vote for A vs if they'd said they would all vote for B is quite small.

Unfortunately, nonresponse rates are rarely that insignificant. I haven't corroborated this, so I don't know if it's an exaggeration or not, but according to Doug Rivers, at their best nonresponse rates were around 30%, and that was decades ago. Obviously the results of a survey would look quite different if 30% of the people they call all go to A or all go to B. So here's a case where nonresponse is a real serious issue.

What's more shocking is that Rivers says that 30% is extremely low compared to nonresponse rates faced by pollsters today. Over time, telephone surveys have been crammed in with massive amounts of telemarketing and other such calls, making it so that people are less and less willing to give their time to people calling to get something from them. There are other factors as well, but the bottom line is Rivers' claim that the typical survey today has a nonresponse rate of 80-85%!

If true, that makes survey data utterly worthless even from Manski's perspective. If 3, the one category that you don't have any information about, is three or four times as bit as 1, how can you possibly think that you'll be able to extract some meaningful information from 1 alone? The width of the bounds is so big as to swallow all of the actual known data!

There is much more to Manski bounds than I have gone into here; he is, for instance, interested in seeing what happens when you start imposing assumptions on 3 to try and narrow the bounds. I'm less interested in that, however. So I hope I've done a decent job of explaining the basic information Manski says you can get from data before you decide you want to try and conjure up some assumptions. I think it's a great insight--just because data is imperfect doesn't mean that it's completely uninformative. If the width of the bounds is reasonable enough, the data might be good to know.

Of course, when survey data is reported in public media outlets, they never include their nonresponse rate to my knowledge. I think one minimal improvement that could be made in the way we do these things would be to create an expectation that people be more open with that kind of information.

UPDATE: In the comments, Jeff has a lot more details on Manski's approach.

Scientific Rigor

My last post dealt with identification problems, such as nonresponders in a survey, and how a lot of the tricks used to solve them fall short. I'd like now to consider the question of whether identification problems can ever be solved.

My short answer is yes, but that the possibility of a solution has more to do with the subject matter than with brilliant on anyone's part.

Let's take a look at the nonresponder problem. This plagues every type of survey, including the political polls that get a lot of attention during election season. So are those polls worthless?

Absolutely not. Political polls, especially the ones reported the closest to the time of voting, have a very powerful feedback mechanism: once the vote is counted up, we can very easily evaluate the accuracy of each poll as a predictor of the outcome. If, at the time of the vote, Gallup is showing candidate A with 60% and candidate B with 40%, and B wins in a landslide, that makes Gallup look pretty inept.

The existence of this hard and fast evaluation criteria has worked wonders on political polling--for the most part, the polls are pretty good. It's understood that there's a margin of error, but for the simple purposes of figuring out who is more likely to win, your basic political polls aren't bad.

The existence of this constraint doesn't just make it possible for pollsters to determine if there's something wrong with the assumptions they're imposing on the data. It makes innovation possible. For a great example of this, see Doug Rivers explaining the interesting work being done by the people at YouGov.com.

Rivers and his colleagues have two parts to their methodology: first, they use some of the enormous databases of consumer information that exist out there. Second, they have millions of people who have opted in to take their surveys (I believe in exchange for payment). Obviously there is a selection issue with just surveying the people in this group. So what they do is they randomly select individual profiles in the consumer database, and then find individuals who have opted in that share the same characteristics as those profiles in a number of specified dimensions. This way they introduce the element of randomness in their selection, while removing the nonresponse.

If the characteristics that they use as criteria are not important, however, the whole exercise could be futile. Fortunately, they can test the effectiveness of their methodology and compare its results to normal polls during election season.

However, the effectiveness of this method in election polling doesn't serve as a scientific basis for assuming its effectiveness in other areas. It could be that the characteristics that they match are important for determining who people will vote for but not how they feel about tariffs or the Iraq War. Since there is no hard and fast evaluation criteria in the latter areas, any innovation isn't much better than artistic expression. The existence of such a criteria is what makes it possible for innovation to be scientific and rigorous.

This is what forms the divide between the social sciences and the physical sciences. It's not as though all physicists are inherently more intelligent than all economists. There are just a lot more hard evaluation criteria in physics than there are in economics. Whether or not a projectile lands anywhere close to where you thought it was is something pretty easy to measure. Whether or not the theories that went into building the atom bomb were valid and precise is pretty clear at this point.

There are a lot of smart economists. When it comes to complex mathematics and sophisticated statistical techniques, I would say the vast majority of economists are in another universe of intelligence from where I could ever hope to be. But there is almost never any scientific criteria for evaluating the kind of hard-hitting analysis that goes on in academic journals. That is the reason that I believe that most of the work being done by economists today is unscientific garbage.

Economists shouldn't feel too bad, though--I think that most studies in the social sciences in general fall into this category.

I do think it is possible to be a good scholar even when it may not be possible to achieve any sort of scientific rigor. I'll leave exploring that distinction for another time. For now, I'll just say that when the best evaluation criteria you have are fuzzy at best, everyone would be better off if you were honest about it.

Pinker on Gladwell

Ouch.

Gladwell is a writer of many gifts. His nose for the untold back story will have readers repeatedly muttering, “Gee, that’s interesting!” He avoids shopworn topics, easy moralization and conventional wisdom, encouraging his readers to think again and think different. His prose is transparent, with lucid explanations and a sense that we are chatting with the experts ourselves. Some chapters are master­pieces in the art of the essay. I particularly liked “Something Borrowed,” a moving examination of the elusive line between artistic influence and plagiarism, and “Dangerous Minds,” a suspenseful tale of criminal profiling that shows how self-anointed experts can delude their clients and themselves with elastic predictions.

An eclectic essayist is necessarily a dilettante, which is not in itself a bad thing. But Gladwell frequently holds forth about statistics and psychology, and his lack of technical grounding in these subjects can be jarring. He provides misleading definitions of “homology,” “saggital plane” and “power law” and quotes an expert speaking about an “igon value” (that’s eigenvalue, a basic concept in linear algebra). In the spirit of Gladwell, who likes to give portentous names to his aperçus, I will call this the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.

Bold added by me.

Identification

Let's say you're interested in knowing the typical adult American's opinion on health care or some other hotbutton issue of the moment. So you put together a survey. You want to be all scientific about it, right?

Well, the good news is that statistics tells us that you can get a picture of the population at large that you can be reasonably confident in from a pretty small sample--1,000 will do. If--and this is crucial--the sample is randomly selected.

So you get a machine that dials phone numbers at random, so that there is no selection bias baked into your survey from the get go. You call a thousand random people, ask them some questions, record their answers, and you're done. Easy, right?

In reality you always face a problem: some of the people that you call choose not to participate in the survey. In fact, quite a large proportion of the people you call do not participate. This introduces nonrandomness into your survey--any systematic difference between the kind of person who does respond and the kind of person who does not will bias your results.

That sample of 1,000 people can tell you a lot about the kind of people who respond to surveys. If you want to be even more confident in your results as they pertain to the kind of people that respond to surveys, you can increase your sample size.

However, no matter how much you increase your sample size, you won't be able to learn anything meaningful about the people who do not respond to surveys, and without knowing that, you won't be able to generalize the results of your survey to the entire population.

Inferences where the confidence in your results can be increased by increasing the sample size are known as statistical problems, while those for which no sample size will provide any information are known as identification problems.

I learned of the distinction from Charles Manski's book, Identification Problems in the Social Sciences. I highly recommend it for anyone who wants to think seriously about what can and cannot be learned from a given set of data.

The nonresponders in a survey is a classic identification problem. Are the nonresponders different from the responders in a way that is going to systematically bias the results? This is not a question that can be answered scientifically, and I think a lot of the techniques that are used to compensate for the nonrandomness that nonresponders introduce into the survey are pretty shoddy.

My understanding is that survey companies will try to weight the people who do respond to attempt to better represent the proportion of the population that their demographic makes up. If the proportion of hispanics who respond to a survey is one-fifth of their percentage of the overall population, they might weight their responses so that each hispanic response is actually counted as five responses, for example.

I think this is absurd. The randomness standard is rigorous; if the ideal was possible to attain, you could be confident in your results. When nonrandomness intrudes upon your results, however, it is unscientific to simply make some haphazard assumptions about getting proportional representation in ethnicity or gender. It is an arbitrary judgment call.

There are a couple of ways to solve identification problems. Manski talks about the case where you observe a person and their image in a mirror moving simultaneously. Which way does causation flow? Simply gathering data about how often the movement of the mirror image is correlated with the movement of the person in front of it does not tell you anything about causation. In this case, prior information about optics solves the identification problem. Optics is a pretty hard science, so the information we get from there can be considered reliable.

On the other hand, in surveys and in social science generally, the response to identification problems is far less credible. All of the models that statisticians and econometricians develop to compensate for nonrandomness amount to is imposing assumptions on their data. These assumptions cannot be justified in any rigorous manner. They are defended on intuitive, logical grounds that are untestable. In short, it comes down to persuasion.

For the really important statistics, it often doesn't even involve that. When it comes to something like the unemployment rate, a government agency simply provides an official estimate and most people take it as a given.

So when and to what extent can you trust the data that is reported? I've got some thoughts on that, but I'll leave them for another time. For now, I'll just say that it's never a good idea to accept the statistics you hear reported at face value. There is a lot of discretion and judgment calls that go into turning the data that they actually have into the numbers that get reported.