Fallacies in Scientific Research: Appeal to Popularity

This is my second post on logical fallacies in scientific research. Today's subject discusses how the "Appeal to Popularity" fallacy can hinder the research environment. This one in particular is a bit tricky because, at the face of it, an individual may use it as evidence.

Definition: Appeal to popularity is a logically fallacious argument in which an individual is lead to believe that something is true (valid, moral...) because it is widely accepted or used. The person arrives at this belief without any reference to evidence supporting the validity of the claim.

Examples:

• The majority of people use brand X car. Then it must be the safest car.
• Laptop Y is very popular among university students. Therefore, it must be the best laptop.
• The majority has opposed this law. It means that the law is bad.

This fallacy is a very delicate one as I mentioned previously. There are two points in every one of the above statements: the "factual" part and the illogical inference. It may be true that the majority favors brand X or Laptop Y, but inferring that it is a good product is wrong. There is no immediate link between these two points. 

It may also be true that car X is one of the safest cars, but it is not because everybody owns one. Such a statement should be validated by data, experimental tests between a variety of cars and so on. Interesting, for the most part, one can revert the above statements and obtain a valid argument. For instance, because car X is one of the safest cars, it has a wide customer base.

At the face of it, it seems that by appealing to popularity, one is using statistical data. This becomes a problem in Scientific research. As usual, examples from personal experience:

• Fluent is the most popular CFD code used. Then it is the best CFD software out there.
• The Finite Volume Method is the most popular discretization technique. Then it must be the best.
• Everybody is getting funding from the industry. Then, this is the best source of funding. 

Again, these are all invalid arguments for making decisions especially in scientific research. To stretch things a bit, these arguments may be massaged a bit to lend them some credibility by isolating the statistical component of each argument and using it as data input for making decisions. Here's how I think these should be amended:

• Fluent is the most popular CFD code used. We should list it as one of the software to consider for purchase. But first, we must compare its performance to the other software we are considering for this particular problem and then make an informed decision.
• Fluent is the most popular CFD software. We should consider it in our modeling efforts to reach a wider audience. (I'm not too fond of this particular way of putting it as this borders on the marketing side).
• The Finite volume method is a very popular discretization method. Based on the literature we reviewed, the method was successfully used to simulate a wide range of physical phenomena. There's also a large amount of evidence that the method is particularly suited for transport phenomena. We should consider it as a viable method for solving our hypersonic design problem.
• I don't have any comments on the last one.

When it comes to science, our conclusions should be entirely based on the data. But when it comes to decision making, data is only a part of the process. There are existing and expected experiences that come into play and those may not be entirely rational. The problem is not also in the statistics. If the statistics point to the fact that 80% of the simulation science is done using the finite volume method, then, in the context of science, this should only mean that we should consider the finite volume method as option and test its performance for our problem. Appeal to the number by itself is meaningless. What percentage have reported positive results in this case? If the argument was: 75% of the scientists have reported positive results for using the finite volume method for compressible flow problems, then things are quite different. This is no longer appeal to popularity, it is an appeal to evidence.

There are many other details about this logical fallacy. For an excellent discussion, please visit the wikipedia entry for this fallacy.

References:

http://en.wikipedia.org/wiki/Argumentum_ad_populum
http://www.nizkor.org/features/fallacies/appeal-to-popularity.html