The Scientific Method

A quick search of Internet resources quickly demonstrates that in reality there is no single uniform description of the scientific method.  Some descriptions have four steps; others have seven or eight.  Some start with hypothesis; others start with observation.  Some emphasize one step of the process at the expense of the others.  The following definition represents a fairly typical and accurate definition of the scientific method:

“The scientific method is the process by which scientists, collectively and over time, endeavor to construct an accurate (that is, reliable, consistent and non-arbitrary) representation of the world.”

Let us examine the statement in more detail:

1.         The Scientific Method is a process
2.         The Scientific Method is used by scientists
3.         The Scientific Method’s objective is to construct an accurate picture of reality.

The process is not a cookbook recipe.  That is to say, there is no owner’s manual or user’s guide.  Many users don’t even agree on the process details! (more)

The Process

Starting in 2004 the Defense Advanced Research Projects Agency (DARPA) sponsored a series of car races with total prize money in excess of $6 million.  The goal of the races was to build a car that could navigate a significant trip without a driver.  The first trip was 150 miles in length.

None of the contestants finished the first race.  The contestants carefully monitored what happened in that race, analyzed the data, and made corrections to their cars.  Four cars successfully completed the second race.  Again, the contestants carefully monitored what happened in the race, analyzed the data, and made corrections to their cars.  Six teams finished the third race.

This is an example of the scientific method in action.  Notice the key elements of the process:

  1. A reality check occurs (e.g. a race with an undesirable result).
  2. Data is collected and analyzed based on (1).
  3. Improvements and corrections are made based on errors found in (2).
  4. Something better occurs next time (more desirable race results) based on (3).

One could depict the process pictorially as follows:

What makes the process so effective is the “feedback loop” shown by the heavy dashed line.  “Feedback” is simply a technical term for “learning.”  You and I do that:  We make a mistake, figure out what we did wrong, and hopefully do not repeat the mistake.  Each time we do that, we are using the scientific method!  If we never learned, we would continue to make the same mistakes over and over.  That is the power of the scientific method.

Some of the contestants didn’t finish any of the races.  Did they fail all the races because they didn’t apply the scientific method?  Not at all.  They failed because they didn’t apply the scientific method rigorously, honestly, thoroughly:  They erred, that is to say they created errors in the process, and the errors overwhelmed or camouflaged the truth, resulting in failure.

The Users

Most descriptions of the scientific method state that it is used by scientists.  Many descriptions further restrict its use to “hard” science such as physics, chemistry, and so on.  In addition, other descriptions restrict its use to situations where results can be physically measured.  Such statements are incorrect and false:  Everybody learns; therefore everyone uses the scientific method.

“Learning” is essential to success, even personal success.  For example, consider the situation of an alcoholic.  Step 1 is to recognize the problem as a problem: “I cannot control my alcohol intake.”  Step 2 is to examine the options, “I can cure myself or I cannot cure myself” and then take appropriate action.  If the answer is “I can’t cure myself,” then I must devise a corrective action plan (Step 3) – perhaps decide to join Alcoholics Anonymous.  I will make improvements after I join Alcoholics Anonymous and begin honestly executing their program.  Before graduating from the program, I will be required to pass a test (Step 4).  When completed, I will be successful and ready to tackle reality (Step 1).  (I recognize the process is more difficult than that, but the loop is correct.)

The Objective

The objective of the scientific method is to construct an accurate picture of reality.  As we can see from ( Truth) “an accurate picture of reality” = Truth.  Therefore the objective of the scientific method is to seek the Truth.  There is only one Truth.  Jesus Christ claimed that He is the way, the Truth and the life, and he invited us to seek Him.  He did not invite us to blindly accept Him.  He invited us to seek Him:  He challenged us to apply the scientific method!

We will examine in more detail how the scientific method can be applied to discern Truth in “Discerning Truth.”

Some sources of error

1)  In the race (Reality check)

Failure to recognize and admit failure.
An extreme example of this failure is the “everybody wins” syndrome.  If a competitor does not recognize that not finishing the race, or not finishing the race fast enough to beat another competitor is a failure, that team will not be motivated to learn and improve their performance in the next race.  One would think that this is very rare, but the syndrome is rampant in our educational system.

Failure to collect all the information needed
Sufficient data are needed to complete a thorough analysis, or to collect erroneous information for the analysis.  Clearly insufficient or erroneous information will result in erroneous conclusions and problem identification, and preclude effective corrective action.

2)  During analysis

Incorrect assumptions.
We can’t know everything and we can’t measure everything.  Therefore we must make some assumptions about how things are, how they work, and so on.  If these assumptions are incorrect, they could result in incorrect conclusions and results.  A common mistake is to presume an outcome before the analysis is completed and then use the analysis to support or justify the presumption.  This is a fatal mistake that prevents all possibility of learning.

Incomplete, inappropriate, or sloppy analysis.
All of these conditions will result in erroneous analytical results, usually identifying the wrong problem as the cause of failure.  Again, since we can’t know everything, shortcomings in analytical approach and rigor result in identifying the wrong problem as cause of failure.  For example, assume that the race failed because the car didn’t finish.  Why didn’t the car finish?  It got lost in the desert.  Why did it get lost in the desert?  The navigation system malfunctioned.  Why?  Etc.  Sometimes we have to ask “Why?” dozens of times before we reach the root cause.  The process is painful.  Therefore it requires extreme discipline to continue questioning past the first or second level, and many don’t.  The resulting misdiagnosis is perhaps the most common source of error and failure.

Error and sensitivity analysis.
One of the most critical, yet most frequently ignored, elements of this activity is error analysis, otherwise known as sensitivity analysis.  This is very difficult.  There is nothing perfect in this world; therefore errors abound everywhere.  One must identify both the sources of such errors and their possible affects on the outcome.  For example, let’s assume that the navigation system of the car uses the odometer to measure distance traveled, and the odometer has an error of 5%.  On a 100 mile course, that will result in a 5 mile error; clearly too much to succeed.  How much error can we tolerate?  How good an odometer can we actually purchase?  However, the odometer’s accuracy may not matter if we can depend on GPS to recalibrate the odometer periodically en-route.  Consequently, when an error analysis is not performed, we may be wasting a lot of energy, time, and resources trying to solve a problem that doesn’t exist!

3)  In the corrective action plan

Inadequate problem identification.
The success of the corrective action plan depends on how accurately the root cause problem was identified in the analysis step.  What good is an improvement plan that does not address the real root cause of the problem?  Why fix something that is not broken, and leave broken something that is broken?  It’s a sure-fire recipe for failure.

Assumptions and thoroughness.
The success of the corrective action plan depends on the same factors as the analysis of the problem:  Accuracy of assumptions, thoroughness, completeness, and truthfulness of the planning process.  What good is a corrective action plan that relies on uncontrollable factors, such as “We failed because it rained and it won’t rain next time”?  The same rigor and diligence is required in developing the corrective action plan as was required for analysis.  Make sure errors produced by the plan fall within acceptable tolerances – or failure is virtually certain!

4)  During validation and testing

Thorough, honest testing.
Improvements must be made honestly and must be thoroughly tested against the requirements of the corrective action plan.  For example, our corrective action plan calls for an improvement in gas mileage from 20 to 30 mpg but the vehicle we build only achieves 25 mpg.  If we don’t test the car’s mileage before the race, the car will run out of gas and we will fail.  If we do test it and find out the truth, we have an opportunity to take other corrective actions and still succeed.  We know that few things work exactly as planned.  Therefore thorough and honest testing is an essential element of success.

Stress and limit testing.
The only certainty is that we will encounter the unexpected.  Therefore we need to conduct stress testing to see how the car will function under very adverse conditions we are likely to encounter such as heavy rain, dust, and so on.  The rescue of Iranian hostages failed in 1979 because the helicopters encountered the stresses of an unanticipated environment:  Dust storm.  Similarly, we need to validate that our car will function properly in the desert.  The same logic applies to limit testing, which validates that the car will always work within the boundaries of its design limits.  For example, we need to validate that our car will withstand the hot temperatures of the desert; otherwise it will overheat and fail the race.

The Most Insidious Failure of all

Lack of intellectual honesty and integrity.
The foundation for the entire process is honesty, accuracy, and truthfulness for each step of the process.  Lacking these human traits, the process is sure to fail because results, reports, and even tests will be falsified.  But such falsification inevitably results in failure during reality check.  Therefore personal honesty and integrity are the key foundations of the entire process.  Without intellectual integrity, the process is nothing more than a charade, futile hand-waving, and a recipe for failure.

Common misconceptions and abuses of the scientific method

All scientists can be trusted to provide “true” results using the scientific method.
That statement is simply not true.  The scientific method depends on personal and intellectual integrity and honesty.  Not all scientists are intellectually honest.  Any statement made by an individual who lacks intellectual integrity and honesty, any statement made by any person who is not dedicated to seeking the truth the whole truth and nothing but the truth (no matter where it leads) should be suspect.  There are numerous instances of scientific fraud.  Two recent spectacular ones immediately come to mind:

•           Stem cell research fraud ( )
•            Global warming email scandal ( )

All sciences use the scientific method.
That statement is simply not true.  Some sciences, particularly the soft sciences, only use parts of the scientific method.  Some sciences such as paleontology and astronomy use a method that lacks the critical feedback loop mechanism for learning.  Therefore these sciences are much more resistant to new findings and new views; they will not readily adapt to new discoveries but tend to ignore and even suppress them.

The following link explains how a number of sciences use a three step model:  1) observation, 2) interpretation, and 3) classification.  There is no feedback loop!  How can learning take place?

All scientific disciplines provide proper validation controls (Step 4).
Most scientific disciplines use the peer review and publication technique as a validation tool.  However, if the peer review and publication process has been politicized or come under the control of a clique with a strong vested interest, then contrary opinions and research will be ignored and black listed.  Lacking publication, the group artificially enforces the status quo to resist change.  This is very subtle but becoming more and more prevalent; it is called scientific in-breeding.

Now, what happens when we have biological in-breeding?  The society experiences deformity, illness, and even death.  Scientific in-breeding will have similar results in the arena of knowledge and truth.

The scientific method has limited application.
That statement is simply not true.  Since the objective of the method is the establishment of Truth, which is core to human existence, the process is applicable in all spheres of endeavor.  A number of links are provided at the end to support this claim.

One of the most commonly accepted limitations of the scientific method is that it can only be applied to “measurable” (ie. physical) objects.  That also is absolutely incorrect because the scientific method is already used to deduce the existence of invisible black holes, as well as undetectable subatomic particles!  In particular the existence of these objects is inferred based on the signature or trace they leave behind.  Then why cannot the scientific method be applied to deduce the existence of immaterial forces on this Universe through the signature or trace they leave behind?  If it can be used to detect immaterial objects, then it must have wide applicability – contrary to claims otherwise.

Warning to Students

Your teacher may not be pleased if you use this as part of a class paper on the scientific method because it doesn’t necessarily have the specific steps he is trying to teach you.  For example, I don’t discuss publication, peer reviews, etc. because those are just specific examples of step (4) for specific situations.   However, every model of the scientific method is a subset, a specific implementation, of the general model discussed in this essay.

Some Additional References


© 2012 notasitlooks

  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: