Adonis Diaries

Fundamentals of controlled experimentation methods: What’s Human Factors in design? 

Posted on: October 14, 2008

What is Human Factors in design? (Article #39, April 1st, 2006)

Fundamentals of controlled experimentation methods

An experiment is designed to study the behavior of the responses of subjects (dependent variables or what are measured as performance), as the values/stimuli of an independent variables or factors are changed, manipulated, or presented randomly or in fixed manner.

There are other factors that need to be controlled because they could have serious effects on the behavior of the selected dependent variables, and thus are held constant or fixed by appropriate techniques, procedures, instructions, experimental setting, and environmental conditions.

Controlled experimentation methods are versions of current simulations methods, but are essentially more structured and controlled.

In a nut shell, an experimental method is a series of controlled observations undertaken in an artificial situation with the deliberate manipulation of variables in order to answer specific hypotheses.

In general, a scientist plans, controls and describes all the circumstances surrounding his tests, in a way they can be repeated by anyone else, which offer dependability for validation.

The requisite of repeatability encourages artificial settings that can be controlled, especially because the participants/subjects in the experiment are not usually involved or engrossed in their tasks, and because it enables a scientist to try combinations of conditions that have not yet occurred.

Controlled experimentation are time-consuming, expensive, and require a staff of skilled researchers and investigators so that they are conducted for basic research, publishing scientific papers, and when sponsored by deep pockets private companies and well-funded public institutions.

There are different types of experiments, some are designed to extract cause and effects among the variables, and especially their interactions in the performance of a system.

Others experiments are not so well structured and are intended to explore a phenomenon at an initial phase.

Experiments varies in their design purposes and levels of control:  there are experiments on inanimate objects or natural phenomena that follow fixed trends and do not change much with time.

Experiments using human subjects in order to select the better performing system or product, and experiments intended to study the cognitive concepts of people such as attitudes, mental abilities, problem solving aptitudes, attention span and the like are very complex, very intricate,  highly time-consuming and expensive to conduct.

The next article entitled “Controlled experimentation: natural sciences versus people’s behavior sciences” is intended to compare the complexity, differences, and levels of difficulties among the various experiments.

This article is striving to establish the fundamental processes or necessary structured steps to conducting a controlled experiment.

In the spectrum of complexity, innovation, and difficulty the experiments in natural sciences are the easiest, and psychology the hardest; within that spectrum fall experiments in the disciplines of agriculture, econometrics, education, social sciences, and marketing.

Early researchers in the phenomenon of electricity had to experiment with simple methods of one dependent and one independent variable, rudimentary equipments, and to rely on an exploratory knowledge of how electricity works and what are the factors that cause definite change in the behavior of certain criteria.

For example, scientists observed that there are relationships among voltage/power, the intensity of the current and the material the current is flowing through, then a scientist set up an experiment to study how the voltage changes when the intensity of the current varies or when the resistance of a material varies.  By conducting several experiments, first by working with a specific conducting material, thus fixing the resistance, and varying the intensity of the current and repeating this simple experiment many times and, second by fixing the current at a certain level and working with different kinds of conducting materials, then the scientist managed to observe a steady mathematical relationship among these three variables.

As the body of knowledge in electricity expanded and more experiments were undertaken, the physical science of electricity discovered many more factors that entered into the mathematical relationship with varying degrees of importance and consequences.

Obviously, physical scientists can now enjoy more powerful, time-saving, and effective experimental designs that can employ several independent variables and several dependent variables in the same experiment thanks to the development in statistical/mathematical modeling and the number crunching computers; these developments in controlled experimentation allow observations of the interactions among the various variables simultaneously, if physical scientists deign to apply them!

Controlled experimentation methods have a set of requisite structured steps that are common to both natural and social studies. Usually, an investigator has to review the research papers on the topic to be investigated, sort out the articles that are scientifically valid and experimentally sound, consider the variables that have been satisfactorily examined and those that were controlled, or not even considered.   Or the scientist may explore the topic by systematic observation of the problem, then he has to propose a hypothesis that could be rejected, but never accepted no matter how often it was not rejected.  He has to conceive a design for the experiment such as the types, numbers of variables, their levels, and how to manipulate the trials, then he has to decide on the best method for selecting the subjects, the materials, or products to be tested, the setting conditions, the procedures, the operations or tasks to be performed, the instructions, the equipments, the appropriate statistical model.

Then the scientist has to conduct the experiment, running the data, analyzing the results, interpreting the results, and finally providing guidelines or practical suggestions to be applied in engineering projects.

The motors of statistical packages used to analyze data are mathematical models or sets of algebraic equations with as many equations as unknown variables and relying on the two main statistical concepts of means and variances among data.

The purpose of controlled experimentation methods is to strictly control systematic errors due to biases and then to sort out the errors that are due to differences among the independent variables and those introduced randomly by human variability.  Once the size of random errors is accounted for then it is possible to study the relationships among the independent variables and to claim that a hypothesis could or could not be rejected at a criterion level of statistical significance, set frequently at 5%.  This criterion level of 5% of statistical significance, means that there is still a 5% chance that an amount of random error might be the cause in the differences of the results.

Types of errors and mistakes committed in controlled experimentation will be reviewed in article #45.   However, it is important to differentiate between evaluation/testing methods and strictly controlled experimentation. In human factors discipline, evaluation methods are applied to compare the effectiveness of several products or systems by measuring end-users behaviors, like/dislike, acceptance/rejection, or satisfying rules and regulations with the purpose that management would be able to decide on the choice among the products offered within specifications.

Controlled experimental methods are mainly applied to study the cause and effects of the main factors on objective measurements that represents valid behaviors of representative samples of end-users with the purpose of reaching design guidelines for products or systems planned for productions.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s




October 2008

Blog Stats

  • 1,426,589 hits

Enter your email address to subscribe to this blog and receive notifications of new posts by

Join 774 other followers

%d bloggers like this: