Adonis Diaries

Posts Tagged ‘controlled experimentations

Idiosyncrasy in “experiments”; (Dec. 30, 2009)

Idiosyncrasy or cultural bias related to “common sense” behavior (for example, preferential priorities in choices of values, belief systems, and daily habits) is not restricted among different societies: it can be found within one society, even within what can be defined as “homogeneous restricted communities” ethnically, religiously, common language, gender groups, or professional disciplines.

Most disciplines (scientific or pseudo-scientific) have mushroomed into cults, with particular terminologies and nomenclature:  They want to impress the non-initiated into believing that they have serious well-developed methods or excellent comprehension of a restricted area in sciences.

The initiated on multidisciplinary knowledge recognizes that the methods of any cult are old and even far less precise or developed than perceived; that the terms are not new and there are already analogous terms in other disciplines that are more accurate and far better defined.

Countless experiments have demonstrated various kinds of idiosyncrasies. Thus, this series on idiosyncrasies.  I have already published one on “conjectures” in mathematics.

This article is intended to compare the kind of controlled experiments that are applied by scientists in (natural science), such as physical natural phenomena, engineering… and those developed by scientists dealing with the behavior of people or employing human participants in the experiments (psychology, sociology, economics, or education).

Although the physical sciences, such as all the branches in physics and chemistry…, used controlled experimentation long time ago, in order to develop the huge body of knowledge on the natural phenomena, it was the social and psychological sciences that tried to develop the appropriate and complex statistical modeling packages in order to study the more complex and more varied human behaviors.

It appears that the restricted and countable number of variables in studying the physical nature, and their relative lack of variability with time, did not encourage the physical scientists to contemplate sophisticated statistical models for their controlled experiments, or even to teaching the design of experiments in the engineering curriculum.

Before we expand on the variability of human behaviors it might be more appropriate to analyze the most critical difference in the two sciences. Knowing that any concept is synonymous with the corresponding necessary set of operations in order to be able to measure it scientifically in experiments, we can understand the big leap forward of the body of knowledge in natural sciences compared to the social and psychological sciences.

Whereas the physical scientists can define the concepts of force, moment, power and the like through the relationships of measurable variables based on length, time, and mass the scientists investigating human behaviors have to surmount that hurdle before seriously contemplating to measure human concepts.

Human behavior and the cognitive concepts of attitudes, mental abilities, and moods, problem solving mechanisms, perception, and the like cannot be measured scientifically until sets of operations are agreed on, for each one of these concepts, through the study of human activities or the things that people do while performing a valid task or a set of purposeful tasks.

For example, saying that color blindness is a deficiency that confuses colors will not cut it; what is needed are a set of instances that could define this illness such as what exactly are the colors of the spectrum with mixtures of two primary colors can a “protanope” (color blind individual) match that are different from normal people, he will confuse a blue-green color with white or gray, will confuse red, orange, yellow, yellow-green, and green when suitable brightness and saturation of these colors are used, and has reduced visibility in the red end of the spectrum.

Two decades ago the air force in the USA contracted out groups of psychologists and human factors professionals to specifically establish a set of operations that could be submitted to potential airplane fighters to measure and evaluate their capabilities for the mental and perception workload needed for the job.

This set of ten or twelve operations measuring short term memory capacity, reaction times, computational abilities, attention span, and types of errors committed in each operation is the kind of hurdles that the study of human behavior have to surmount.

The operation measurements of a single human concept may be a life project for a group of scientists that require secure and continuing funding from concerned parties who have vested interests in thorough study of the concept.  It is obvious that a few human concepts will enjoy deeper and more complete investigations than others.

Maybe because the physical scientists did not face the problems of establishing sets of operations that the method of controlled experimentation was not deemed essential enough to rigorously teach in high school programs, and ultimately failed to initiate the students to the experimental methods.

Social sciences made significant in-roads into the educational programs in the last decade.  This lack of early initiation of students to experimental methodology might also be the main reason why rational thinking and the experimental mind is not that widespread throughout all societies and are just confined to the privileged who could afford higher education at select universities.

Physical scientists rely on equipment to “objectively” observe and measure, and the more the equipment are precise the more accurate are the data.  Scientists of human behavior have to rely on people’s responses and observations.

It has been proven that man is Not a good observer of complex events; even when viewers are forewarned that they are to see a movie about a crime, and that they are to answer questions about details later on the accuracy of the observation, subjects were discovered not to be that accurate.

Man is unable to be an objective recorder of the events that transpire because he gets involved in the scene actions.  Man has a very narrow range of attention and barely can satisfactorily attend to a couple of stimuli. This observation deficiency is compounded by our sensory differences and illusions; for example, one in sixteen is color blind, many suffer from tone deafness, taste blindness and so on.

Man does not think of himself objectively but rather has convictions, feelings, and explanations based on very restricted experiences, hearsay  memories and he tends to generalize and develop a set of beliefs concerning the operation of the mind (idiosyncrasies).

Man usually expects to see, and then see what he wants to see, and hardly deviates from his beliefs, even when faced with facts.  For example, many scientists have overlooked obvious data because they clanged to their hypotheses and theories.

Man has to generate an abundance of reliable information and assimilate them before he could eliminate a few systematic biases that he acquired from previous generations and his personal experiences.  Consequently, experimenting with people is more complex and more difficult than the physical scientists or engineers have to cope with.

First, there are no design drawings for people’s mind and behavior as engineers are familiar with because the structure of human organisms is approximately delineated and the mechanisms are imperfectly understood.

Second, people vastly differ in anthropocentric dimensions, cognitive abilities, sensory capabilities, motor abilities, personalities, and attitudes.  Thus, the challenge of variability is different from physics where phenomena behave in stable fashions, are countable, and can be controlled with minimal management.

Third, people change with time; they change in dimensions, abilities and skills as well as from moment to moment attributable to boredom, fatigue, lapse of attention, interactions among people and with the environment.  People deficiencies in senses, physical abilities and cognitive capabilities changes with time and thus, the techniques of selecting subjects have to account for the differences in age, gender, specific deficiencies, training, educational levels, communication skills, and incentives to participate in an experiment.

Fourth, the world is constantly changing and systems used by people are changing accordingly.  Thus, interfaces for designing jobs, operations and environment have to be revisited frequently to account for new behavior and trends.

Fifth, everyone feels is an expert about human behavior on the basis of common sense acquired from life and specific experiences and we tend to generalize our feelings to all kinds of human behaviors but not so expert in the fundamentals of natural sciences such as physics or chemistry.

We think that we have convictions concerning the effects of sleep, dreams, age, and fatigue; we believe that we are rather good judges of people’s motives, we have explanations for people’s good memories and abilities, and we have strong positions on the relative influence of nature and nurture in shaping people’s behavior.  Consequently, the expertise of psychologists and human factors professionals are not viewed as based on science.

Six, physical scientists may enjoy the possibility of “testing to destruction” of prototypes or the materials under study, a luxury that experiments on people forbid or are impossible to do outside the safety range allowed by moral standards, laws, and regulations.  Research on people has to circumvent this major difficulty by using dummies, animals, or willing subjects thoroughly aware and educated to the dangers of the procedures.

Seventh, research on people is regulated by privacy laws and concepts such as consciousness, mental images, fatigue, and motives are highly personal experiences and not open to public inspection while science must be a public affair and repeatable by other researchers.

Consequently, human and social sciences developed terminologies that natural scientists cannot comprehend.  For a experimental natural scientists a variable is a variable.  What is on the left hand side of an equation is the data and what are on the right hand sides are variables and coefficients.

For social scientist you have dependent variables (data), independent variables (factors, manipulated variables, within group variables, between group variables, confounding variables, control variables, treatment variables, sub-group variables, and on).

Controlling an experiment in social sciences is a major project that requires months in preparations to eliminate biases related to people selections and material used by the subjects and the experimenter.

Social sciences have developed many “sophisticated” statistical analyses packages and each discipline prefers its own set of “experimental design” because the members are familiar with the interpretation of results and not because the experiments are pertinent or useful for practical usage.

Multidisciplinary studies are important for a university student to get clear on the many idiosyncrasies of disciplines and start reflecting seriously on what is objective, what is experiment, how valid are research results, how biased are research, and how to correctly interpret results and read scientific studies.

Producing a good reflecting “scientist’ is not an easy task; we are not searching for the appropriate equation but for a good formed scientific and empirical mind. Courses in experimental designs are fundamental even for philosophy students, especially in religious schools.

 “An exercise for taxonomy of methods”

Article #14 in Human Factors

I am going to let you have a hand at classifying methods by providing a list of various methods that could be used in Industrial engineering, Human Factors, Ergonomics, and Industrial Psychology.

This first list of methods is organized in the sequence used to analyzing part of a system or a mission;

The second list is not necessarily randomized though thrown in without much order; otherwise it will not be an excellent exercise.

First, let us agree that a method is a procedure or a set of step by step process that our for runners of geniuses and scholars have tested, found it good, agreed on it on consensus basis and offered it for you to use for the benefit of progress and science.

Many of you will still try hard to find short cuts to anything, including methods, for the petty argument that the best criterion to discriminating among clever people is who waste time on methods and who are nerds.

Actually, the main reason I don’t try to teach many new methods in this course is that students might smack run into a real occupational stress which they are not immune of, especially that methods in human factors are complex and time consuming.

Here is this famous list of a few methods and you are to decide which ones are still in the conceptual phases and which have been “operationalized“.

The first list contains the following methods: operational analysis, activity analysis, critical incidents, function flow, decision/action, action/information analyses, functional allocation, task, fault tree, failure modes and effects analyses, time line, link analyses, simulation, controlled experimentation  operational sequence analysis, and workload assessment.

The second list is constituted of methods that human factors are trained to utilize if need be such as: verbal protocol, neural network, utility theory, preference judgments, psycho-physical methods, operational research, prototyping, information theory, cost/benefit methods, various statistical modeling packages, and expert systems.

Just wait, let me resume.

There are those that are intrinsic to artificial intelligence methodology such as: fuzzy logic, robotics, discrimination nets, pattern matching, knowledge representation, frames, schemata, semantic network, relational databases, searching methods, zero-sum games theory, logical reasoning methods, probabilistic reasoning, learning methods, natural language understanding, image formation and acquisition, connectedness, cellular logic, problem solving techniques, means-end analysis, geometric reasoning system, algebraic reasoning system.

If your education is multidisciplinary you may catalog the above methods according to specialty disciplines such as: artificial intelligence, robotics, econometrics, marketing, human factors, industrial engineering, other engineering majors, psychology or mathematics.

The most logical grouping is along the purpose, input, process/procedure, and output/product of the method, otherwise it would be impossible to define and understand any method.

Methods could be used to analyze systems, provide heuristic data about human performance, make predictions, generate subjective data, discover the cause and effects of the main factors, or evaluate the human-machine performance of products or systems.

The inputs could be qualitative or quantitative such as declarative data, categorical, or numerical generated from structured observations, records, interviews, questionnaires, computer generated or outputs from prior methods.

The outputs could be point data, behavioral trends, graphical in nature, context specific, generic, or reduction in alternatives.

The process could be a creative graphical or pictorial model, logical hierarchy or in network alternative, operational, empirical, informal, or systematic.

You may also group these methods according to their mathematical branches such as algebraic, probabilistic, or geometric.

You may collect them as to their deterministic, statistical sampling methods and probabilistic characters.

You may differentiate the methods as belonging to categorical, ordinal, discrete or continuous measurements.

You may wish to investigate the methods as parametric, non parametric, distribution free population or normally distributed.

You may separate them on their representation forms such as verbal, graphical, pictorial, or in table.

You may discriminate them on heuristic, observational, or experimental scientific values.

You may bundle these methods on qualitative or quantitative values.

You may as well separate them on their historical values or modern techniques based on newer technologies.

You may select them as to their state of the art methods such as ancient methods that new information and new paradigms have refuted their validity or recently developed.

You may define the methods as those digitally or analytically amenable for solving problems.

You may choose to draw several lists of those methods that are economically sounds, esoteric, or just plainly fuzzy sounding.

You may opt to differentiate these methods on requiring high level of mathematical reasoning that are out of your capability and those that can be comprehended through persistent efforts.

You could as well sort them according to which ones fit nicely into the courses that you have already taken, but failed to recollect that they were indeed methods worth acquiring for your career.

You may use any of these taxonomies to answer an optional exam question with no guarantees that you might get a substantial grade.

It would be interesting to collect statistics on how often these methods are being used, by whom, for what rational and by which line of business and by which universities.

It would be interesting to translate these methods into Arabic, Chinese, Japanese, Hindu, or Russian.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

adonis49

adonis49

adonis49

November 2020
M T W T F S S
 1
2345678
9101112131415
16171819202122
23242526272829
30  

Blog Stats

  • 1,429,207 hits

Enter your email address to subscribe to this blog and receive notifications of new posts by email.adonisbouh@gmail.com

Join 777 other followers

%d bloggers like this: