Research Design - Three Questions to Ask about Generalizability of Research Results

Research design refers to the manner by which a research project is set up. Research design sets the parameters of the research project. One of the aims of a research project is to achieve results which could be generalized to the wider population within which the study was conducted.

A Simple Example

For instance, let's say I want to know which leadership styles are deemed most effective in Cambodia. I set up my research project to survey a certain number of the members of Cambodian society in order to gain insight into effective leadership styles. If I want to gain a clear understanding of this phenomena throughout all of Cambodia then I have to solicit from a random sample of all parts of Cambodia with an adequate number of participants, if I want to gain results which are generalizable to the whole Cambodian population.

Three Questions to Ask to Ensure Generalizability

Kerlinger and Lee (2004) point out three questions or criteria of for assessing research design. Those three questions are:

1. Does the design answer the research question (or does the design adequately test the research hypotheses)?

2. Does this design adequately control independent variables including extraneous variables?

3. How much can we generalize the results of the study to other participants, groups, and or conditions?

Generalizability Protects External Validity of Research Results

Of the question of generalizability Kerlinger & Lee observed "this is the probably the most complex and difficult question that can be asked of research data because it touches on technical matters, but also on larger problems of basic and applied research" (p.474, 475). What is more, they see generalizability not necessarily the goal of most basic research. Still, it is an important question and goes towards establishing the external validity of the conducted research.

Conduct Confirmatory Research to Validate Prior Results

With that said, a good scientist will not simply assume that country specific data is or is not generalizable, but will conduct a new test to confirm whether or not the results of one particular study are generalizable to other situations. Van de Vijver (2009) noted two types of generalizable studies including structure-oriented and level-oriented generalizability studies. He explained that in "structure-oriented generalizability studies, one would accumulate data from various countries with the instrument and check to what extent the two-dimensional structure is found in each of these. In other words, the generality of the structure elsewhere is addressed." In regards to level-oriented generalizability studies he noted, "Level-oriented generalizability studies usually build on studies in Western countries and examine to what extent differences observed there can be generalized to other cultures."

Kerlinger, F. N. & Lee. H. B. (2004) Foundations of Behavorial Research, 4e. Belmont, CA: Cengage Learning.

van de Vijver, F. J. R. (2009, March; revised). Types of Comparative Studies in Cross-Cultural Psychology. Online Readings in Psychology and Culture (Unit 2, Chapter 6). ©International Association for Cross-Cultural Psychology