Tsang & Kwan 1999: Replication and Theory Development in Organizational Science: A Critical Realist Perspective
I read this as a critical realist piece in management. In addition, its treatment of replication in managemetn and organizational sciences (including qualitative research) is pertinent to my study of systematic literature reviews.
- Rationale: Replication is an important aspect of developing scientific knowledge. However, in the social sciences, particularly in management and organization studies, replication of empirical findings is relatively rare.
- Objectives: This conceptual article argues for the value of replication in social science research, and presents critical realism as a philosophical basis that justifies its applicability in the social sciences.
- Theoretical background: Critical realism is the underlying philosophical and theoretical perspective: it sees social sciences as essentially treating the subjects that are equally as real as in the natural sciences. However, because of the openness of social systems, predictiveness of social science research is infeasible; replication is nonetheless valuable for building theory.
- Methodology: This is a conceptual research essay; it uses logical argumentation to make its case.
- Key questions and findings:
- “Is replication possible in the social sciences?” (p. 763) Replication is possible in the social sciences just as it is in the natural sciences; natural science experiments are not completely closed–they also share some of social sciences’ openness. Although social researchers’ theoretical prejudices might influence their observations, they can agree on the raw facts of the social phenomena regardless of their theoretical biases, and they can use measurement instruments that are independently developed, thus limiting their biases in studying their phenomena of present interest.
- What kinds of studies can be classified as replications? There are six kinds of replications, based on a 2×3 matrix (p. 766):
- What is the role of replication in building theory in the social sciences? For theory development, replications are important for two main reasons: first, whereas any theory can be tweaked to accommodate or fit the empirical data, a replication serves as a test of a prediction based on applying the preformulated theory to independent data; an accurate prediction (that is, successful replication of results) would provide strong support for the theory. Second, although the open systems of social sciences are not subject to the strict falsification criteria called for by Karl Popper (and positivist social scientists), successive replications do serve to increase or decrease the confidence in a theory.
- Key contribution to knowledge: Replications are not only feasible in social science research, but they are actually carried out. It is important, though, to understand the different kinds of replication (categorized in Table 2 above) in order to identify what exactly is called replication.
- Key implications: The “scattered pattern” of knowledge accumulation (as employed in organizational learning research) involves mainly unique studies with relatively little replication; when replication is carried out, it is usually of the “generalization and extension” variety. Although this approach generates multiple innovative studies, it is not as helpful for building a body of knowledge. In contrast, the “multifocal pattern” (as seen in research on vertical integration in transaction cost economics) involves more kinds of replication that make results of studies more comparable; this facilitates building and refinement of a theoretical knowledge base. The authors recommend what they call an integrated approach that involves both the scattered and the multifocal patterns.
- Comments: First, I find this a particular powerful argument for the need of replication in social science research, and particularly in qualitative research, in contrast to common claims that replication is not applicable in this domain.
Although Tsang and Kwan suggest that the main reasons for few replications are because of a failure of appreciating its theoretical value and because of a qualitative research bias against it, I feel they skirt one very pragmatic issue: editors of top journals value innovative research over replications of innovative research. Although Tsang and Kwan do refer to a survey by Madden, Easley and Dunn (1995) (p. 759; I didn’t read it) that surveys editors of both natural science and social science journals, they do not mention if top natural science journals are more welcoming of replications than are top social science journals. From my limited experience of journals like Science and Nature, and my more extensive experience with computer science and software engineering journals, it seems that the standard is much the same: “top” journals don’t want to publish old, stale stuff, such as replications (as they perceive them). Yes, replications are important, but let someone else publish them. This is just my suspicion.
Another more minor note is that although Tsang and Kwan propose an “integration” of the scattered and multifocal patterns, I feel what they propose is actually a plurality; that is, they are recommending to do both; they do not propose how to integrate the two approaches for holistic knowledge synthesis.