Lee and Hubona 2009: A Scientific Basis for Rigor and Relevance in Information Systems Research
Lee, A.S., and Hubona, G.S. (2009) "A Scientific Basis for Rigor and Relevance in Information Systems Research," MIS Quarterly (33:2), pp. 237-262, June 2009.
- Context: IS research is multi-paradigmatic and multi-method. Although qualitative research is well-accepted, it is not always clear what is the philosophical or scientific basis for determining the rigour of a study, especially when compared to positivist quantitative studies, whose scientific rigour, hailing from methods derived in the natural sciences, is well established and widely accepted.
- Objectives: Present the MPMT (modus ponens/modus tollens) framework as a perspective to evaluate the validity of IS research, regardless of its paradigm; specifically, positivist and interpretivist, and to a lesser degree action and design research, are examined using this framework.
- Methods: This research essay uses syllogistic reasoning using the classic modus ponens/modus tollens arguments to construct the framework. It uses this framework to assess three highly cited positivist IS articles, and assesses six interpretivist articles to distinguish formative and summative validity.
- As in positivist research in the natural sciences, modus tollens is the normal method used for hypothesis testing: A hypothesis is put forth (major premise); a null hypothesis is presented that contradicts the major premise (minor premise); a treatment is applied or tested. If the conclusion validates the null hypothesis, then the falsity of the null hypothesis proves the falsity of the major premise (this is modus tollens), and hence the theory is proven false. However, if the null hypothesis is not invalidated by the test, then the conclusion is that the theory is not invalidated, and can continue to be tested. This structure is valid for assessing positivist, interpretivist, action and design research.
- If the empirical study is well designed and "validly" tests what it attempts to test, according to the criteria for that research paradigm, then the test of the theory has "formative validity." However, the theory itself is supported only by "summative validity," which requires a distinct test that uses the conclusions of the formative study to predict an outcome on distinct data. If the prediction is as expected, then the study has "summative validity." Such tests are required for theories to be considered properly tested and validated. The authors demonstrate (with further details in appendices) how to conduct such tests of summative validity, particularly for quantitative positivist studies. Tests of summative validity should not be confused with repeating tests of formative validity, which risks the fallacy of affirming the consequent.
- Conclusions: Interpretivist, action and design research can be just as scientifically rigorous as positivist studies using the MPMT framework. In order to fully validate theories, tests of summative validity are required, beyond the typical tests of formative validity.