Weber 2012: Evaluating and Developing Theories in the Information Systems Discipline

Weber, Ron 2012. “Evaluating and Developing Theories in the Information Systems Discipline,” Journal of the Association for Information Systems (13:1), pp. 2-30.
 

• Rationale: There is much interest in theory building, yet detailed guides on how to do this are still lacking in clarity.
• Objectives: Weber uses a formal ontological framework to define the elements of a theory, and thus expound in detail the elements of high-quality theory. He illustrates his evaluatory techniques by evaluating Griffith et al (2003) in detail.
• Theoretical background: Bunge's (1977, 1979) general ontology of elements of things in the world provides the framework that provides precise descriptions of the elements of a theory.
• Key questions: What is the definition of a theory? What are the necessary elements of a high-quality theory?
• Key findings:

1. Definition of theory: A theory is defined as "a particular kind of model that is intended to account for some subset of phenomena in the real world" (p. 4).
2. Parts of a theory: All theories have at least three parts: constructs (attributes or variables), associations (relationships), and states (the set of values of constructs). Some also have a fourth part, events, defined as the record of change from one state to another. For each of the four parts, the boundaries of what is within the theory and what is beyond the scope of the theory must be clearly defined. Each of the parts must be clearly specified.
3. The theory as a whole: A high-quality theory must demonstrate importance (it is non-trivial), novelty (it is not a rehash), parsimony (it is not excessively complex), an appropriate level (not too general nor too tied to specific data), and falsifiability (it must be sufficiently precisely specified that empirical tests can be built to validly attempt to refute it).

• Key contribution to knowledge: Bunge's ontology provides a robust framework for specifying the elements of a theory's parts. Weber's specification of these important parts is new and tremendously helpful. His specification of the elements of the whole is not in itself new, but is necessary in the context of his novel specification of the parts.
• Key implications: This paper will likely become a standard evaluation guide for theory building and evaluation in IS and in other fields for years to come.
• Comments: Overall, I believe this is a fantastic piece of work, for reasons I mention above in "Key findings" and "Key contribution". Nonetheless, I see two significant shortcomings:

1. Weber's definition of "theory" is very disappointing and not very useful. It serves its purpose within his article, since he dedicates an entire section of seven paragraphs to explain his definition. However, it is not an exportable definition–it is not usable as-is in other papers that would want to talk about theory, since it does stand alone as a clear and general definition. I believe, though, that his definition could be reworked to become exportable from the perspective of the framework he described in the paper. His use of the word "phenomena" is particularly murky, as seen in my next criticism.
2. In light of Burton-Jones et al's work [contact me for reference] on the three types of theoretical conceptualization in IS–variance, process and systems theories–Weber's framework does not clearly embrace the diversity of theoretical conceptualizations. He mainly talks about "constructs", which seem to me to refer exclusively to variables in variance theory. I believe that much of the lack of clarity in his framework lies in his use of the word "phenomena". Based on his very clear definition of this word (p. 5), his meaning (things, properties, states and events) seems to correspond with Burton-Jones et al's "concepts" (variables, events, and systems). Weber's presentation adopts mainly a variance theory approach. He does recognize the process theory approach in his recognition of events, but he tends to see them as ancillary, rather than as focal phenomena that are more important than constructs (variables) in some theories. In contrast, he barely considers things themselves (Burton-Jones et al's systems) as focal phenomana. I could say a lot more on this, but this suffices for now. However, I believe his framework is sufficiently robust that it could be successfully expounded to fully incorporate these elements of shortage.

References

• Bunge, Mario, 1977. Treatise on Basic Philosophy: Volume 3: Ontology I: The Furniture of the World Springer.
• Bunge, Mario, 1979. Treatise on Basic Philosophy: Volume 4: Ontology II: A World of Systems Springer.
• Griffith, T. L., Sawyer, J. E., and Neale, M. A. 2003. “Virtualness and knowledge in teams: Managing the love triangle of organizations, individuals, and information technology,” MIS Quarterly, pp. 265–287.