Experimental Design And Bayesian Interpretation By Glenn W. Harrison

Experimental Design and Bayesian Interpretation

Glenn W. Harrison

†

October 2020

H. Kincaid and D. Ross (eds.),

Modern Guide to the Philosophy of Economics (Cheltenham, UK: Elgar, forthcoming 2021).

Table of Contents

1. Missing Links to Economic Theory and Econometrics. . . . . . . .......................... -3-

2. Welfare Analysis From the Intentional Stance........................................ -5-

3. Bayesian Econometrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ......................... -9-

A. Examples.............................................................. -10-

B. The General Case for Bayesian Methods..................................... -12-

4. Insights from Some Debates Over Medical Ethics in Clinical Trials ..................... -15-

A. Experimental Design Prior to the Experiment................................ -16-

B. During and After the Experiment.......................................... -19-

5. Implications.................................................................. -20-

6. Conclusion . . . . . .............................................................. -24-

References ..................................................................... -28-

†

Department of Risk Management & Insurance and Center for the Economic Analysis of Risk,

Robinson College of Business, Georgia State University, USA. Harrison is also affiliated with the

School of Economics, University of Cape Town. Valuable comments from Don Ross are

appreciated. E-mail contact: gharrison@gsu.edu.

Experimental methods have grown in importance in economics in the past 60 years. There have

been several broad stages in this evolution. In the 1960s there was the use of social experiments to

examine major policies in natural settings, in the 1970s there was the use of laboratory experiments to

test economic theories in artefactual settings closer to theory, in the 1990s there was the use of field

experiments to test economic theories in artefactual and natural settings closer to theory, and in the

2000s there was the use of randomized experimental interventions in developing countries. Along the

way, experimental designs evolved to address different question. The appropriate design depends on

the question being answered, and the type of inferences to be made.

It is perfectly appropriate for an experimental design not to have any randomization at all, such

as when one is evaluating whether double-oral auction markets converge to an equilibrium price

determined by induced demand and supply curves (e.g., Smith [1962]). Or when one is presenting

subjects with risky lottery choices in order to infer risk preferences, and test which theories of risk

preference characterize which individuals (e.g., Hey and Orme [1994]). And randomized interventions

are not unique to field settings, and have been widely used in laboratory experiments to study the

effects of futures markets on the informational efficiency of asset markets (e.g., Forsythe, Palfrey and

Plott [1984] and Friedman, Harrison and Salmon [1984]).

It is also perfectly appropriate for an experiment design to be initially tethered to some

economic theory, such as when selecting parameter values to equalize expected payoffs when evaluating

theoretical predictions from single-unit auctions with varying numbers of bidders (e.g., Cox, Roberson

and Smith [1982]). Or making sure that the key axioms of models of bargaining behavior are

operationalized when testing them (e.g., Roth and Malouf [1979]). And it is appropriate for

experimental designs to be motivated by the need to extend initial theories as suggested by prior

experiments, such as models of sealed-bid behavior (e.g., Cox, Smith and Walker [1984; §V]).

A general concern with experiments spanning this variety of applications in economics is the

-1-

link between the design of the experiment and the interpretation of results. Increasingly, with

specialization and the rise of academic silos, we have separated these. I want to argue for a rejection of

that separation, and for the necessity of a Bayesian approach to both. There are two themes to the case

for this position.

The first theme is the need for the design and interpretation of experiments not to be divorced

from economic theory and the science of econometrics if we are just to do our descriptive job well. I

have argued for this point elsewhere, and summarize in section 1. The Bayesian approach provides an

easily justified contribution to the documentation of empirical regularities and statistical tendencies.

The second theme is the derived demand for a closer connection between the design of

experiments and their interpretation for normative reasons. If we believe that our experiments might

affect the welfare of subjects, or even if subjects believe they might, then we must take this into

account in the design phase. By “take into account” it is important to allow for the special case of

implicitly ignoring it, since that characterizes many of the practices we observe (e.g., the “equipoise”

issue discussed below). And this is referred to as a “derived demand” in part because it rests on

developments in behavioral welfare economics, reviewed in section 2, that purport to evaluate the risk

of doing harm to subjects from interventions. It is also a “derived demand” because it rests on

developments in Bayesian econometrics, reviewed in section 3, that allow us to make statements about

the risk of doing harm to subjects at the granular level of an individual choice.

Case studies of medical ethics of clinical trials provide key insights into how these issues tightly

connect experimental design and interpretation. Section 4 reviews the debates over the early clinical

trials of the Extracorporeal Membrane Oxygenation (ECMO) surgical procedure that provides external

support for the heart and lungs with artificial oxygenation of red blood cells. The specific ECMO trials

concerned newborn babies in distress, at the point of being close to death without any treatment.

-2-

/ 33

363

FAQs of Experimental Design and Bayesian Interpretation by Glenn W. Harrison

What are the main themes of Glenn W. Harrison's work?

Glenn W. Harrison's work focuses on the intersection of experimental design and Bayesian interpretation in economics. He discusses how experimental methods have evolved over the past six decades, emphasizing the importance of integrating these methods with economic theory and econometric practices. The document also highlights ethical considerations in experimental design, particularly regarding the welfare of subjects involved in experiments. Harrison argues for a Bayesian approach to ensure that experimental designs are not only methodologically sound but also ethically responsible.

How does Harrison address ethical concerns in experimental design?

Harrison addresses ethical concerns by emphasizing the need for rigorous welfare analysis in experimental design. He argues that researchers must consider the potential harm to subjects when designing experiments, particularly in fields like economics where interventions can significantly impact welfare. The document reviews case studies from medical ethics to draw parallels with economic experiments, suggesting that ethical considerations should be integrated into the design phase. This approach ensures that the welfare of subjects is prioritized and that the implications of experimental findings are responsibly interpreted.

What types of experiments does the document discuss?

The document discusses various types of experiments, including social experiments, laboratory experiments, and field experiments. It outlines how these methods have been employed to test economic theories and evaluate policies over the years. Harrison highlights the use of randomized interventions, particularly in developing countries, and how these approaches have evolved to address different research questions. The importance of selecting appropriate experimental designs based on the specific questions being investigated is also emphasized.

What is the significance of Bayesian methods in this context?

Bayesian methods are significant in Harrison's work as they provide a framework for integrating prior knowledge with experimental data. This approach allows researchers to make informed inferences about risk preferences and welfare outcomes based on observed behaviors. Harrison argues that Bayesian reasoning enhances the interpretation of experimental results, making it possible to assess the potential impacts of interventions on subjects' welfare. By applying Bayesian methods, researchers can systematically pool data from various sources, improving the robustness of their findings.

Experimental Design and Bayesian Interpretation by Glenn W. Harrison

Key Points

FAQs of Experimental Design and Bayesian Interpretation by Glenn W. Harrison

Related of Experimental Design and Bayesian Interpretation by Glenn W. Harrison