Blog by Nate Archives: Registering Experiments (Jan 20, 2013)

[My content migration continues to my new blog.  You can’t wait.  Can you?  This is a blog post from 2013.  Update:  I have fully drank the registration cool-aid.  This includes a special issue call for papers at Comparative Political Studies. ]

Registering Experiments

Blogging has been very light lately do to a bunch of administrative responsibilities.  Two things.

1.  I am kicking around the idea of putting together an edited vol on null results are failed experiments.  Drop me an email if you have thoughts or are interested in contributing.

2.  Jamie Monogan circualted the following email on the Political Methodology listserv on registering experiments:

The most recent issue of Political Analysis (vol. 21, issue 1) features a Symposium on Research Registration, or the idea that scholars in many cases can prespecify a research design prior to observing the outcome variable. The merits of preregistration are still debated, and the discipline currently does not have a comprehensive central registry. However, we write at this time to draw people’s attention to a handful of proto-registries that are available to researchers. From our experience, the best way to figure out what you think about registration is trying it in one of your own projects!  In particular, any researcher who is interested in self-registering a study is welcome to take advantage of the the Political Science Registered Studies Dataverse (http://dvn.iq.harvard.edu/dvn/dv/registration).

This dataverse is a fully-automated resource that allows researchers to upload design information, pre-outcome data, and any preliminary code. Uploaded designs will be publicized via a variety of free media. List members are welcome to subscribe to any of these announcement services, which are linked in the header of the dataverse page.  Besides this automated system, there are also a few other proto-registries of note:

EGAP: Experiments in Governance and Politics: <http://e-gap.org/design-registration/> The EGAP website has a registration tool that now accepts and posts detailed preanalysis plans. In instances when designs are sensitive, EGAP offers the service of accepting and archiving sensitive plans with an agreed trigger for posting them publicly.

J-PAL: The Abdul Latif Jameel Poverty Action Lab: <http://www.povertyactionlab.org/Hypothesis-Registry> J-PAL has been hosting a hypothesis registry since 2009. This registry is for pre-analysis plans of researchers working on randomized controlled trials, which may be submitted before data analysis begins.

The APSA Experimental Research Section: <http://ps-experiments.ucr.edu/> The experimental research section of APSA hosts a registry for experiments at its website. (Please note, however, that the website is down for maintainence until February.)  Additional information about the method and the resources are printed in this month’s issue of Political Analysis. Additionally, anyone who has questions about these resources is welcome to contact Jamie Monogan off-list (monogan@uga.edu).