Dan Nexton kicked off a debate on the value of academic job talks.  Dan provides a nice follow-up that links to blog posts by Tom Pepinsky, Jeremy Wallace.  Tom Oatley has a provocative post on a potential historical explanation for the  emergence of the job talk.

Tom Oatley has a nice point that we should look for some data to address this debate. He focuses on a historical explanation.  But I also have a contemporary one on the use of talks as part of an evalaution process.

What are (job) talks for?

In my original post I mentioned that department speaker series often have the same format as job talks.  The point was that job talks may not be the most efficient ways to make a decision, but they: a) can complement other pieces of information for evaluation and b) they have other positive externalities.  We have speaker series because they are valuable to a dept.  A job talk can also provide some value outside of just evaluation.

I have another example.

At WashU we’re on the second week of listening to our Ph.D. students present their 3^rd year papers in the department.  These paper presentations take the form of a 20-30 min talk.  We actually have more of these presentations in a year than we have job talks.

I was on the committee that enacted this new requirement two years ago.  Our process for the 3rd year papers is that we have a faculty committee read the papers and we also included a public presentation of the work.  What was the logic of having a public presentation?

At least one faculty member advocating for presentations did say argue that students would eventually have to give a job talk and this would be good practice.  This is a similar logic to Tom Oatley’s path dependence argument.  But the more important reasons (at least in my mind) were:

• Public presentations actually incentivize more effort.  Not only do you have to produce a paper; you also have to anticipate questions, know your data, and be able to explain your methods.  Papers can be narrow, but presenters have to almost always be more broad than their written work.
• This was a low cost way to communicate research not only to faculty but to other graduate students in the program.  First year graduate students probably don’t read these papers (or job market candidate files), but they can learn quite a bit during a research presentation.
• Attending these talks and really engaging the speaker is a way to signal to the students in our program that we take this process seriously and we our invested in the graduate program.

We could play with the format and have the presentation 5 min or 45 min.  The point is that these presentations serve multiple purposes.  If we just wanted to evaluate the work as efficiently as possible, we would circulate the papers to the faculty.  Hell, why not just circulate it to one or two faculty.  Who said an efficient process needs to impose uniformly high costs on all faculty members when delegation is possible?  But these presentations are more than just about evaluation.

Some of these reasons may not apply to a job talk.  But I think the quick point is that attending a job talk is a pretty low cost activity that teaches you about the speaker, the project, and informs the speaker about your department.

Making Better Decisions

My main point is that the job talks serve a evaluation function along with an intellectual and professional function.  But I think I might be alone in thinking about job talks in this way.  So let me go back to the big question on how to evaluate candidates.

I think there is a tremendous amount of uncertainty in junior faculty hiring for a number of reasons.

I’m all for a better search process.

The one common criticism is that job talks allow faculty to be lazy and make decisions based on the talk.  I’m not sure what the process looks like at other departments, but here is ours:

• Job candidates are selected by a search committee that has already read the work.
• Most of us have some form of a one-on-one or group meeting with the candidates.
• Many of us read at least some of the written work.
• The search committee has to circulate a report 72 hours before the faculty meeting when we discuss the candidates.
• We have a faculty meeting where the search committee sets the agenda and then we discuss the candidate(s).

Most meetings have a few faculty that had to miss the talk for teaching or travel (although most faculty attend).  Faculty also have a range of exposure to the written work.  Committee members read a ton and a bunch of faculty outside of the committee almost always engage the written work.

Is it plausible that removing the job talk would incentivize more people to read more and engage the written work?  Sure, it seems plausible to me.  But would this lead to a better decision?

I personally agree that some people overweigh the value of the job talk.  I think letters are more informative, but many people place way too much stake in them as well.  I’m mostly a “read the dissertation” type person, although others have cautioned me that dissertations can easily be largely ghost written by a dissertation advisor.  Who’s right?

My colleague Matt Gabel has some work on group decision-making in medicine.  The simple point is that the best decisions are made when individuals that make different types of errors are in the same room together.  It doesn’t matter who is right (or who is wrong).  What matters is how me make a decision as a group.

I think it is equally plausible that having people that over weigh the job talk or the letters or the dissertation in the same room leads to better decisions.  A decision solely based on a job talk would be a terrible decision.  But having a faculty member or two advocate for or against candidate based on the talk seems like it could be a part of a healthy discussion about hiring.

This last part about decision making is really just a mix of conjecture and my application of group decision making (or portfolio theory) to political science hiring.  I’m not 100% sold on the job talk, but like I said, I’m mostly a “read the dissertation” type person.  I also like to attend the job talks.