By: Sebastian Tello-Trillo
Letters of recommendation (LOR) for the economics Ph.D. market is a tool that has been long-used to assess information about a candidate. There are many aspects of a LOR that can vary, but there is no tangible information other than one’s network and peers on what people tend to do. In this article (the first of a series), we hope to shed light on some of these practices and opinions on LOR. We hope this is informative for prospective students in the market, writers of the letters, and hiring committees.
To capture this information, we have constructed an anonymous survey that was shared among peers and on Twitter on February 11, 2020. We will be discussing the results from this survey in this and other future articles on this topic.
Who responded?
As of the writing of this article (February 18, 2020), there are 42 responses and 41 “valid” responses. 50% of our valid sample (henceforth, “sample”) self-identify as a health economist. 95% of our sample are academics. Among those, 62% where people in associate/full professor ranks and 38% reported assistant rank. Of the whole sample, 39% self-describe as female. Most (92%) were from institutions in the U.S. 90% indicated that they had written at least 1 letter, 41% have written between 1-5 letters, and the rest have written more than 5 letters.
Length of Letters
The average letter written by a survey respondent is 2.6 pages, while the median is 2.5. 11% were less than 2 pages, and about 15% were more than 3 pages. When we ask people to report on the letters they receive, the average length is 2.7 pages, while the median is 3.
Most people believe that a “short letter” is any below 1.5 pages. About 80% of our respondents indicated this is the case (average is 1.27, and the median is 1). On the other end, 76% responses indicate that a letter of 4 or more pages is “too long.”
Revealing family (or personal) information on letters
70% of respondents do not include personal information about spouses or family in their letters. The rest said that they have done it once the student asks for it or when they think it seems relevant.
When it comes to practices that the respondents have seen, 13% of the letters they read have some personal information on the student. (75% of respondents indicated that at least 1 of 10 letters came with personal information on the student, whereas 25% reported 0 letters.) Through comments, some people explain that sometimes they feel the need to reveal a preference for a location that is due to a family situation or to explain a fit about a location.
In terms of opinions, 52% of respondents think letters are not the right place to mention these issues while 41% responded Yes/Sometimes. The rest were not sure. For the respondents that answer “Yes” or “Sometimes” they mention two main reasons of why they think it is okay for to have this type of information: (1) because a student request it and (2) mentioning geographical preferences is valuable for the hiring committees of places where it is traditionally an issue with their applicants. A quote from a respondent illustrates this is “it may be the only place to signal location preference, which for us is very important.” Similarly, some mentioned that they put this information in other places “Usually when I have discussed this, it’s vis-a-vis a particular job, so I don’t put it in the letter but will say it in email.”
Information about the recommender
62% of people reported that “a couple of sentences (as a max)” is fine for talking about the recommender. Of those who thought more was acceptable, one particular responder suggested to explain, “Why are they qualified; in what domains do they know the person.” Another responder added “If the recommender isn’t well known, or is in a nontraditional position, some description is helpful.”
Comparisons with other students
62% of the respondents reported that “Sometimes,” they make comparisons (of many sorts) in their letter. 24% reported “yes” and 14% no/never. For the set of people who reported Yes/Sometimes, 56% reported that they include comparisons relative to the recommenders’ students. 45% mention that they include comparisons relative to students from the program. (These are not mutually exclusive). The other responses may potentially be about advanced junior or senior letters as they mentioned they compare the candidate to the “trajectory to an established economist”. Others comment that they use “Comparison to close comparables, sometimes categorical rank (e.g., best in several years)” or talk about the similarities of those students with other junior economists “S/he reminds me of X.”
When asked about the letters that they read, out of 10 letters, about 6.5 letters report some comparisons. 85% of the respondents indicate that at least 5 of the 10 letters include some comparisons.
Finally, in terms of opinions, about 67% people think these comparisons are helpful, 21% report sometimes or not sure, and the rest report that these are not helpful. A couple of respondents caveat their answer, mentioning that they understand these comparisons are subjective. One response indicated that “Not for top schools. But when a professor at a lower-ranked school says this is their best student, we consider that student.” Someone explained that they answered “no” because they don’t know anything about the other students. Finally, most people who found it helpful commented that they could observe how the comparisons have developed over the years and have a better understanding of the recommenders’ expectations for that person. Here is one quote from a responder “Extremely helpful, because it’ hard to disambiguate all the enthusiastic letters otherwise. These sorts of comparisons are one of the first things I look for but only appear half the time.”
Next Up
These are a few insights into the current results of the survey we have sent out. In the next set of articles, we plan to report on the rest of the characteristics mentioned in the survey (do recommenders mention any drawbacks of their students? How accurately do they rate students, and do they make calls/emails)? In addition, provided we get more respondents, we may be able to break it down by some of the general characteristics. If you have any questions, please let us know you can email me at sebastian.tello@virginia.edu or ASHEcon newsletter editor David Slusky at david.slusky@ku.edu.
Take the ASHEcon Letter of Recommendation Survey
As the job market process is wrapping up this year, we wanted to nudge you to help us fill out a survey regarding practices on letters of recommendation for job market candidates. You can take it here. It should take you about 5-10 minutes. We would greatly appreciate it if you can answer it and incentivize other letter writers also to answer them as well.
Sebastian Tello-Trillo is an Assistant Professor of Public Policy and Economics at the University of Virginia and a Co-editor of the ASHEcon Newsletter.