By: Sebastian Tello-Trillo
In the last newsletter we posted the summary of the results from the survey on LOR. We included descriptive information on who responded in the survey, the length of letters, whether people reveal personal information on letters, amount of information about the author of the letter, and if letter writers included comparisons with other students. For those results, please check this link. In this article we continue to explore findings from the survey.
Modifying Letters for different types of jobs
We asked a question about if authors of letters “modify” their main letter to cater to different types of jobs. Out of 36 respondents, about 58% of respondents indicate that “sometimes” they modify the letters. About 22% responded that they always tailor and 20% respond that they do not tailor them. In terms of opinions, 87% of respondents think that letter should contain different information for different types of jobs. The comments on this question reveal that most people think this is optimal but have time constraints and hence they only do it for some jobs.
Describing the student’s drawbacks
Of the 37 responses approximately 70% reported that they have never mentioned drawbacks. Out of the people who reported “sometimes”, they stated that the drawbacks were comments on their progress within graduate school. As in comments on the degree of independence in research projects, the speed and pace of progress of research, and if they have “issues” finishing projects. Some adjust this depending on the perceived “quality” of the institution they are writing for.
The 70% reporting not-writing drawbacks is supported by the fact that around 67% of respondents report that less than 1 percent of the letters they read have negative comments. The remaining people reported between 2-3 letters out of 10 they read had some negative comments.
Finally, in terms of opinions, 58% of people report that one should never include any drawbacks on the letters. Around 34% responded “Yes” or “Sometimes” and 5% were not sure. The comments on the “Yes/Sometimes” can be summarize on the quote of one of the responders “I mean, if you want to do the best for your student, no, because that’s the equilibrium we’re in. But it would make them more credible in the aggregate.” Other respondents thought it would be “better” from the hiring side if people were transparent about the drawbacks but understand why this is not common.
Describing the students’ strengths
We ask a question on how “accurate” the authors describe the student’s strength. About 39% of individuals say they “accurately” report their student’s strengths, and about 58% report they “overstate modestly” with the leftover 3% reporting they overstate significantly. We then ask about how accurately you think other authors describe their student’s strengths. When asked about their own reporting, no one reported that they “overstate significantly” in contrast about 26% of people think others “overstate significantly”. About 64% of individuals think others “overstate modestly”, while only 8% of individuals report that others report “accurately” about their students (in contrast to the 39% of self-reported). In other words, there is an overlap between the “overstate modestly” but a large discrepancy on “overstate significantly” and “Accurately”. There is an interesting heterogeneity across the gender of the authors of the letters (with the caveat that our sample size is small), Male authors tend to “overstate modestly” in a higher share than female authors
Informal channels of recommendations
This question asked, “How often do you call or send direct emails about your own students?” About 42% of respondents reported that they do this for the majority or every student they have. About 30% of them report that they do this for a select group of students and 25% responded that they rarely or never do this. This is in contrast with the share of people who report receiving a personalized call or email. On average, people report that the receive calls or emails for 10% of their applications.
Effectiveness & Importance of the letters
We then asked, “How effective you think your letters are?” About 11% of respondents reported that they consider their letters effective, about 47% reported their letters being “somewhat effective”, 36% reported “neutral” or neither effective nor ineffective, and finally 5% of them reported somewhat ineffective. In terms of importance, 51% of respondents reported that they consider LOR as “extremely important”, 42% reported as somewhat important while the reminder 5% somewhat unimportant.
To understand this in a deeper level. We gave individuals 6 items that form part of the application process: job market paper, LORs, research that is not the job market paper, recommendations made outside of the main process (such as calls, emails, or conversations), the teaching needs of a particular job and the research and teaching statements. Each of them had to rank them from “most” (1) to “least” (6) important. Comparing LORs relative to other application materials, it was ranked as the second’s most important. We average the ranking for each item and the results are the following:
- Job Market Paper…………………………………………………2.02
- LOR………………………………………………………………………2.62
- Non-JMP Research………………………………………………3.03
- Informal recommendations…………………………………3.57
- Teaching-needs fit……………………………………………….3.97
- Statements (Research, Teaching, etc.)………………….4.17
When should students remind letter-writers about their letter?
We also asked a question on the importance of “who” the author is (beyond the content of the letter), the results reveal that the author itself carries weight: 37% of people reported extremely important, 57% of people reported “somewhat important”. The rest reported somewhat unimportant and neither important nor unimportant (5%).
Finally, we asked when students should remind their letter-writers about their letter. Around 43% of respondents reported to remind them between 1 and 2 weeks before the deadline. Around 21% reported one month before the deadline. So overall 64% of respondents prefer to be reminded within the 4-weeks prior to deadline. The other 40% of respondents had different answers that were harder to categorized.
Sebastian Tello-Trillo is an Assistant Professor of Public Policy and Economics at the University of Virginia and a Co-editor of the ASHEcon Newsletter.