By Betsy Q. Cliff

Finding the best data to fit your research question is an ongoing challenge for health economists. One tool, perhaps underused in the profession, is open-records laws that entitle the public to request data or reports from government. At a federal level, the Freedom of Information Act (FOIA), can be used to obtain data from any federal government agency. All states have similar laws that apply to state and local governments. A FOIA request, which people often use to refer to any open-records request, is typically filed in instances in which you know information exists but it is not public. While gathering data this way takes more time and effort than traditional off-the-shelf data sources, the reward is a unique dataset and potentially the ability to answer questions in a novel way.

From my own experience, using FOIA to gather information isn’t too tricky but there are some nuances to it. I emailed with two experts in the process: Sarah Kliff, a health reporter at The New York Times who regularly investigates spending and quality in the American health system, and Joe Shapiro, an economist at the University of California Berkeley who has used open records laws such as FOIA extensively in his work, notably to understand the impact of clean water laws.

Question: What kind of information do you FOIA?

Sarah: I am usually looking for something specific when I file a FOIA request, rather than using it as exploratory. For the reporting that I do, the information has most often been inspections of health care facilities and investigations of hospitals.

Joe: I have filed FOIA requests for specific data needed for a research paper that I’m certain are not already in the public domain. In some cases, a staff person at a government agency can access the data but recommends filing a FOIA as an administrative mechanism to organize how to prepare the data for sharing. In other cases, some form of public records indicate the existence of data but I cannot find a person to speak with about it or find a public record of the actual data, so a FOIA helps access it.  In still other cases, a public version of a government database excludes years, variables, records, or other attributes necessary for research, and public records indicate that this information is available in the internal version of the data. Another version of this same issue is that some public databases show only current records and overwrite past versions, and a FOIA can obtain a 5- or 10-year-old snapshot of what is otherwise only publicly available for the present.

Question: What is the general process? Does it differ by agency or level of government?

Sarah: It usually starts with submitting a request. How you do it can vary by who you are submitting to — some places, like the Food and Drug Administration, have a form you can use, while others have an email address you send your FOIA to. The request should be pretty basic: who you are, what you are requesting, and why you feel it is in the public interest to have that information be made available.

Joe: The process differs by agency and branch of government. Many groups have a web form you complete or there is an email address to send a formal letter.

Question: How long do you wait for a response and what do you do if you don’t get one?

Sarah: Under federal law, agencies are required to respond to a FOIA request within 20 business days — not necessarily with what you are asking for but at least acknowledging that they are working on it. I will also often email the public information or FOIA officer to check in with them about what might be the hold up.

Joe: Many of these laws require a response within a few weeks, though in practice this regularly doesn’t happen and I sometimes have to follow up several times. Often the response within 30 days is an acknowledgement that the request is received, but fulfilling the request can take much longer. On average, I’d estimate that average response times are around 5-6 months, though I’ve seen cases of a year or longer. When I am organized, I try to set reminders every couple of months to check in and ensure the request didn’t get stuck somewhere. In some cases, the government supplies data in response to a request, and the data are incomplete or incorrect, and a follow-up is required to get the correct records. In other cases, the FOIA requests leads to back-and-forth exchanges to clarify what is available, needed, or feasible.

Question: Any other strategy or tips?

Sarah: One thing I always try to remember when filing FOIAs is there is a real human being at the end of the process. You want to make their job as easy and quick as possible, so I try to be as specific as I can about what I am requesting. For example, an inspection might have a public number associated with it or a date it was conducted. I also will often reach out to the FOIA office before I file my request. I’ve found the people who work in those can often be helpful in giving guidance about how to write your FOIA in a way where it’s most likely to succeed, since they know the ins and outs of the process.

Joe: I try to be conservative and request only what is needed once a project is somewhat far along, since any request takes time and resources for the researcher and government. There are times I can’t determine what variables a database contains so I request all available variables, including but not restricted to the specific fields that are especially important. Sometimes, a federal agency has explained that state or local offices had the required data and separate requests were required to each local agency/office.

Some jurisdictions charge fees for staff time or paperwork. In some cases, these are waived for educational and public interest purposes. In other cases, they end up charging fees of a few hundred dollars. I don’t think I’ve had fees for a request ever exceed $1000. I’ve also had responses saying that a request was too costly for any level of fee and the government could only supply a sample of records, e.g., 50 observations out of thousands available, and then have navigated from there.