The FBI’s Next Set of Crime Data Is Going to Be a Big Mess

Crime—particularly the spike in murder that the country has seen over the past two years—is thought to be weighing heavily on the minds of voters, with November’s midterm elections no longer all that far away. When the FBI reports national crime estimates for 2021 this fall, they have the potential to be a Big Deal, politically speaking. If murders are still way up relative to a few years ago, as most experts anticipate, the Republicans are sure to blame Democrats both as the party in power.

But whatever numbers the FBI releases, they are unlikely to provide the clarity into crime trends that the public deserves. And that’s because of a recent, and significant, change to how the FBI collects crime data. As a result, saying with any confidence whether crime is up or down may be virtually impossible.

On paper, the new National Incident-Based Reporting System (NIBRS) is clearly superior to the Uniform Crime Report Summary Reporting System, which it replaces. NIBRS enables agencies to collect data on multiple offenses within the same incident (such as a robbery that leads to a murder), provides greater insight into a wider array of a jurisdiction’s crimes compared with the old system, and allows for novel breakdowns of crime victimization by age, sex, and race. In the long run, NIBRS will provide “much more detail and context around crimes,” per the FBI, than the legacy system did, including the possibility of tracking nonfatal-shooting victims for the first time.

But there are major issues. To begin with, of the nation’s nearly 19,000 law-enforcement agencies, more than 7,000 are not yet reporting data to NIBRS. Though they have had years to prepare for this switch—the FBI announced the change in 2015 and gave out more than $120 million to help agencies make the transition—only 62 percent, covering just 65 percent of the U.S. population, are reporting to NIBRS for 2021, according to the FBI.

NIBRS is voluntary at the federal level, so the FBI cannot compel reporting. All agencies had to either switch to NIBRS from the legacy system or report no crime data to the FBI. Marcus Berzofsky, a senior research statistician at RTI International who is working on the estimation problem of the NIBRS transition, anticipates that 32 of the 72 largest agencies will not report a full year’s worth of NIBRS data for 2021. Eight agencies covering 1 million or more people each will not report NIBRS data, including both the NYPD and LAPD.

Under the legacy system, the FBI based its estimates on reporting from agencies that represented more than 95 percent of the nation’s population each year. These extremely high coverage rates made “the amount of uncertainty negligible,” Berzofsky said in a January 2022 webinar.

Some switching costs are to be expected. Transitioning to NIBRS is a complex technical process that can be expensive for agencies and take up to two years to complete. Agencies require a fair amount of technical assistance, financial support, and political will to move away from the legacy system they may have used for upwards of 90 years.

The result is that national crime estimates in the next few years will carry more uncertainty than ever before. This is especially true of the 2021 national crime data. The FBI says that for these data, it will not be “producing a percent change/difference and comparing estimates to past years.” NIBRS estimates for 2021 will be accompanied by confidence intervals, which will attempt to express the level of uncertainty in the reporting, though how large the confidence intervals will be is not yet clear.

Even worse, some states will have no crime estimates for 2021 at all, and perhaps even beyond. The FBI said in an email that state estimates “will not be published if there is less than 80 percent population coverage”—in other words, if the law-enforcement agencies that do report their data to NIBRS represent less than 80 percent of a state’s population,  the FBI won’t estimate that state’s overall crime figures. The FBI established this threshold to prevent any state from submitting data that is so limited that it skews the overall perception of crime in that state. But the unintended consequence is that too many states aren’t ultimately represented in the nationwide data.

This will result in substantially less insight into national crime trends at a crucial time. Murder rose at a historic rate in 2020, and the available evidence from big cities suggests another—albeit smaller—rise in 2021. The NIBRS transition will deprive policy makers, academics, law enforcement, and community members of certainty about local, statewide, and national trends, and about the effectiveness of violence-reduction efforts. NIBRS may eventually improve researchers’ ability to study local and national crime trends, but those capabilities are unlikely to make much difference in understanding and reversing the current murder spike.

People are used to certainty in crime-data reporting, and most won’t understand, come November, how little certainty is available to them on our recent crime numbers. But unfortunately, not much can be done to “fix” NIBRS aside from completing the transition so that new reporting rates reach the legacy system’s level of coverage. But that may be years off, and people will be voting long before then.