In 2012, the city of New Orleans Police Department was placed under one of the most sweeping consent decrees ever issued by the U.S. Department of Justice (DOJ). In policing, these binding orders are issued in the wake of an investigation finding that a police department has engaged in systemic misconduct or bias. In a document that spanned more than 100 pages, the DOJ outlined a series of reforms that NOPD would need to implement, and regularly report on, to come into compliance with federal standards.
Like other cities under consent decree, NOPD struggled to meet the enormous administrative burden required to track the department’s progress. It responded by developing Management Analytics for Excellence (MAX), a data management system that collects traditional crime statistics alongside internal performance metrics to give the department an unprecedented view of its operations. New Orleans has since been hailed as a model for how data can be used to accelerate police reform. FiveThirtyEight called the city the “queen of open police data.”
MAX has three major components. The first is a dashboard that automatically pulls data from different sources and displays them in user-friendly charts and tables. The second is a series of maps that allow for place-based analysis of, for example, criminal activity or community policing projects. The third component is a set of detailed reports that allow for more granular analysis of specific issues, such as why call response times are high at a certain time of day or which detectives have outstanding police reports.
Most data management systems — most notably the CompStat system used by many police departments — focus on the collection of crime statistics. What makes MAX remarkable is the integration of a wide range of internal performance measurements, such as use of force, citizen complaints against officers, and consent decree compliance, which are visible to the public in real time — an exceptional level of transparency for a police department.
The efforts are paying off. NOPD was recognized by the DOJ Consent Decree Monitors to have “proven itself to be a respected, forward-thinking, reform-minded police agency to which other agencies now routinely come for guidance.”
The architect behind MAX is NOPD Director of Analytics Benjamin Horwitz, who manages the system in close collaboration with Deputy Chief of Compliance Danny Murphy, who oversees the department’s implementation of consent decree reforms. Both work closely with FUSE executive fellow E. Aminata Brown, who is helping NOPD further modernize and digitize its systems and procedures. Ultimately, the department is seeking to minimize the administrative burden on officers so that they can better serve the community.
FUSE spoke with Director Horwitz and Chief Murphy about MAX, what impact it has had on reform efforts, and what other cities might learn from it.
Q. A lot of police departments collect crime data. Why was it also important for MAX to include performance metrics like use of force and response times?
Horwitz: I think it’s the general philosophy that the police department is responsible for more than just crime. The community expects us to answer their call. They expect us to execute procedural justice, expect us to use force responsibly, to interact with the public professionally, and to solve cases.
Murphy: Without the data-driven elements of MAX, we were unable to actively manage many facets of the organization. We relied on anecdotes in many areas. Now, we can pair anecdotes with hard data that shows whether we’re performing well in various areas. The idea is that if we can manage the organization as a whole in a better and more effective way, we’ll be more effective in dealing with crime, or response time, or consent decree compliance. Over and over again, we’ve seen that by taking a data-driven approach to these different areas, we’ve been able to improve the quality and effectiveness of our work dramatically.
Q. What kinds of questions did MAX enable you to ask that you couldn’t have asked before?
Horwitz: We might have talked about response times before, [but we didn’t have the hard data to support those conversations.] Now we can specifically say, ‘Why did you have a high response time around 2:00 pm for this past week?’ We can ask a day watch sergeant or lieutenant, ‘Tell me why you still have three or four dozen unapproved reports, and what are you going to do to make sure next time that list is down to a single digit?’
Q. How was this data managed before MAX?
Horwitz: Prior to MAX, it took a lot more administrative work to compile and prepare the data, and the detailed reports weren’t available. Supervisors had to spend a considerable amount of time looking for lists of unapproved reports or forms that were missing without having an easy lookup function. In the case of detectives, these types of reporting capabilities did not exist before. Either detectives did a lot of work by hand or didn’t do it at all, because they lacked some tools to help them do these extra facets of the job. As it pertains to topics outside of crime, these were not managed from a data-driven perspective. We relied on anecdotes.
Q. Were there any models that you could look to or was this something that you had to conceptualize from scratch?
Horwitz: It was a little bit of both. What we found was that the most successful departments who got out of consent decree found ways to rigorously measure consent decree compliance. And then we also saw that some of the departments that were doing a successful job of community policing, for example, had found ways to try to measure community policing projects.
Murphy: We certainly had to innovate in some areas where the best practice really hadn’t been defined. On community policing, something I don’t think we had really seen in some of the other systems was gathering feedback from the individual districts and the individuals living in those districts about how they want their policing to be focused in that area. It’s definitely a mix of taking the best practices from today and innovating where we needed to elevate our performance for the community.
Q. Why is it important that all of this information is visible to the public?
Horwitz: We’re really proud to be part of the White House’s Police Data Initiative, and to have one of the most open data sets for a police department around the country. It’s all part of this idea that we are an open and transparent department, and that this is critical to increasing policing legitimacy and gaining public trust.
Murphy: A critical part of our work will be sustaining those reforms long-term, after the consent decree. Right now, there are multiple consent decree compliance tabs within MAX that show anyone how the different districts are doing based on internal audits. Providing that information to the public after the consent decree process closes is going to be critical to maintaining the high level of accountability that we’re achieving right now.
Q. What did it take to implement MAX?
Horwitz: I became director of analytics in January of 2016 and one of my charges was to go out and reinvigorate our CompStat process. At the same time, we were trying to wrap up a project to build an early intervention system, which necessitated improving the data quality of our source systems and integrating our source systems into a single reporting framework that would enable analytics. We’ve got all these things starting to get going around winter of 2016. The project formally kicked off in March and we created a working group with a handful of district commanders in it. We met regularly to go through what was in CompStat. What was there currently? What is discussed in district meetings? What are the important questions they asked? What are the important questions they get asked? And we started from there.
Starting about May or June, we worked on building the data prototype for approval from the Superintendent, which we received at the end of June. And then we had a product built for internal release in August of 2016. That gave us six weeks to review it and start getting used to those metrics as we were shutting down our current CompStat. Even at that stage, it was still an iterative process. Right up until launch [in October], we received feedback, made improvements, and continued to make fixes. That practice continues today and something that we consider is vital to making MAX a success.
Q. Were there any challenges that you had to overcome over the course of this process?
Horwitz: Sure. As people see what is capable of being managed or reported on and all the different benefits therein, they start asking for more support. There were a lot of one-time projects that came up. ‘Oh, you know what would be great?’ Or, ‘Could we get this developed? Can you help make this easier?’ Sometimes that works really well, because it’s a one-off ask for a unit that is unique. But sometimes we might have eight different districts who might want to do things eight different ways. On one hand, MAX enables standardization. On the other hand, there are unique needs out there. It’s a fine line to walk.
Q. How do you make those decisions about whether or not to incorporate a certain metric or data set into MAX? Are there any guidelines or criteria that you use?
Murphy: If it aligns with the needs of many levels of the organization, I think that’s a key part of including it in MAX overall. Because, really, MAX is our departmental management tool and we don’t want to overly burden it with individual statistics for discrete units. If it’s overly broad, we lose the focus of MAX. The focus of MAX is the panorama, basically, of departmental management. Throwing too many one-off things in there would distract from it.
Q. What impact has MAX had internally on the department?
Horwitz: A very concrete, specific example is that, in October of 2015, we got to emergency calls within 20 minutes, on average. Now, our average response time is between 10 and 12 minutes. A huge reduction accomplished by the tools provided into MAX. Here’s another example; one of the things that we needed to do in our consent decree was change how we handled calls with people experiencing crisis. For those calls, our new policy requires submitting a form. In order to be compliant with the consent decree, we needed to make sure that all these forms were submitted and compliant. When the policy came out, we were submitting forms about 20 percent of the time, because it was brand new and there is usually a user adoption phase. The only way we could manage it was telling them, “Get your forms in” and the response was “What forms? Which forms? What calls? Which calls are missing forms?”
Then a month later, we created a simple report for officers and supervisors that show, “These are the forms that are missing.” Two months later, we’re at 97 percent consistently. This is an example of how just one little piece of reporting can help move us toward compliance.
Murphy: Being able to look, for example, at body worn camera footage, analyze the results, feed that into MAX, and really hold leadership accountable for improving the results of their performance based on these audits has been incredibly transformative for the department. MAX has helped us become smarter on crime. We’ve seen key drops in armed robberies and other indicators. We’ve gotten much better at community policing. Consent decree compliance and reforms have really taken off as a result of the management through MAX. Just overall, the management of everything we’re doing has improved dramatically.
Q. What is the impact on the community of having a tool like this?
Horwitz: I think it’s just one of the things that contributes to people perceiving that we’re more transparent; that we are managing the department from a data-driven perspective on all facets of our operations.
Murphy: One difficult thing about right now is that we’re doing so many reforms at once, it’s hard to untangle what is producing what result sometimes. But we’ve seen really dramatic increases in public trust. People have much greater faith in the leadership of the department, and the direction it’s headed than in recent memory.
Q. What advice would you give to another city that wanted to take this same approach? What are the teachable lessons that you might share with them?
Horwitz: I think one of the things that’s really important is that you have to have leadership buy-in — not just the police chief and the superintendent, but through to your top-level commanders. Who are the people who are responsible for leading your districts or your divisions like homicide? Once you have the buy-in that this is what matters, then you’ll have the ability to start measuring those things and using them to create change.
Murphy: MAX is really founded on making the data as useable as possible. Having open lines of communication with the people using the system has been really critical. Presenting the data in a useable, visually striking way brought more people into the conversation.
Horwitz: And the last thing I would add is, invest in data infrastructure. Sometimes there are easy entry points that get you going and then you can refine and expand, make it better. But start investing in that data infrastructure now and start trying to build up the type of data reporting capacity to support it.
Murphy: I would say to that point, investing in IT infrastructure is a force-multiplier. A lot of different police departments are struggling with staffing adequately right now. We used to have multiple people manually compiling stats for our CompStat meetings. Now, it updates automatically based on the MAX program. It can really be a force multiplier in many different ways.
Rikha Sharma Rani is a Bay-area based writer and journalist. She is the founder and principal writer at Rickshaw Writers. Her work has been featured in The New York Times, USA Today, Politico Magazine, The Atlantic, HuffPost, Oakland Magazine, and more.
[Photo credit: NOPD]