The Current Justice Metrics Landscape
By: M. Jerry McHale, Q.C.
A Roadmap for Change - the final report of the Action Committee on Access to Justice in Civil and Family Matters (“AC”) - calls for the nation-wide development of justice system metrics of success and systems of evaluation:
Reliable and meaningful metrics and benchmarks need to be established across all levels of the system in order to evaluate the effects of reform measures. We need better information in the context of increasing demand, increasing costs and stretched fiscal realities.
The metrics deficit has been described as follows:
The problem revolves around the lack of reliable justice data, the lack of empirical justice research, inconsistent justice metrics and the inability of justice systems to measure the performance or effectiveness of their programs and processes … This lack of data and research capacity impairs policy development, handicaps effective planning and weakens claims for justice funding. It is, generally, a serious impediment to access to justice reform.
There has been considerable activity in the last year or so in response to this call. There is currently national recognition of the nature and significance of the metrics deficit and a wish for some form of national coordination in working, as a priority, towards a solution. This blog identifies a number of Canadian committees and other bodies that are presently working to enhance justice system metrics and build measurement capacity.
The following groups are taking steps to respond to the need for more and better justice metrics:
· The Access to Justice BC, Measurement Working Group (A2JBC): has developed a measurement framework to support a shared approach to measuring, monitoring and evaluating improvements in access to justice in BC. The framework is inspired by the “Triple Aim approach” originally developed in the health sector. The framework identifies a wide range of elements for potential use in measuring justice outcomes. It is a flexible framework that can be used by justice system stakeholders in BC to coordinate and align their various monitoring and evaluation efforts, and to learn from each other’s experience with access to justice innovations. A copy of this groups “Access to Justice Measurement Framework” paper will soon be available online.
· University of Victoria, Access to Justice Centre for Excellence (ACE): is working on the establishment of a “BC Research Framework Working Group”. Representatives from the Ministry of Justice, UVic, the three courts, and other justice bodies will use this group as a forum to work collaboratively to promote objective performance measurement and empirical research, starting with the development of justice metrics and enhanced data quality. Tasks will likely include:
o a data inventory: complete an assessment of existing data resources, data quality and data gaps;
o data definitions: develop common definitions and standardized performance metrics to facilitate coordination and sharing of research;
o data observatory: develop the mechanisms, technical capacity and protocols needed to make data from diverse justice sectors within BC accessible to researchers, practitioners and others examining questions of access to justice;
o coordination: coordinate research inquiries to ensure that optimal research opportunities are identified; to identify potential gaps, overlaps or duplication of effort; to allow interested parties to maximize scarce resources and expertise; to facilitate partnerships and alliances with other entities in BC holding justice data, and to disseminate research.
· The Action Committee on Access to Justice in Civil and Family Matters (AC): as one of the groups that called for the nation-wide development of metrics of success and systems of evaluation in its Roadmap for Change report, the AC has recently followed up by consulting with its national membership on the question of national coordination of efforts to enhance justice system metrics. At a meeting of AC members held in Vancouver in March 2017 there was a consensus that the AC should work to help coordinate and facilitate metrics development and information exchange between jurisdictions. The purposes of such coordination would include:
odefine what data is needed, for what purpose,
ouse common event definitions indicators and coordinated goals,
oensure some measure of quality and consistency in the data between jurisdictions,
opublish and share data,
oobtain a clearer profile of system users,
otrack cases mediated or settled out of the system,
oobtain data on user satisfaction (outcomes).
No final decision has yet been made about how the AC will respond to this goal, but there was much support for the idea that the Action Committee move to create a metrics sub-committee with representation from all provinces and territories.
· CREATE Justice was launched in March of 2017 out of the University of Saskatchewan, College of Law. It is a centre for research, evaluation, and action on the topic of access to justice, specifically in the areas of access to legal services, dispute resolution, and systemic justice. Its primary objective is to address key gaps in data and to promote action respecting access to justice research in Canada. Its research will be oriented towards transforming legal and justice services and the removal of systemic barriers to justice. While much of existing research focuses on “access” to legal services and institutions, CREATE Justice’s focus will be more on substantive, community-engaged, social justice research.
· Federal Provincial Territorial Assistant Deputy Ministers’ Metric’s Committee: This Committee was convened in 2015 and is co-chaired by Andrea Strom of the Ontario Ministry of Attorney General and an ADM from Alberta (yet to be named), with a Working Group of representatives from Ontario, Alberta, Nova Scotia and British Columbia. This Committee is exploring the coordination of government metrics and is working to assemble a draft table of justice measures, based on justice objectives articulated in the various AC reports.
· The Accès au droit à la justice (ADAJ): This project is under the leadership of Pierre Noreau, professor at the Faculty of Law at the Université de Montréal, and a researcher at the Center de recherche en droit public. ADAJ brings together a consortium of 42 researchers who have identified some 20 research topics relating to access to law and justice. A preliminary report from this group examines judicial metrics used throughout Western democracies to measure justice inputs, outputs and outcomes. The goal is to develop a set of metrics reflecting Canadian values and adapted to the Canadian judicial system. These researchers identified gaps and inconsistencies in data as well as weaknesses in quality and reliability. This led to recommendations for a second phase of their study which would be to develop, for Canada and each of the provinces, metrics and data creation and collection practices for benchmarking performance of the respective judicial systems. The report also recommends the creation of an expert committee that would make decisions and advise in respect of the standardization of data across Canada.
· The Western Canadian Data Analytics Task Group: was convened in 2016 and is comprised of representatives of the 4 western provinces. These jurisdictions are working to share and inventory information related to data analytics in BC, Alberta, Saskatchewan and Manitoba. It is currently assembling an inventory of current, publicly reported justice performance measures in these jurisdictions.
· The Canadian Centre for Justice Statistics (CCJS): This Statistics Canada body collects criminal and civil justice data from all jurisdictions and makes available datasets as well as analysis and reports on major trends in the system.
Implications and next steps
These eight Canadian justice metrics initiatives are not currently coordinated. As noted above, the importance of national coordination of this kind of work was reflected in a national survey of the jurisdictions completed this spring, and endorsed by the membership of the national Action Committee on Access to Justice in Civil and Family Matters at its annual meeting in Vancouver in March of this year.
The next questions to be answered include:
- Who will take responsibility for coordinating justice metrics development initiatives?
- Who should be involved?
- What does “coordination” actually mean? What should it look like? What model would make it possible, without becoming unmanageable, to engage with all interested jurisdictions and stakeholders?
- How would this work be resourced?
“Roadmap for Change” (2013) Research and Funding Goal 8.2, online at http://www.cfcj-fcjc.org/sites/default/files/docs/2013/AC_Report_English_Final.pdf p.23
 “The Justice Metrics Problem” Background Paper (2017) Action Committee on Access to Justice in Civil and Family Matters, Vancouver BC, March 23 & 24, 2017, online at http://www.uvicace.com/resources/
 See “Summary of Justice Metrics Survey Responses” (2017) Action Committee on Access to Justice in Civil and Family Matters, Vancouver BC, March 23 & 24, 2017, online at http://www.uvicace.com/resources/
 Chaired by Chief Justice R. J. Bauman, “A2J BC” is BC’s response to the national call for action to make family and civil justice more accessible. Members include leaders from all major justice system organizations, and representatives from related sectors such as health and municipalities.
 “The term ‘Triple Aim’ refers to the simultaneous pursuit of: 1. improving the population’s access to justice, 2. improving people’s experience of the justice system when attempting to resolve a legal problem, and 3. ensuring that the costs of providing access to justice are sustainable. It is essentially, at a high level, a cost-benefit approach to thinking about access to justice...” (from Measurement Working Group paper, “Access to Justice Measurement Framework” p.4)
 The report is entitled Performance des systèmes de justice : Qu’est-ce qui compte? See online http://adaj.ca/accueil