1. BC JUSTICE RESEARCH FRAMEWORK working group
A “research framework” is a structure or system that would enhance production, quality, organization and sharing of justice data. The need for a research framework was first explored at a Colloquium of senior BC justice representatives sponsored by ACE at the University of Victoria in May of 2016. That meeting was focused on the state of data, metrics and research in the BC justice system. As the Colloquium Report states, "It was agreed that there is a strong need for more evaluation, performance measurement and empirical research in the BC justice system, and that the system’s current capacity for such research is extremely limited. The root problems include a paucity of data, existing data that is inaccessible or unusable, uncoordinated data collecting across diverse sectors, lack of mechanisms for sharing data, institutional resistance to collecting data and a culture that lacks experience with and understanding of empirical methods."
One of ACE’s objectives in convening the Colloquium had been to determine what role it might play in advancing provincial justice research. The first task emerging for ACE out of the Colloquium was to lead an informal consultation with Colloquium participants to define terms of reference and to establish a provincial, multi-sector Research Framework Working Group (“RFWG”) to pursue the design and implementation of a justice research framework. This work ultimately resulted in the convening of the first meeting of the RFWG in February of 2018, where the following mandate was adopted:
Articulate an overarching metrics vision for the BC civil justice system,
Articulate an overarching strategy to enhance the quality and accessibility of justice metrics,
Formulate a two year plan to work toward an integrated and coordinated system-wide data infrastructure sufficient to meet the objectives set out above, and
Begin to implement of the vision, strategy and plan.
ACE will play an ongoing role in advising and supporting the RFWG in this work.
2. Research alternative metrics Framework models
This project will explore work done to enhance justice data and empirical measurement in other jurisdictions. This will be done to ensure that BC is fully aware of all current or previous attempts in any other jurisdiction to enhance justice data quality, encourage data sharing and expand the use of empirical research and justice program evaluation. The aim is to capitalize on any existing work already completed on the justice metrics problem.
3. Articulate civil justice system objectives
An Australian paper entitled “Building an Evidence Base for the Civil Justice System” states, "The first question that needs to be answered in order to build a better data base for the civil justice system is ‘what information do we want?’ Information that demonstrates that one service, one program or one organisation is performing efficiently and effectively is useful. However, it will not help to make the case for the essential importance of a well-functioning civil justice system to the welfare of the Australian community. To do that we need to be clear about what Australians are looking for in the civil justice system (objectives) and what it is delivering to them whether as individuals, small businesses, volunteer organisations or large trading corporations (achievements). As a first step we need to identify appropriate objectives are for the civil justice system."
Adopting this rationale, this project will unfold in two stages. First, ACE will produce a draft report articulating possible over-arching strategic justice system objectives as well as ancillary key-objectives and sub-objectives. These objectives would be gleaned from the many Canadian access to justice reports and related academic articles published over the last twenty years. The project will identify goals and measures that will assist in responding to key questions such as “what do we want to know?” and “what do we measure to know if we are being effective?”
The second stage of this project will involve consultation with representatives from the diverse justice sectors in BC on the question of objectives and values. Feedback from the consultations will inform the final version of a report. This report will ultimately serve as an ongoing reference and guide to the development of a metrics framework and to any justice research ultimately undertaken.
4. Assess the merits of a BC justice data scan
This project will produce a report exploring a) the advisability, b) the feasibility and, if appropriate, c) options for undertaking a BC justice data scan. A data scan would be a comprehensive inventory and catalogue of the data currently collected by various justice entities throughout the province. A data scan could, potentially involve:
identifying existing data – what data exists; where does it exist; what is the quality of the data; what is the potential utility of the data and can it be shared?
Identifying needed data - what data is needed; what gaps are there in existing data? This raises the question of what access questions we wish to be able to answer.
Identifying potential data sources – what additional data could be collected by government and courts, and what contribution could NGOs and community service providers make in generating useful data? What relevant or useful data might be held outside the justice system by entities such as the Ministry of Health?
Deciding how justice data is best stored – how should data be collected, accessed and shared? Should / could justice data or access to data be aggregated in a single location or through a single path? Should BC work toward a “data observatory” model to warehouse and make accessible data from across the justice system?
These questions, in turn, raise a number of issues respecting appropriate access, data governance and stewardship, privacy, anonymization, physical security, and the practical matter of data transfer and maintenance where required. The report will touch on all of these matters and, as such, will help BC to assess the utility and the feasibility of undertaking this potentially large and challenging task.
5. justice research data repository
This project will focus on the questions of what empirical access to justice research already exists and how that research could best be collected and shared. The scope of this inquiry is necessarily national, if not international. We know that there have been quite a few access studies and program evaluations completed in Canada over the last 20 years, and that they are not particularly easy to find. Some research has been collected by the Canadian Forum on Civil Justice through its online “Clearinghouse” and ACE may work with the Forum to enhance that collection. It makes sense to collect and utilize existing research and, in any event, what is learned about locating, archiving and organizing research through this study will also be useful as the body of justice research in BC begins to grow under the framework.
6. Identify BC research questions and priorities
This important report will focus on the question “what do we want to know?” That is, what research questions do we ultimately wish to be able to answer and what are the priorities as between the questions? The information for this report will come from two sources: consultation with BC justice stakeholders and research exploring what academics and policy analysts in other jurisdictions have said about this question. This project will involve consultation with diverse stakeholders.
7. attrition study follow-up
On September 15, 2015 the Canadian Forum on Civil Justice (CFCJ) published “Civil Non-Family Cases Filed in the Supreme Court of BC – Research Results and Lessons Learned”. That study was sponsored jointly by CFCJ and ACE, and was conducted by Focus Consultants of Victoria, B.C. in 2014 and 2015 in the Supreme Court of British Columbia. It was premised on the fact that, while we know that only approximately 2% of superior court cases go to trial, we know very little about what happens to the other 98% of filed cases. The study confronted multiple problems in trying to collect data about that 98% from the 500 BCSC motor vehicle accident and general civil (non-family) files captured by the review. This follow-up study will:
consider what changes would need to be made to data collection processes in the BC Supreme Court in order to be able to answer the original questions and meet the original research objectives, and
provide a high level plan for making such changes, including identifying any legal, financial or administrative challenges that would be encountered in making such changes to the system.
8. National Justice Metrics coordination
The Action Committee Metrics Working Group’s (the Working Group’s) mandate is to work toward inter-jurisdictional cooperation and coordination in the development of justice metrics. National coordination is in service of the common goal of making a larger amount of useful data more readily available to empirical researchers in order to enhance the capacity of justice systems to empirically measure program performance and evaluate user outcomes.
The purposes of national coordination include:
to move toward enhanced data and performance measurement as efficiently and expeditiously as possible by pooling scarce resources and limiting duplication of work; and
to harmonize data definitions, data points, software programs and so on, in order to,
maximize the ease and viability of conducting cross-jurisdictional research; and
better ensure that access research conducted in one jurisdiction will ultimately be useful and relevant in another.
Members of the Working Group are:
Saskatchewan through CREATE Justice
Brea Lowenberger, Director, CREATE Justice and Access to Justice Coordinator, College of Law, U of S
Janelle Anderson, Innovation and Legislation Division, Ministry of Justice, Saskatchewan
Quebec through the Centre de Recherche en Droit Public at l’Université de Montréal
Professor Pierre Noreau, Faculté de droit de l’Université de Montréal
Professor Moktar Lamari, Ecole nationale d'administration publique, Quebec, Quebec
Ontario through the Canadian Forum for Civil Justice
Professor Trevor Farrow, Academic Director, Winkler Institute for Dispute Resolution, Chair, Canadian Forum on Civil Justice
British Columbia through the UVic Access to Justice Center for Excellence
Professor M. Jerry McHale, QC, Director, UVic Access to Justice Centre for Excellence
This Working Group works under the sponsorship of the Action Committee on Access to Justice in Civil and Family Matters in the context of the Roadmap for Change recommendations 8.1 and 8.2:
8.1 Promote a national access to justice research and innovation agenda that is both aspirational and practical.
8.2 Develop metrics of success and systems of evaluation reliable and meaningful metrics and benchmarks need to be established across all levels of the system in order to evaluate the effects of reform measures.
Membership remains open to any jurisdiction wishing to participate. The Working Group operates on the basis that its activities are fully transparent to all jurisdictions. All useful information acquired and all work product generated will be shared.
9. Out-of-pocket: calculating the real costs of family court
This research will develop a methodology for quantifying the cost of family law processes to litigants beyond court and legal fees. ACE is working with the Law Foundation of Ontario and researchers Sarah McCoubrey and Meredith Brown on this project. This project is now live.