Situation

Our client represented a plaintiff group in a class action regarding against a large corporate entity, filed in the Supreme Court of New South Wales. Many of the stakeholders in the plaintiff group were sophisticated and as such there was a large amount of electronic data that our client needed to review to prepare their client’s case, and otherwise comply with their disclosure requirements. The collection process uncovered over 120,000 potentially relevant electronic records for review.

Solution

Our class action specialists worked with the legal team to set up a continuous active learning workflow to help speed up the review. An initial training round was completed where the legal team trained the AI model on what they considered to be a relevant document. The team then began their review with the AI model serving documents to the reviewers that it thought they would consider relevant. The model was serving relevant documents to the review team 85% of the time.

Our team was also able to use various reports to improve review accuracy flagging documents for a second review where the model had identified it as highly relevant or highly irrelevant and the reviewer had made the opposite decision. For most anomalies the issue was not with the model but with the initial human review. Eventually the review got to a point where there were very low levels of relevant documents being put forward and the review was ended. At this point further elusion testing was used to identify any final anomalies for review by the legal team. The elusion test confirmed that there was an extremely high level of accuracy and a very low likelihood of any relevant documents remaining in the unreviewed documents.

Further, as the model had been trained and proven as having a high level of accuracy, it was able to be quickly and effectively used to identify relevant documents in subsequent tranches of data provided by the end-client. This meant that the team could skip straight to a detailed review on the relevant set, conducting high value tasks like analysis and privilege redactions.

Impact

Our work in designing, implementing, and overseeing the continuous active learning workflow allowed our client to:

• Reduced the risk of human error and inconsistencies when reviewing for relevance

• Reduced the number of documents the legal team had to review at first pass by close to 60%

• Increased the review efficiency of subsequent tranches of documents, in some cases reducing review requirements by over 90%

• Use the elusion testing process to track the accuracy of the active learning workflow and support its use as something that achieves an accurate and more cost-effective review outcome