This week, lawyers in the UK are competing against AI software. Asked to predict the outcomes of the same legal problems, the two sides are competing for who (or what) can achieve the highest accuracy. In the end, it is the public who wins as awareness of legal AI is increased.

From now until October 27th, 100+ lawyers are studying the same legal problems as the young start-up CaseCrunch’s predictive AI software in a real-time competition to see who can determine the results more accurately. We will announce the winner later this week, but in the meantime, here’s why law’s man vs machine contest was developed and what the organizers hope to achieve.

The competition ended moments ago and the results are in: AI beat the lawyer-humans, with an 86% success rate vs 62% for lawyers. Check out @Case_Crunch for more information on the competition’s outcome.

An interview with CaseCrunch Managing Director Ludwig Ball and Marketing Director Rebecca Agliolo.

1. Who came up with the idea to have a “Watson-like” competition?

We realized that the best way to showcase the utility of our technology was to host a transparent competition against lawyers.

How complicated was it to organize, promote, etc?

The Lawyer Challenge is as complicated as any event, and has several moving parts: participants, venue, sponsorship, etc. However, CaseCrunch has been fortunate to have established a network of legal, media, and corporate contacts over the past year, who have assisted in organizing and promoting the event.

3. Who is participating?

We have over 100 confirmed participants, ranging from barristers to Magic Circle partners. Various law firms have entered teams of participants. Participants include lawyers at associate and partner level from Eversheds, Pinsent Masons, A&O, Clyde & Co, DLA Piper, BLP Law, DAC Beachcroft and many more.

4. Can you briefly describe how it works?

Lawyers login to a website and are presented with problems. They can use any resources (including ROSS) to make predictions. Our system will also make predictions on the entire dataset. Scores will be verified by our technical judge Ian Dodd, Director of Premonition UK.

5. What do you think the biggest misconceptions are about AI and legaltech?

There are tons. The biggest one is “replacing” the legal profession. That’s ridiculous. A lawyer’s job has thousands of individual tasks. Some of them we can automate. Some of them machines can even do better. But no way machines could replace all these different things lawyers do so quickly. In the future — who knows?

6. What do you hope people will learn by participating in the competition? And what do you think the public will learn?

We hope people will see that AI has a real contribution to law. This is true from both the business perspective where it saves time and money, but also from the academic perspective, where AI can tell us new interesting things about legal knowledge. We hope that participants and the public see that in certain niches the use case for AI is very strong.

7. Why are people so fascinated with the Man vs Machine concept?

Probably pure species-ism. Humans are very afraid of creating a better species. So the nerves get very excited whenever a machine takes on humans.

8. And finally, what do you think the outcome will be? Who will be the “winner?”

It’s 50/50 at this point. But it’s not about winning. It's an experiment that makes a contribution to legal AI in the long run.

_____________________________________________________________

Perhaps Fredrik Tunvall, a senior client engagement leader at IBM Watson said it best in Inverse, “As IBM has shown with Watson, the real work begins after the match, extending innovative technologies to tackle societal challenges like improving health care, education and the environment. We believe that when machines win, humans win.”

‍