I’ve had a twenty-year obsession with translating legal language – specifically, contract language – into data. It’s a problem we started to solve at Exari, but it’s a big and complicated problem, so we didn’t quite finish the swing. Nonetheless, it’s a problem worth solving, and it’s a solvable problem.
The need is beyond question. Business and legal teams know that many of their contracts contain hidden risks, errors, gaps and surprises. They just don’t know which ones and how much damage will be caused if those particular deals blow up. As stressful as this may be, the cost of having experts read every page and track every risk is prohibitively high, so they suck it up and hope for the best. Maybe that scary indemnity will never be triggered. Maybe those uncapped sales contracts never come to light. Maybe that rogue MFN clause never hits your revenue projections. Maybe that missing amendment was totally harmless.
If a machine can find all these risky contract terms quickly and at a fraction of the cost, everyone wins. The lawyers don’t lose sleep waiting for bad contracts to blow up. The business can mitigate and manage the risks that come to light. And the weak contract haircut doesn’t spoil your financials.
Easy to say. Very hard to do.
Many of you have heard about magic contract AI. Some of you may have tried it. Most of it has been disappointing. This does not mean the idea was doomed from the start. It just means it wasn’t fully baked yet. It takes plenty of science and art to get this bread to rise.
One secret to building a contract data machine that works is a team with deep experience, knowledge and skills, spanning the legal, business and technical domains. This is probably the most important ingredient. Many will claim to have this multi-disciplinary depth. Catylex has it. A rare bunch of legal-language-business-data obsessives who are solving complexity so that you don’t have to. You can just eat the results.
So, I’m excited to join Catylex, where the dream of building a fully functional contract data machine is a shared obsession, and where the promise of such a machine is now becoming reality. This might just be the best thing since sliced bread.
I have to admit that “clause tracking” sounds good. Let’s analyze how many deals contain our preferred limitation of liability clause. Let’s track how many times we had to use fallback #2. It feels like these would be really useful things to know. Which is probably why so many contract managers ask for it.
Not Quite a Limitation of Liability Clause…
But as you peer behind the shiny facade of clause tracking, things start to look a little grimy. To explain why, I’m going to use a real life example that I recently plucked from the EDGAR database:
10. Limitation of Liability. IN NO EVENT SHALL EITHER PARTY BE LIABLE TO THE OTHER FOR EXEMPLARY, CONSEQUENTIAL, SPECIAL OR PUNITIVE DAMAGES, ARISING FROM OR RELATING TO THE AGREEMENT, WHETHER IN CONTRACT, TORT (INCLUDING NEGLIGENCE) OR OTHERWISE, EVEN IF SUCH PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES; PROVIDED, HOWEVER, THAT THIS LIMITATION SHALL NOT APPLY TO EACH PARTY’S (A) BREACH OF SECTION 4 (CONFIDENTIALITY); (B) BREACH OF SECTION 7 (OWNERSHIP AND INVENTIONS); (C) INDEMNIFICATION OBLIGATIONS; OR (C) RECKLESSNESS, INTENTIONAL MISCONDUCT OR FRAUD. THE PARTIES FURTHER AGREE THAT THE MAXIMUM LIABILITY UNDER THIS AGREEMENT, REGARDLESS OF THE TYPE OF CLAIM OR NATURE OF DAMAGES, SHALL EXCEED THE TOTAL VALUE OF SERVICES PROVIDED BY DAVA TO AROG HEREUNDER.
The most glaring problem with this clause should be clear to an experienced contract negotiator fairly quickly. The word “NOT” is missing. Rather than saying “SHALL NOT EXCEED” it says “SHALL EXCEED”. And so what was probably intended as an aggregate cap on the supplier’s liability, instead becomes a strange declaration of liability exposure. Clearly, someone made a mistake. Quite a big one.
Which brings me to clause tracking. How do I build a machine that can spot this problem and flag it for remediation?
If my clause unit is the whole numbered paragraph 10 above (which seems reasonable on its face), then a clause tracking tool might use a statistical bag of words method to tell me that (because of the missing 3 letters) this is a 99% match to my preferred limitation of liability clause. But given the difficulty of perfectly matching in the grubby world of scanning errors, I am tempted to ignore 1% errors as immaterial. So this clause probably gets a green light.
This is the “perfect match” conundrum. In the real world of imperfect scans and OCR errors, I am almost never going to get a perfect match, so I treat 99% as OK. But in many cases, including the example above, a 99% match could actually contain a very high risk error.
If the other party produces the first draft (“third party paper”), your chances of clause matching go from bad to worse. Even if their template includes an acceptable version of the clauses you require, there’s a vanishingly small chance they used the same words as your clause library. So even the “matched” clauses will be well below 100% and you’ll have to double-check everything.
To be fair, a clause matching tool might redline the missing words and help a human negotiator to catch any mistakes. But redlining isn’t exactly ground-breaking. All it does it augment a skilled human process. We can unlock far more value by eliminating human tasks rather than augmenting them.
Another (arguably better) approach is to train a machine to understand the meaning of a “clause” and to define your risk playbook semantically, rather than by reference to precise word combinations. In this approach, you might split Section 10 into two semantic concepts. Sentence #1 is an exclusion of indirect loss, which is good. Sentence #2 is (or would be, but for the error) an aggregate cap on liability, which is also good (for the Supplier).
By training a machine with many examples of these two concepts, we can produce an AI service that assigns the correct semantic meaning to any sentence/clause processed, regardless of the precise words used. This offers far more value than precise word matching because it works not just for deals on our paper, but also for deals on third party paper.
We still need to solve the problem of training our machine to understand mistakes. Sentence #2 looks extremely similar to a liability cap. But it isn’t. So I need a “typo” training class to teach my AI that sloppy drafting like this should not be confused with the real thing.
Don’t get me wrong. There is some value to “perfect match” clause tracking if you are working with high fidelity documents that don’t suffer from OCR noise. But semantic machine learning analysis has far greater potential to automate contract review, especially if a large chunk of your deals are on third party paper.
Historically this question was relatively simple to answer. “Yes, that sounds like a wonderful idea. If you don’t want to be a doctor, then lawyer it is. Study hard kid. You can do it.” However, with lawyer numbers trending downwards, one wonders whether the correct answer is more like this: “Absolutely not. Lawyers are AI roadkill waiting to happen. You should be a data scientist. Go do your math homework.”
Not all lawyers are doomed, of course. There is plenty of work left for a profession that favors quick wits and creativity, especially at the high end. But looking at the data, it appears we have moved past “peak lawyer” and are now entering a downward trend. As the BLS graph shows, the number of “production” lawyers peaked around 2007, and never really recovered.
In his November 2019 paper, BLS economist Joseph Valentine shows that, since the “great recession”, both lawyer numbers and law firm price inflation have gone down and stayed down (see Producer prices in the legal services industry after the Great Recession). Indeed, 2018 lawyer numbers were back down at 2003 levels.
With the impact of COVID-19, it seems reasonable to predict that things are getting worse, not better. And as Legal AI continues to mature and take hold, it seems likely that lawyer jobs will continue to fall. It is possible that we’ll enter another AI winter, and that the jobs outlook will turn around. But based on my experience, Legal AI is starting to deliver results and value, if applied in a thoughtful and targeted way. Some AI may be over-hyped, but winter isn’t coming any time soon.
The average corporate law department spends just under $30 million annually on external legal, or 61% of the total legal budget. With that kind of money you could afford roughly 2 million hamsters. That might sound like a lot, but don’t forget that Elon Musk reckons you’d need 50 billion hamsters to fill Tesla’s Gigafactory. At that rate it would be 25,000 years before you need to find a new hamster habitat. We could be living on Mars by then.
First, spend on technology and alternative legal service providers (ALSPs) is relatively modest. Compared to an average $528K technology outlay, legal departments spend about twenty times (20x) on their in-house team ($19.2M), and about thirty times (30x) on external legal fees ($29.8M). Average ALSP spend is under $1M.
Second, although most industries are still spending more externally than internally, that is no longer true for tech companies. The technology sector appears to have made the switch to spending the majority of its legal budget internally.
Third, life sciences legal departments (including biotech and pharma) appear to be super-sized compared to most other industries. They have the largest headcount by far (191 on average) and their legal operations teams are somewhere between double and triple the size of any other industry. Not sure why this is, although there could be a COVID factor creeping in?
The report also reaffirms that companies spend on average just over 1% of revenue on legal, although small companies spend proportionately more (over 3%) than large companies (under 0.4%).
The CLOC survey is based on 146 companies, including 27 Fortune 500 companies, across 17 countries. The hamster price is based on the $15.99 price I paid for our dwarf winter white hamster called Snowball. I discounted this back to $15 a hamster (a modest volume discount).
COVID-19, as we all know, has been rather disruptive in 2020. First, the pandemic itself is creating havoc as a global health crisis. Second, government-ordered lockdowns, shutdowns and travel bans adds another layer of economic, political and social pain. Nobody yet knows just how much economic loss and damage the pandemic will ultimately cause, but it seems safe to assume this will be a very large number. All of which points to a frenzy of litigation as the lawyers argue about who should ultimately bear the cost.
Force majeure has been a hot topic for contracts lawyers since the pandemic began. So I was curious to see how many court rulings have addressed the question whether the pandemic itself, and the government actions that followed, should be considered force majeure events that excuse contract suspension or termination. Cases in Canada and the U.S. have ruled that force majeure does excuse non-payment of rent, at least to some extent. A French court ruled that force majeure automatically suspended an energy supply agreement when the pandemic triggered a collapse in prices. And an Egyptian court ruled that the pandemic was a valid reason for delaying an election.
These decisions provide a few clues about how the humble force majeure clause will be interpreted over the years to come. But given the wildly variable nature of these clauses, you’ll need to do some careful analysis of the specific language lurking in your contract portfolio before drawing too many conclusions about how they protect or expose you.
According to this nifty COVID complaint tracker published by Hunton Andrews Kurth, there are probably many years of dispute ahead of us. As of August 2020 there are more than 4,600 COVID claims in the U.S. court system, including 1,103 insurance-related claims, 523 employment-related claims, 346 real estate claims and 290 contract disputes.
Of the 290 contract disputes, it appears that a majority have at least a force majeure element to them. Hot topics include (a) failure to provide a refund, (b) failure to close a deal (plenty of real estate examples), and (c) disputed termination (plenty of energy examples).
The large number of insurance (<1,000) and employment (>500) claims is to be expected. Many businesses will be discovering the devil in the details of insurance coverage and exclusions (who expected business interruption to last so long?). And with U.S. unemployment still above 8% in August, and the number of permanent job losers above 3 million, it’s not surprising that 277 unfair dismissal claims account for more than half the number of employment disputes.
I have long predicted that legal work will slowly but surely be sucked in-house (where efficiency is rewarded), leaving less work for outside counsel (where efficiency is the enemy of the billable hour). The recent rise of Legal Operations is a major factor in making this happen, and Gartner has provided some data to back it up.
First, in-house legal teams without dedicated Legal Ops staff typically spend 30% more than those with a Legal Ops capability. This is a meaningful number, and it makes sense. Legal Ops will be looking at efficiency, value and ROI. Without them, you’re probably wasting some of your legal spend. 30% of wasteful spend, it turns out.
Second, in-house teams can shift over 60% of their legal work to a self-service model, offering dramatic productivity improvements to those who invest in automation. This also makes sense. Guided requests, automated drafting, virtual legal advisers and other self-service tools are no longer cutting-edge technologies. At risk of sounding old, I personally built my first automated contract template more than 20 years ago. With legal ops staff to steer these technologies to successful adoption, the conditions are finally in place for automation to take off.
Speaking of self-service, Gartner has also predicted that by 2023, “lawbots” will handle 25% of all internal legal requests. This also makes sense. Self-service has a number of powerful selling points. You don’t need to wait. You don’t need to talk to anyone. And robots are cheaper than people.
The legal profession enjoys a number of protections. It’s illegal, for example, for non-lawyers to provide legal services. That’s called unauthorized practice of law. It will get you in trouble in most places. But there’s another important shield protecting lawyers from the competitive heat of automation and innovation: it’s what I call the “legal data firewall.”
What is the legal data firewall? In simple terms it’s a lack of structured data. Laws are written in words. Contracts are full of words. Lawyers are free to express their words in almost unlimited ways, and most take full advantage of that freedom. The result is an explosion of what geeks would call unstructured data. Loads of fuzzy text. Very little standardization. Oodles of nuance.
This is great fun if you’re a lawyer. You can read a fifty page contract and find all the juicy words that might expose your client to risk. You can translate that mess into some advice that’s slightly easier to digest, and you get paid for your efforts.
But it’s an absolute nightmare if you’re trying to automate something. If you want a machine to take over some of what lawyers do, you need structured data. You need to translate all the legalese into a common language of yes and no. You need to speak binary. You can’t say “it depends.”
So, that’s what I mean by the Legal Data Firewall. Without the ability to translate legal words into standardized or normalized, structured data, it’s almost impossible to automate legal tasks and analysis.
Suppose you want to automate a contract approval process. Rather than have every contract reviewed by Legal, you want a machine to assess an incoming document for materially risky terms. If there are no materially risky terms, you give the business a green light and let them sign the deal without further delay. This immediately begs the question: how does the machine decide what materially risky terms look like?
You might tell the machine that lack of price controls in a buy-side contract is materially risky. Now the machine needs to know what buy-side looks like, and what a pricing clause looks like. But that’s not enough. Risk often lurks in the specifics of a clause, not just its high level classification. A price review clause that indexes changes to CPI is relatively low risk for a buyer (once you’ve taught your machine about inflation and indexing you’re all set). A clause that locks in fixed annual percentage increases may be risky if that number is set too high. But “too high” is both subjective and fluid: it depends on context and changes in market conditions.
As you can see, the devil is in the details. Which means a structured data model must address many details before you can build a machine that replaces a human expert.
But saying it’s complex and difficult is not the same as saying it’s impossible. It is possible to create a very fine-grained data model into which most contracts can be translated. It is possible to normalize legal words into data. It’s really just a matter of time and effort.
The road to structured legal data has two possible paths: (1) standardization, where a standard model of words and data is agreed through a formal collective process; and (2) normalization, where an objective data model is developed as a translation layer for whatever legal words are used in the wild.
I am skeptical that a standardized model (path 1) will emerge any time soon. Lawyers have agreed standardized language in certain narrow industry contexts (derivatives, insurance and construction, to give three examples). But there are powerful forces working against broader standardization of legal language. First, lawyers like to argue. Big time. Second, standardization is seen by many as a path towards commoditization. Not a selling point.
Normalization (path 2), on the other hand, is in the hands of data-driven disruptors. Rather than dictating standard words, a normalization approach creates an objective model of issues, objects, risks and data capable of accepting any legalese through a mapping exercise. This is already happening (Coupa has a universal contract model, for example), and like many disruptive forces, it will reach a tipping point of maturity where it takes off because it works.
Think about it. In the current paradigm you ask an expert whether a contract is OK to sign and, some days later, you get a meandering “maybe” reply. Or, in the normalized, data-driven paradigm, you get a machine analysis with a green light (yes, you can sign), graphic visualization of risk factors, benchmarking of this deal against industry norms, and recommended price adjustments to offset expected risks. Which world would you choose?
Contracts are extremely valuable to your business. But contracts are weak link assets. It is your weakest contract that will cause you pain. It is the weakest terms of that contract that will do the same. And your contract negotiators are like a soccer team.
Wait. What? Let me explain.
About a year ago, I was in Las Vegas, as a freshly-inducted member of the Coupa village (Coupa had just acquired the CLM software company I co-founded roughly 20 years before). It was Inspire time – the annual Coupa conference – with Malcolm Gladwell as one of the featured speakers. And Malcolm gave a truly inspiring talk about strong and weak links.
As with much of Malcolm’s work, he took an interesting idea from somewhere else, and wove it into a compelling story of his own. The original idea comes from Chris Anderson and David Sally’s book, The Numbers Game: Why Everything You Know About Soccer Is Wrong. Malcolm’s adaptation was to apply the strong vs weak link analysis to educational philanthropy, and to organizational theory more generally.
Here’s the idea. Sometimes you get the most bang for buck by fixing weak links. Sometimes you get the most bang for buck by investing in strong links. Soccer, it turns out, is a weak link game. You’re much better off building a team with no weak links, rather than splashing all your cash on superstars like Messi or Neymar. Why? Because in a low scoring game like soccer, mistakes lose games more often than miraculous goals win them. Basketball, on the other hand, is a strong link game. With Michael Jordan, Scottie Pippin and Dennis Rodman, the ‘96 Bulls were destined for greatness, notwithstanding the relative weakness of their Australian team mate, Luc Longley. Basketball is a high scoring game where the odd mistake won’t kill you, but you need your strong players to get points on the board.
Reflecting on this idea of strong and weak links, it occurred to me that contracts are totally weak link assets. When times are good, and business is rolling along, contracts tend to stay in the bottom drawer. Plenty of contracts never see daylight after they’re signed, sealed and delivered. Strong protection? Weak protection? Who cares? But in times of crisis, when bad things happen, many contracts are going to be scrutinized under a bright, shiny light. Those deals better not have any weak links, because if they do, a not-so-friendly litigator is going to tear right through them.
This is quite the frustrating predicament for contract management professionals. When the pressure is on to wrap up negotiations and close a deal, the commercial decision-makers will focus on the big ticket issues (liability caps, etc.), with little patience for fighting over dozens of “rats and mice” clauses that could be future weak links. And they have a point. It’s often not worth blowing a deal up over an impasse on any one weak link issue. And so they linger on. Hundreds, sometimes thousands of weak links. Waiting for the moment to rear their ugly heads.
This problem is compounded by the fact that corporate and financial due diligence efforts typically adopt strong link analysis techniques to contract portfolios. Assessments of legal risk often apply materiality thresholds that limit review to major or high value contracts, and limit clause analysis to issues likely to have a large impact on the balance sheet or P&L. These keep everyone focused on strong link risks. This would be fine if contracting was basketball. But it isn’t. It’s soccer.
Force majeure is just the latest example of a potential weak link clause that was often overlooked. Most force majeure events rarely happen. If they do happen, they don’t last very long. But when these events happen, the pain can be very high, and everyone will be looking closely at the details, perhaps for the first time ever.
With the COVID-19 pandemic and government shutdowns triggering billions of dollars of losses across the globe, the humble force majeure clause may be the weak link that makes those losses fall disproportionately hard on your business. Maybe you had a handle on where the clause offered protection and where it did not. Your playbook ensured a mutual right to suspend performance for the duration of any force majeure event. But what about termination? Pre-COVID, many negotiators viewed force majeure as short, isolated blips, rather than extended, global, recession-inducing game-changers. The inclusion of a force majeure termination trigger, and the timing of that trigger, is thus much more random than we might hope. And if this weak link results in the unexpected early termination of multiple critical supply agreements, the disruption and costs will be substantial.
What’s the solution to this dilemma? The obvious starting point is to get a handle on the problem. Sure, everyone has weak links in their contracts, but not all weak links are the same. If you start by tracking and measuring where you have weak or risky terms, then you’ve got some chance of understanding the scale of pain that might be coming your way. Armed with that insight, you can take action to mitigate the risks. If you don’t even know where the weak links are lurking, you’ll feel a full load of pain. You can’t mitigate after the fact.
One of the keys to fixing contractual weak links is being able to translate all the legal mumbo-jumbo into data you can measure, and data you can trust. I will save my thoughts on the trusted data challenge for another time.
A good blog has focus. Kassandra will try to stay focused on the three P’s of prediction, perceptrons and professional services. Prediction, because insight into future events is extremely valuable — just ask Biff Tannen. Perceptrons, because AI is probably the biggest technological disruption of our generation, and perceptron — a type of neural network — conveniently starts with a P. Professional services, because this line of work is a perfect case study for the looming disruption, and I’ve spent most of my career trying to automate it (specifically, the legal branch of professional services).
Think of it as a personal challenge. Set the bar high. Kassandra was blessed to see the future. I will merely try to predict parts of it, and I am sure to get much of it wrong.
Kassandra with a K gives me some chance of differentiation from the graph database spelled with a C.
And like Cassandra, I fully expect people to disagree with me, which makes life infinitely more fun.
Kassandra, Esq. 