My favourite moment this week happened during a customer call. I was onboarding a new customer based in Melbourne onto Flank agents and explaining what supervision looks like once an agent is up and running, the process we go through to get there etc. She patiently absorbed all the context I was eagerly sharing, and then summed it up in nine words: “it’s like hiring a new junior onto the team”. I was elated.
This is someone who gets it.
I think it can actually be quite difficult to let go of pre-conceived notions when exploring agentic AI – preconceived notions developed over decades on extensive legal practice using helpful tools. Unlike buying a really good toolbox to get through jobs seamlessly, deploying an agent means onboarding a new hire. As a result, the process for deployment is completely different to that of installing new software and letting workers ‘use it’.
An agent is a dynamic addition to the team – just like a new hire, it needs an intentional onboarding process, followed by regular touchpoints for optimum calibration. Gradually after that, it needs less and less hand holding to the point that it only loops you in when strictly needed and otherwise works autonomously.
This means that the skill this paradigm actually asks from a legal team is management.
It starts with a set of rules
For most tasks performed by the legal team, there is a methodology to it. That methodology may or may not have been formulated yet, but it can certainly be extrapolated from precedent if needed.
This can be rules for how legal requests are triaged and routed to the correct individual or team, rules for how house templates are selected and drafted, rules on how counterparty objections on first party paper are dealt with or rules on how third party paper contracts should be handled at first round.
At Flank, a significant portion of my team’s work with Alignment involves codifying rules. We study existing playbooks (even if this means navigating three different versions across 7 teams and an equal number of jurisdictions), past negotiations, how escalated items were dealt with. There is a wealth of information that lives in this context and all of this is incredibly valuable to construct a comprehensive playbook for an agent. Whilst a junior picks things up by osmosis (although even this can be increasingly difficult these days in remote working environments), an agent relies on context that it can use as instructions or inference points.
Being aligned on expectations
An agent will do what it is built to do. That is the whole promise but it can also be the whole problem. That is why, the design stage is critical because, unlike a human, it will not read the room and adapt its tone or approach depending on whether it's dealing with Jane Doe or Joe Bloggs. It is reliable, consistent and operates 24/7 – it is therefore imperative that the team deploying the agent to enterprise as an extension of themselves is happy that the agent’s baseline meets the high quality bar required and is representative of that team.
Take an NDA review agent for example. You can build one that adopts a minimal redline, pragmatic approach to negotiation, with direct, no fluff comments to the counterparty. Or you can build one that is too thorough for its own good, picks up all issues under the sun and hands back something that looks like a bloodbath, making the reviewer wish they’d done it all themselves to start with.
Technically, both agents would be working correctly – the difference being them having been calibrated to different standards or perhaps less intentional and deliberate calibration if we look at the poor agent scenario.
Day-to-day, good calibration means adapting the redlining posture to the desired level to match the negotiation appetite, being happy with the tone and style of the agent’s comments (its persona), agreeing with the level of detail the agent shares in its responses (we ideally want the agent to operate like a good, commercially aware and pragmatic junior lawyer).
“But agents are too literal”
Many people get irritated when their AI does not do exactly what they need it to do. I’m confident that this mainly concerns people who haven’t seen an agent that was set up properly. Agents are rigid, they take everything literally, they over-redline, and you end up reviewing the review.
Being honest, I’ve seen it too and I would agree that’s a fair representation of what an agent does when none of the considerations I shared earlier on got thought through. That’s why calibration and having the right control room set up is absolutely essential to having an agent that does exactly what its organisation and humans in the loop need – you just need to treat it like onboarding a new hire.
It might sound like I am oversimplifying agentic AI deployment. I’m not. Training an agent is really as straightforward as training a person – I would argue that, provided the agent’s orchestration stacks up, it is even easier than onboarding a human to a new team or organisation.
Coachable, within the rules
Just like an ideal team member should be coachable, you want an agent that is equally coachable. This might, at first glance, appear as a contradiction to my two earlier rules which seem to be grounded in consistency and rigidity but bear with me. We do not expect perfection from a junior lawyer and we should not expect perfection from an AI agent either. That is why having the ability and by that I mean the logistical set up, to supervise an agent and train it to be more and more precise over time is the key to being able to rely on it consistently and long term.
At Flank, we do this through the supervision cockpit - this is where the human-in-the-loop is looped in, so to speak. That’s where our customers are able to view all agentic actions, step in and provide human judgment on escalation points, catch instances where the agent is doing a good job but the playbook rules may be too much of a departure from market practice or counterparties’ boundaries and therefore internal policy changes may be required. This birdseye view and ability to shape the agent’s work is the ultimate requirement for a lawyer to be able to stand behind what’s ultimately, the work product of an AI.
What does this mean for lawyers?
If you ask me, this is the part that excites me the most about how legal work is transforming. Everyone will need to develop the muscle for management – whether you’re using an AI assistant for your life admin or an AI agent to support all your deal team’s contract reviews, the system around those powerful agents is what will make or break long term success. This means building, directing and maintaining a system designed to work for you.
