← Articles

Let It Look. Be Careful What You Let It Do.

· By Dialogs

Let It Look. Be Careful What You Let It Do.

If your AI can send, change, or spend anything, read this first

A year or two ago, AI in a business mostly talked. You asked, it answered, and a person decided what to do with the answer.

That’s changing fast. The AI tools showing up in small and mid-sized businesses now do things: send the follow-up email, update the customer record, book the appointment, issue the refund. The industry calls these “agents,” which just means AI that takes actions instead of only answering. (If the vocabulary’s getting away from you, our field guide AI Isn’t One Thing covers it in plain English.)

Agents are where a lot of the real value is. They’re also where the one idea in this article matters most. It comes from Mozilla’s new State of Open Source AI report, and it’s simple enough to use in your next meeting:

**There’s a big difference between an AI looking at something and an AI doing something.**

Looking vs. doing, in Mozilla’s words

Mozilla splits everything an AI agent can do into two buckets. Engineers call them “reads” and “writes.” We’ll call them looking and doing.

Looking is pulling up a document, checking a calendar, searching the CRM. Mozilla calls these “reversible and low-consequence,” things that “can largely be permitted by default. A bad read costs little and can be repeated safely.”

Doing is “sending a message, spending against a budget, modifying a record, executing a transaction.” Actions with “side effects that are costly or irreversible.” And Mozilla is specific about where your protection belongs: “This is where confirmation, approval thresholds, cost caps and revocation must concentrate.”

Now the uncomfortable part. Mozilla mapped every piece of software that goes into running an AI agent, from the model itself to the tools, memory, and monitoring, and found products available for nearly every job. Except one. In their words: “Every sub-layer has products except permission.” And: “No portable model defines which writes an agent may perform unattended.”

In plain English: nobody sells a standard, off-the-shelf answer to “what should this AI be allowed to do without asking?” Whoever sets up your AI has to decide that. If nobody decided on purpose, the answer is usually “whatever its login allows.”

“We use a big-name product, so we’re covered.” Maybe not.

It’s natural to assume a well-known vendor has handled this. Mozilla’s report has a section titled, bluntly, “Closed is not the same as secure.”

It lists four serious, publicly disclosed security flaws in AI features from major business software companies, the kind of products a lot of companies already pay for. Each was different in its details. Mozilla describes them as having “one shape: retrieval checked, output not.”

Translated: in each case, the checks were on the way in: what the AI could reach, and whose permissions it used. Nothing was checking what came out, or where it went. In the first case on Mozilla’s list, an attacker sent an email with hidden instructions. Nobody opened it. The AI assistant read it anyway, alongside the company’s own files, and tucked private data into a web link pointing at a server the attacker controlled.

Different products, same blind spot: the way out.

These weren’t small companies cutting corners. They’re some of the most sophisticated software vendors in the world. That’s the point: this is new territory for everyone, and it’s worth checking on your own setup rather than assuming. We’ve written more about why AI can be talked into things like this in Stop Trying to Make the Model Safe. Make the System Safe., and about what happens when a capable AI finds a door someone left open in The AI Didn’t Break Out. The Door Was Open.

What good looks like

Here’s how we think about it when we look at what a business has built. None of it requires exotic technology. It’s the same thinking you already apply to a new employee.

Let it look freely, within reason. An AI that can read your knowledge base, search past orders, and check the calendar is useful and low-risk. Keep it to what the task needs; the customer-service assistant doesn’t need the payroll folder.

Sort every “doing” action into three piles:

  1. Safe to do on its own. Easily undone, low stakes. Drafting a reply into a queue, adding a note to a record, tagging an email.
  2. Needs a human yes. Hard to undo or visible to outsiders. Sending an email to a customer, changing an order, posting publicly.
  3. Needs a hard limit, no matter who says yes. Money and deletion. A refund cap per day. No bulk deletes, ever. A ceiling on spend that the AI cannot raise.

Have an off switch you’ve actually tested. Mozilla lists “revocation” alongside approvals and caps. Plainly, that means you can cut the AI’s access in one step, and you know who’s allowed to do it.

Check what goes out, not just what comes in. The four incidents above all failed on the way out. Pay attention to where your AI can send things: which email addresses, which websites, which systems.

You’d never give a new hire the company card, admin access to the CRM, and permission to email every customer on their first day. Not because you distrust them, but because that’s how a sensible business works. An AI agent deserves the same starting point.

What to do on Monday

Grab whoever set up your AI tools, or just open the settings yourself, and make one list: every action your AI can take without a person clicking “approve.”

Then, for each item, write down three things:

  1. Can we undo it? If not, it moves to “needs a human yes” or “needs a hard limit.”
  2. Who approves it? A name, not a department.
  3. What’s the cap? Dollars, messages, records, whatever the unit is.

If the list is short and every line has answers, you’re in good shape. If nobody can produce the list at all, that’s your answer too, and it’s worth fixing before you add the next tool.

Mozilla found that only about 21% of companies report mature governance of their AI agents. You don’t need a governance program to be ahead of that. You need one honest list.

If you’d like a second set of eyes on it, that’s a good use of a free half-hour with one of our senior engineers. And if the list turns up things that need rebuilding, that’s what Last Mile is for.


Share this article

LinkedIn X Email

← Back to all articles