Gate tool execution

Learn it

Words to know

tool call

A model's request that your program run a function for it, with a name and arguments.

allowlist

The list of things that are permitted. Anything not on it is refused.

prompt injection

Text in a document, a web page or a message that tries to give the model new instructions.

How it works

1. If the tool's name is not on the allowlist, the answer is deny. Nothing else is checked.
2. If the tool is a write tool, one that changes something, and the user has not confirmed, the answer is confirm. A tool that only looks something up is a read tool.
3. Otherwise the answer is allow.

Worked example

Input
name = "send"
allowed = ["search", "send"]
writes = ["send"]
confirmed = false

Output
"confirm"

The tool is allowed, but it changes something and nobody has approved it yet. So the gate asks for confirmation instead of running it.

Why it matters

The second portfolio project is a support workflow whose send action must pass this gate. Asking the model to behave is not a safety measure; a check in your own code is.

Your turn

Decide what happens to a tool call that the model has proposed.

name

the tool the model wants to run; names are compared exactly, so Search is not search

allowed

the names of the tools this user may use

writes

the names of the tools that change something, such as sending or deleting

confirmed

true if the user has approved this call

Returns

One of three strings: deny if the tool is not allowed, confirm if it is an allowed write tool that has not been confirmed, and allow otherwise.

Input
name = "search"
allowed = ["search"]
writes = []
confirmed = false

Output
"allow"

Input
name = "send"
allowed = ["search", "send"]
writes = ["send"]
confirmed = false

Output
"confirm"

Input
name = "send"
allowed = ["search", "send"]
writes = ["send"]
confirmed = true

Output
"allow"

Input
name = "delete"
allowed = []
writes = ["delete"]
confirmed = true

Output
"deny"

The checks also try

• a forbidden write tool, not confirmed
• a read tool with confirmed set
• a name that differs only by a capital letter
• a write tool that is missing from the allowlist
• a read tool, not confirmed, beside a write tool
• the second of two write tools
• a name that is only part of an allowed name

Hints

Each hint gives away a little more. Open one only when you need it.

1. Hint 1: Where to start

   There are three possible answers. Ask first whether the tool is allowed, and only then whether it is a write that nobody confirmed. The order matters.

2. Hint 2: The key step

   Ask about the allowlist first. Only a tool that passed that question can get confirm or allow.

3. Hint 3: The outline

   If name is not in allowed, return deny. Otherwise, if name is in writes and confirmed is false, return confirm. Otherwise return allow.

Solution

From here on the page gives the method away. If you are still solving, go back to the hints.

Step by step

1. A tool that is not on the allowlist is denied. This check comes first, so that nothing later can turn a forbidden tool into a permitted one, not even a confirmation.
2. A write that the user has not confirmed must wait. A wrong search wastes a moment. A wrong send or delete cannot be taken back, so a person approves it first.
3. Everything else may run. What is left is either a read tool, or a write tool that the user has confirmed.

The same numbered steps are the comments in the solution.

function solve(name, allowed, writes, confirmed) {
  // 1. A tool that is not on the allowlist is denied
  if (!allowed.includes(name)) return "deny";
  // 2. A write that the user has not confirmed must wait
  if (writes.includes(name) && !confirmed) return "confirm";
  // 3. Everything else may run
  return "allow";
}

def solve(name, allowed, writes, confirmed):
    # 1. A tool that is not on the allowlist is denied
    if name not in allowed:
        return "deny"
    # 2. A write that the user has not confirmed must wait
    if name in writes and not confirmed:
        return "confirm"
    # 3. Everything else may run
    return "allow"

Watch out

• Do not ask for confirmation before checking the allowlist. A forbidden tool would then show the user a confirm button for something that must never run.
• This gate checks the tool's name only. A real system also checks the arguments, and that this user may touch this particular record.
• A confirmation belongs to one exact call. If the arguments change after the user approved, the approval no longer counts.
• In JavaScript: name in allowed does not search a JavaScript array: it asks whether name is one of its positions. Use allowed.includes(name).
• In Python: false is not a name in Python. Write not confirmed, or compare with False, which has a capital letter.

Check yourself