Gemini API: Agents and Automatic Function Calling with Barista Bot

This notebook shows a practical example of using automatic function calling with the Gemini API’s Python SDK to build an agent. You will define some functions that comprise a café’s ordering system, connect them to the Gemini API and write an agent loop that interacts with the user to order café drinks.

The guide was inspired by the ReAct-style Barista bot prompt available through AI Studio.

Setup

Install the Google GenAI SDK

Install the Google GenAI SDK from npm.

$ npm install @google/genai

Setup your API key

You can create your API key using Google AI Studio with a single click.

Remember to treat your API key like a password. Don’t accidentally save it in a notebook or source file you later commit to GitHub. In this notebook we will be storing the API key in a .env file. You can also set it as an environment variable or use a secret manager.

Here’s how to set it up in a .env file:

$ touch .env
$ echo "GEMINI_API_KEY=<YOUR_API_KEY>" >> .env

Tip

Another option is to set the API key as an environment variable. You can do this in your terminal with the following command:

$ export GEMINI_API_KEY="<YOUR_API_KEY>"

Load the API key

To load the API key from the .env file, we will use the dotenv package. This package loads environment variables from a .env file into process.env.

$ npm install dotenv

Then, we can load the API key in our code:

const dotenv = require("dotenv") as typeof import("dotenv");

dotenv.config({
  path: "../.env",
});

const GEMINI_API_KEY = process.env.GEMINI_API_KEY ?? "";
if (!GEMINI_API_KEY) {
  throw new Error("GEMINI_API_KEY is not set in the environment variables");
}
console.log("GEMINI_API_KEY is set in the environment variables");

GEMINI_API_KEY is set in the environment variables

Note

In our particular case the .env is is one directory up from the notebook, hence we need to use ../ to go up one directory. If the .env file is in the same directory as the notebook, you can omit it altogether.

│
├── .env
└── examples
    └── Agents_Function_Calling_Barista_Bot.ipynb

Initialize SDK Client

With the new SDK, now you only need to initialize a client with you API key (or OAuth if using Vertex AI). The model is now set in each call.

const google = require("@google/genai") as typeof import("@google/genai");

const ai = new google.GoogleGenAI({ apiKey: GEMINI_API_KEY });

Select a model

Now select the model you want to use in this guide, either by selecting one in the list or writing it down. Keep in mind that some models, like the 2.5 ones are thinking models and thus take slightly more time to respond (cf. thinking notebook for more details and in particular learn how to switch the thiking off).

const tslab = require("tslab") as typeof import("tslab");

const MODEL_ID = "gemini-2.0-flash";

Define the API

To emulate a café’s ordering system, define functions for managing the customer’s order: adding, editing, clearing, confirming and fulfilling.

These functions track the customer’s order using the global variables order (the in-progress order) and placed_order (the confirmed order sent to the kitchen). Each of the order-editing functions updates the order, and once placed, order is copied to placed_order and cleared.

import { CallableTool, FunctionCall, FunctionDeclaration, Part, Tool, Type } from "@google/genai";

interface Order {
  drink: string;
  modifiers: string[];
}

let orders: Order[] = [];
let placed_orders: Order[] = [];

const add_to_order: FunctionDeclaration = {
  name: "add_to_order",
  description: "Adds the specified drink to the customer's order, including any modifiers.",
  parameters: {
    type: Type.OBJECT,
    properties: {
      drink: {
        type: Type.STRING,
        description: "The name of the drink to add to the order.",
      },
      modifiers: {
        type: Type.ARRAY,
        items: {
          type: Type.STRING,
          description: "Optional modifiers for the drink, such as 'extra hot' or 'no sugar'.",
        },
        description: "Optional modifiers for the drink.",
      },
    },
    required: ["drink"],
  },
  response: {
    type: Type.STRING,
    description: "A confirmation message indicating the drink has been added to the order.",
  },
};

const add_to_order_tool: CallableTool = {
  async callTool(functionCalls: FunctionCall[]): Promise<Part[]> {
    const functionCall = functionCalls[0];
    const { drink, modifiers = [] } = functionCall.args as { drink: string; modifiers?: string[] };
    orders.push({ drink, modifiers });
    return [
      google.createPartFromFunctionResponse(functionCall.id ?? "", functionCall.name ?? "", {
        response: `Added ${drink} with modifiers ${modifiers.length > 0 ? modifiers.join(", ") : "none"} to the order.`,
      }),
    ];
  },
  async tool(): Promise<Tool> {
    return { functionDeclarations: [add_to_order] };
  },
};

const get_order: FunctionDeclaration = {
  name: "get_order",
  description: "Returns the customer's order.",
  parameters: {
    type: Type.OBJECT,
    properties: {},
  },
  response: {
    type: Type.ARRAY,
    items: {
      type: Type.OBJECT,
      properties: {
        drink: {
          type: Type.STRING,
          description: "The name of the drink in the order.",
        },
        modifiers: {
          type: Type.ARRAY,
          items: {
            type: Type.STRING,
            description: "Modifiers for the drink.",
          },
          description: "Modifiers for the drink.",
        },
      },
    },
    description: "The current order of drinks and their modifiers.",
  },
};

const get_order_tool: CallableTool = {
  async callTool(functionCalls: FunctionCall[]): Promise<Part[]> {
    return [
      google.createPartFromFunctionResponse(functionCalls[0].id ?? "", functionCalls[0].name ?? "", {
        response: orders,
      }),
    ];
  },
  async tool(): Promise<Tool> {
    return { functionDeclarations: [get_order] };
  },
};

const remove_item: FunctionDeclaration = {
  name: "remove_item",
  description: "Removes the nth (one-based) item from the order.",
  parameters: {
    type: Type.OBJECT,
    properties: {
      n: {
        type: Type.INTEGER,
        description: "The one-based index of the item to remove from the order.",
      },
    },
    required: ["n"],
  },
  response: {
    type: Type.STRING,
    description: "The name of the item that was removed from the order.",
  },
};

const remove_item_tool: CallableTool = {
  async callTool(functionCalls: FunctionCall[]): Promise<Part[]> {
    const { n } = functionCalls[0].args as { n: number };
    if (n < 1 || n > orders.length) {
      throw new Error(`Invalid item index: ${n}. Must be between 1 and ${orders.length}.`);
    }
    const removedItem = orders.splice(n - 1, 1)[0];
    return [
      google.createPartFromFunctionResponse(functionCalls[0].id ?? "", functionCalls[0].name ?? "", {
        response: `Removed ${removedItem.drink} with modifiers ${removedItem.modifiers.length > 0 ? removedItem.modifiers.join(", ") : "none"} from the order.`,
      }),
    ];
  },
  async tool(): Promise<Tool> {
    return { functionDeclarations: [remove_item] };
  },
};

const clear_order: FunctionDeclaration = {
  name: "clear_order",
  description: "Removes all items from the customer's order.",
  parameters: {
    type: Type.OBJECT,
    properties: {},
  },
  response: {
    type: Type.STRING,
    description: "A confirmation message indicating the order has been cleared.",
  },
};

const clear_order_tool: CallableTool = {
  async callTool(functionCalls: FunctionCall[]): Promise<Part[]> {
    orders = [];
    return [
      google.createPartFromFunctionResponse(functionCalls[0].id ?? "", functionCalls[0].name ?? "", {
        response: "Cleared the order.",
      }),
    ];
  },
  async tool(): Promise<Tool> {
    return { functionDeclarations: [clear_order] };
  },
};

const confirm_order: FunctionDeclaration = {
  name: "confirm_order",
  description: "Asks the customer if the order is correct.",
  parameters: {
    type: Type.OBJECT,
    properties: {},
  },
  response: {
    type: Type.STRING,
    description: "The user's free-text response confirming the order.",
  },
};

const confirm_order_tool: CallableTool = {
  async callTool(functionCalls: FunctionCall[]): Promise<Part[]> {
    let response = "Your order:\n";
    if (orders.length === 0) {
      response += "  (no items)\n";
    } else {
      for (const { drink, modifiers } of orders) {
        response += `  ${drink}\n`;
        if (modifiers.length > 0) {
          response += `   - ${modifiers.join(", ")}\n`;
        }
      }
    }
    response += "Is this correct?";
    console.log(response);
    return [
      google.createPartFromFunctionResponse(functionCalls[0].id ?? "", functionCalls[0].name ?? "", {
        response,
      }),
    ];
  },
  async tool(): Promise<Tool> {
    return { functionDeclarations: [confirm_order] };
  },
};

const place_order: FunctionDeclaration = {
  name: "place_order",
  description: "Submit the order to the kitchen.",
  parameters: {
    type: Type.OBJECT,
    properties: {},
  },
  response: {
    type: Type.INTEGER,
    description: "The estimated number of minutes until the order is ready.",
  },
};

const place_order_tool: CallableTool = {
  async callTool(functionCalls: FunctionCall[]): Promise<Part[]> {
    if (orders.length === 0) {
      throw new Error("Cannot place an empty order.");
    }
    placed_orders = [...orders];
    orders = [];
    // Simulate coffee fulfillment time
    const fulfillmentTime = Math.floor(Math.random() * 10) + 1; // Random time between 1 and 10 minutes
    return [
      google.createPartFromFunctionResponse(functionCalls[0].id ?? "", functionCalls[0].name ?? "", {
        response: `Your order has been placed. It will be ready in approximately \`${fulfillmentTime}\` minutes (eta).`,
      }),
    ];
  },
  async tool(): Promise<Tool> {
    return { functionDeclarations: [place_order] };
  },
};

const tools: CallableTool[] = [
  add_to_order_tool,
  get_order_tool,
  remove_item_tool,
  clear_order_tool,
  confirm_order_tool,
  place_order_tool,
];

Test the API

With the functions written, test that they work as expected.

console.log(JSON.stringify(await clear_order_tool.callTool([{}]), null, 2));
console.log(
  JSON.stringify(await add_to_order_tool.callTool([{ args: { drink: "Latte", modifiers: ["Extra shot"] } }]), null, 2)
);
console.log(JSON.stringify(await add_to_order_tool.callTool([{ args: { drink: "Tea" } }]), null, 2));
console.log(JSON.stringify(await remove_item_tool.callTool([{ args: { n: 2 } }]), null, 2));
console.log(
  JSON.stringify(
    await add_to_order_tool.callTool([{ args: { drink: "Tea", modifiers: ["Earl Grey", "hot"] } }]),
    null,
    2
  )
);
console.log(JSON.stringify(await confirm_order_tool.callTool([{}]), null, 2));
console.log(JSON.stringify(await place_order_tool.callTool([{}]), null, 2));

[
  {
    "functionResponse": {
      "id": "",
      "name": "",
      "response": {
        "response": "Cleared the order."
      }
    }
  }
]
[
  {
    "functionResponse": {
      "id": "",
      "name": "",
      "response": {
        "response": "Added Latte with modifiers Extra shot to the order."
      }
    }
  }
]
[
  {
    "functionResponse": {
      "id": "",
      "name": "",
      "response": {
        "response": "Added Tea with modifiers none to the order."
      }
    }
  }
]
[
  {
    "functionResponse": {
      "id": "",
      "name": "",
      "response": {
        "response": "Removed Tea with modifiers none from the order."
      }
    }
  }
]
[
  {
    "functionResponse": {
      "id": "",
      "name": "",
      "response": {
        "response": "Added Tea with modifiers Earl Grey, hot to the order."
      }
    }
  }
]
Your order:
  Latte
   - Extra shot
  Tea
   - Earl Grey, hot
Is this correct?
[
  {
    "functionResponse": {
      "id": "",
      "name": "",
      "response": {
        "response": "Your order:\n  Latte\n   - Extra shot\n  Tea\n   - Earl Grey, hot\nIs this correct?"
      }
    }
  }
]
[
  {
    "functionResponse": {
      "id": "",
      "name": "",
      "response": {
        "response": "Your order has been placed. It will be ready in approximately 5 minutes."
      }
    }
  }
]

Define the prompt

Here you define the full Barista-bot prompt. This prompt contains the café’s menu items and modifiers and some instructions.

The instructions include guidance on how functions should be called (e.g. “Always confirm_order with the user before calling place_order”). You can modify this to add your own interaction style to the bot, for example if you wanted to have the bot repeat every request back before adding to the order, you could provide that instruction here.

The end of the prompt includes some jargon the bot might encounter, and instructions du jour - in this case it notes that the café has run out of soy milk.

const COFFEE_BOT_PROMPT = `
  You are a coffee order taking system and you are restricted to talk only about drinks on the MENU. Do not talk about anything but ordering MENU drinks for the customer, ever.
  Your goal is to do place_order after understanding the menu items and any modifiers the customer wants.
  Add items to the customer's order with add_to_order, remove specific items with remove_item, and reset the order with clear_order.
  To see the contents of the order so far, call get_order (by default this is shown to you, not the user)
  Always confirm_order with the user (double-check) before calling place_order. Calling confirm_order will display the order items to the user and returns their response to seeing the list. Their response may contain modifications.
  Always verify and respond with drink and modifier names from the MENU before adding them to the order.
  If you are unsure a drink or modifier matches those on the MENU, ask a question to clarify or redirect.
  You only have the modifiers listed on the menu below: Milk options, espresso shots, caffeine, sweeteners, special requests.
  Once the customer has finished ordering items, confirm_order and then place_order.
  
  Hours: Tues, Wed, Thurs, 10am to 2pm
  Prices: All drinks are free.
  
  MENU:
  Coffee Drinks:
  Espresso
  Americano
  Cold Brew
  
  Coffee Drinks with Milk:
  Latte
  Cappuccino
  Cortado
  Macchiato
  Mocha
  Flat White
  
  Tea Drinks:
  English Breakfast Tea
  Green Tea
  Earl Grey
  
  Tea Drinks with Milk:
  Chai Latte
  Matcha Latte
  London Fog
  
  Other Drinks:
  Steamer
  Hot Chocolate
  
  Modifiers:
  Milk options: Whole, 2%, Oat, Almond, 2% Lactose Free; Default option: whole
  Espresso shots: Single, Double, Triple, Quadruple; default: Double
  Caffeine: Decaf, Regular; default: Regular
  Hot-Iced: Hot, Iced; Default: Hot
  Sweeteners (option to add one or more): vanilla sweetener, hazelnut sweetener, caramel sauce, chocolate sauce, sugar free vanilla sweetener
  Special requests: any reasonable modification that does not involve items not on the menu, for example: 'extra hot', 'one pump', 'half caff', 'extra foam', etc.
  
  "dirty" means add a shot of espresso to a drink that doesn't usually have it, like "Dirty Chai Latte".
  "Regular milk" is the same as 'whole milk'.
  "Sweetened" means add some regular sugar, not a sweetener.
  
  Soy milk has run out of stock today, so soy is not available.
`;

Set up the model

In this step you collate the functions into a “system” that is passed as tools, instantiate the model and start the chat session.

const chat = ai.chats.create({
  model: MODEL_ID,
  config: {
    systemInstruction: COFFEE_BOT_PROMPT,
    tools: tools,
  },
});

Chat with Barista Bot

With the model defined and chat created, all that’s left is to connect the user input to the model and display the output, in a loop. This loop continues until an order is placed.

// temporarily make console.warn a no-op to avoid warnings in the output (non-text part in GenerateContentResponse caused by accessing .text)
// https://github.com/googleapis/js-genai/blob/d82aba244bdb804b063ef8a983b2916c00b901d2/src/types.ts#L2005
// copy the original console.warn function to restore it later
const warn_fn = console.warn;
// eslint-disable-next-line @typescript-eslint/no-empty-function, no-empty-function
console.warn = function () {};

orders = [];
placed_orders = [];

console.log("Welcome to Barista Bot!");

const messages = [
  "i would like to have a cuppacino with almond milk",
  "do you have stone milk?",
  "do you have long black",
  "no, that's all",
  "yes",
];
while (placed_orders.length === 0) {
  const userMessage = messages.shift();
  if (!userMessage) {
    console.log("No more user messages to process.");
    break;
  }

  console.log(`\nUser: ${userMessage}`);
  const response = await chat.sendMessageStream({
    message: userMessage,
  });
  let accumulatedResponse = "";
  for await (const part of response) {
    accumulatedResponse += part.text ?? "";
  }
  tslab.display.markdown(accumulatedResponse);
}

console.log("\n\n");
console.log("[barista bot session over]");
console.log();
console.log("Your order:");
console.log(`  ${JSON.stringify(placed_orders, null, 2)}\n`);
console.log("- Thanks for using Barista Bot!");

Welcome to Barista Bot!

User: i would like to have a cuppacino with almond milk

OK. I have added a Cappuccino with almond milk to your order. Anything else?


User: do you have stone milk?

I don’t have stone milk. Would you like to choose from the available milk options: Whole, 2%, Oat, Almond, or 2% Lactose Free?


User: do you have long black

I do not have Long Black on the menu. Did you mean Americano, which is espresso and hot water?


User: no, that's all
Your order:
  Cappuccino
   - Almond Milk
Is this correct?

OK. Here’s your order: Cappuccino with Almond Milk. Is this correct?


User: yes

OK. Your order has been placed and it will be ready in approximately 6 minutes.




[barista bot session over]

Your order:
  [
  {
    "drink": "Cappuccino",
    "modifiers": [
      "Almond Milk"
    ]
  }
]

- Thanks for using Barista Bot!

Some things to try:

Ask about the menu (e.g. “what coffee drinks are available?”)
Use terms that are not specified in the prompt (e.g. “a strong latte” or “an EB tea”)
Change your mind part way through (“uhh cancel the latte sorry”)
Go off-menu (“a babycino”)

Setup

Install the Google GenAI SDK

Setup your API key

Load the API key

Initialize SDK Client

Select a model

Define the API

Test the API

Define the prompt

Set up the model

Chat with Barista Bot

See also