Deep Dive into ellmer: Part 2

I parsed through the source code of ellmer so you don’t have to
ellmer
Author

Howard Baik

Published

September 16, 2025

Introduction

ellmer is an R package designed to easily call LLM APIs from R. It supports a wide variety of model providers, including Google Gemini/Vertex AI, Anthropic Claude, OpenAI, and Ollama. Although there are resources on how to use ellmer, as of this writing, there is no explanation on how ellmer works.

This blog series will dive deep into the source code of ellmer, aiming to build an understanding of how ellmer interfaces with LLM APIs. We will use the Google Gemini API, which provides a generous free tier.

Note that the ellmer R package is currently at version 0.3.0, indicating it is still in the early stages of development and subject to change.

library(ellmer)
# > packageVersion("ellmer")
# [1] ‘0.3.0’

Part 2 takes a deep dive into tool calling. Tool calling provides LLM with tools it can call. A tool is like a function and is defined by the function name, description, and a list of arguments. As Joe Cheng shares in his “Demystifying LLMs with Ellmer” talk, think of an LLM as a brain in a jar. It does not have any ability other than the questions you ask it. Tool calling equips this brain with the ability to interact with other systems.

According to the tool calling vignette in ellmer, this is how tool calling works:

Diagram of tool calling

The user asks the assistant a question, “What is the weather right now at Fenway Park?” along with a tool, which consists of a name of a function, a description of what it does, a list of arguments, and argument types. The assistant responds with a request to call the tool. Then, the user calls the tool, returns the result to the assistant, which uses it to generate the final answer.

Walkthrough of Tool-Calling

In the rest of this blog post, we will examine the tool calling functionality within the $chat() method of the Chat object. We will draw an example from the tool calling vignette in ellmer.

Setup

First, we initialize the chat object: chat <- chat_openai(). We will use OpenAI’s GPT 4.1 model. I will skip the details, as they were covered with great detail in Deep Dive into ellmer: Part 1.

# Arguments to chat_openai()
system_prompt = NULL
base_url = Sys.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
api_key = openai_key()
model = NULL
params = NULL
api_args = list()
api_headers = character()
echo = "output"

# Code inside chat_openai()
model <- set_default(model, "gpt-4.1")
echo <- check_echo(echo)
params <- params %||% params()

provider <- ProviderOpenAI(
  name = "OpenAI",
  base_url = base_url,
  model = model,
  params = params,
  extra_args = api_args,
  extra_headers = api_headers,
  api_key = api_key
)

# Initialize chat object
chat <- Chat$new(provider = provider, system_prompt = system_prompt, echo = echo)

# Needed for later calls
private <- list()
private$provider <- provider

Then, we define a function to get the current time, get_current_time():

#' Gets the current time in the given time zone.
#'
#' @param tz The time zone to get the current time in.
#' @return The current time in the given time zone.
get_current_time <- function(tz = "UTC") {
  format(Sys.time(), tz = tz, usetz = TRUE)
}

We turn the above function into a tool with tool():

get_current_time <- tool(
  get_current_time,
  name = "get_current_time",
  description = "Returns the current time.",
  arguments = list(
    tz = type_string(
      "Time zone to display the current time in. Defaults to `\"UTC\"`.",
      required = FALSE
    )
  )
)

Note that the tool calling vignette introduces a helper function that uses an LLM to generate this tool() call, create_tool_def(). This function may make mistakes, so the user must review the generated code. However, for more complicated functions, create_tool_def() is definitely a time-saver.

The final setup is to give our chat object access to our tool:

private$tools[[get_current_time@name]] <- get_current_time

$chat() method

In this section, we walk through the source code that powers the $chat() method (as it pertains to tool calling) with the prompt "How long ago did Neil Armstrong touch down on the moon?":

user_prompt <- "How long ago did Neil Armstrong touch down on the moon?"

turn <- user_turn(user_prompt)
echo <- check_echo(echo %||% private$echo)

As shown in the diagram of ellmer’s chat implementation in Deep Dive into ellmer: Part 1, the chat implementation method $chat_impl() leads to $submit_turns(), which then leads to chat_perform().

To better understand the “whole game” of tool calling, I’ve created a diagram of tool calling functions in ellmer:

Whole Game of Tool Calling
# Setup for chat_perform()
user_turn <- turn
yield_as_content <- FALSE
type = NULL
stream = TRUE

response <- chat_perform(
        provider = private$provider,
        mode = if (stream) "stream" else "value",
        turns = c(private$.turns, list(user_turn)),
        tools = if (is.null(type)) private$tools,
        type = type
      )

Calling chat_perform() results in a generator instance called response. A generator instance is created by calling a generator function:

> response
<generator/instance>
function(provider, req) {
    resp <- req_perform_connection(req)
    on.exit(close(resp))

    repeat {
      event <- chat_resp_stream(provider, resp)
      data <- stream_parse(provider, event)
      if (is.null(data)) {
        break
      } else {
        yield(data)
      }
    }
  }
<environment: namespace:ellmer>

In order to unpack the iterators within a generator, we can iterate manually with

coro::loop(for (x in response) {
  print(x)
})

Alternatively, we can collect all values of an iterator with coro::collect():

all_chunks <- coro::collect(response)

Then, we simplify the response with stream_merge_chunks():

result <- NULL
for (chunk in all_chunks) {
  result <- stream_merge_chunks(private$provider, result, chunk)
}

We observe the streaming response data where the function name is get_current_time() and the function argument is tz: "UTC":

[[1]]$delta$tool_calls[[1]]$`function`
[[1]]$delta$tool_calls[[1]]$`function`$name
[1] "get_current_time"

[[1]]$delta$tool_calls[[1]]$`function`$arguments
[1] "{\"tz\":\"UTC\"}"

Then, we create a Turn object for the assistant turn:

turn <- value_turn(private$provider, result, has_type = !is.null(type))

And we insert our tool function (get_current_time()) in the Turn object:

turn <- match_tools(turn, private$tools)

So far, we have the following user_turn (user turn) and turn (assistant turn):

> user_turn
<Turn: user>
How long ago did Neil Armstrong touch down on the moon?
> turn
<Turn: assistant>
[tool request (call_0rAWkevHlh4ZXHdfQbob3Aw8)]: get_current_time(tz = "UTC")

We add these turns to the Chat object:

# self$add_turn(user_turn, turn)
private$.turns[[length(private$.turns) + 1]] <- user_turn
private$.turns[[length(private$.turns) + 1]] <- turn

This concludes the private$submit_turns() method. To recap, the user asked the assistant a question, How long ago did Neil Armstrong touch down on the moon? and the assistant responded with a tool request, asking the user to invoke the tool function get_current_time(tz = "UTC").

Invoke tool

Next, we invoke the tool since the assistant turn has a tool request:

# assistant_turn <- self$last_turn()
assistant_turn <- turn
user_turn <- NULL

private$callback_on_tool_request <- CallbackManager$new(args = "request")
private$callback_on_tool_result <- CallbackManager$new(args = "result")

tool_calls <- invoke_tools(
            assistant_turn,
            echo = echo,
            on_tool_request = private$callback_on_tool_request$invoke,
            on_tool_result = private$callback_on_tool_result$invoke,
            yield_request = yield_as_content
          )

The invoke_tools() function is a generator function that invokes all the tool requests in the assistant turn with the similar sounding invoke_tool() function.

Let’s take a look at the invoke_tool() function, which invokes an individual tool:

invoke_tool <- function(request) {
  if (is.null(request@tool)) {
    return(new_tool_result(request, error = "Unknown tool"))
  }

  args <- tool_request_args(request)
  if (is_tool_result(args)) {
    # Failed to convert the arguments
    return(args)
  }

  tryCatch(
    {
      result <- do.call(request@tool, args)
      new_tool_result(request, result)
    },
    error = function(e) {
      new_tool_result(request, error = e)
    }
  )
}

It takes a list of arguments for the tool function and uses do.call() to execute a function call. Then, it stores the result and the original request in a ContentToolResult object.

Tool calling in ellmer involves two S7 classes, ContentToolRequest for tool requests and ContentToolResult for tool results.

Properties of ContentToolRequest:

  • id: Tool call id
  • name: Name of tool function
  • arguments: Arguments to call function
  • tool: Tool metadata

Properties of ContentToolResult:

  • value: The result of calling tool function
  • error: Error message when calling tool function and it throws an error
  • extra: Optional additional data
  • request: The ContentToolRequest associated with tool result.

The request, a ContentToolRequest object, looks like:

> request
<ellmer::ContentToolRequest>
 @ id       : chr "[call_0rAWkevHlh4ZXHdfQbob3Aw8]"
 @ name     : chr "get_current_time"
 @ arguments:List of 1
 .. $ tz: chr "UTC"
 @ tool     : <ellmer::ToolDef> function (tz = "UTC")  
 .. @ name       : chr "get_current_time"
 .. @ description: chr "Returns the current time."
 .. @ arguments  : <ellmer::TypeObject>
 .. .. @ description          : NULL
 .. .. @ required             : logi TRUE
 .. .. @ properties           :List of 1
 .. .. .. $ tz: <ellmer::TypeBasic>
 .. .. ..  ..@ description: chr "Time zone to display the current time in. Defaults to `\"UTC\"`."
 .. .. ..  ..@ required   : logi FALSE
 .. .. ..  ..@ type       : chr "string"
 .. .. @ additional_properties: logi FALSE
 .. @ convert    : logi TRUE
 .. @ annotations: list()

The result, a ContentToolResult object, looks like:

<ellmer::ContentToolResult>
 @ value  : chr "2025-09-12 02:03:03 UTC"
 @ error  : NULL
 @ extra  : list()
 @ request: <ellmer::ContentToolRequest>
 .. @ id       : chr "call_r2el9wHpfQEmvBFw9kTKQGMP"
 .. @ name     : chr "get_current_time"
 .. @ arguments:List of 1
 .. .. $ tz: chr "UTC"
 .. @ tool     : <ellmer::ToolDef> function (tz = "UTC")  
 .. .. @ name       : chr "get_current_time"
 .. .. @ description: chr "Returns the current time."
 .. .. @ arguments  : <ellmer::TypeObject>
 .. .. .. @ description          : NULL
 .. .. .. @ required             : logi TRUE
 .. .. .. @ properties           :List of 1
 .. .. .. .. $ tz: <ellmer::TypeBasic>
 .. .. .. ..  ..@ description: chr "Time zone to display the current time in. Defaults to `\"UTC\"`."
 .. .. .. ..  ..@ required   : logi FALSE
 .. .. .. ..  ..@ type       : chr "string"
 .. .. .. @ additional_properties: logi FALSE
 .. .. @ convert    : logi TRUE
 .. .. @ annotations: list()

Finally, the output of invoke_tools() is a generator instance and is created by calling a generator function.

We loop through tool_calls and store our tool results in tool_results:

tool_results <- list()

coro::loop(for (tool_step in tool_calls) {
    if (is_tool_result(tool_step)) {
      tool_results <- c(tool_results, list(tool_step))
    }
})

The output of tool_results:

> tool_results
[[1]]
<ellmer::ContentToolResult>
 @ value  : chr "2025-09-05 03:37:59 UTC"
 @ error  : NULL
 @ extra  : list()
 @ request: <ellmer::ContentToolRequest>
 .. @ id       : chr "call_0rAWkevHlh4ZXHdfQbob3Aw8"
 .. @ name     : chr "get_current_time"
 .. @ arguments:List of 1
 .. .. $ tz: chr "UTC"
 .. @ tool     : <ellmer::ToolDef> function (tz = "UTC")  
 .. .. @ name       : chr "get_current_time"
 .. .. @ description: chr "Returns the current time."
 .. .. @ arguments  : <ellmer::TypeObject>
 .. .. .. @ description          : NULL
 .. .. .. @ required             : logi TRUE
 .. .. .. @ properties           :List of 1
 .. .. .. .. $ tz: <ellmer::TypeBasic>
 .. .. .. ..  ..@ description: chr "Time zone to display the current time in. Defaults to `\"UTC\"`."
 .. .. .. ..  ..@ required   : logi FALSE
 .. .. .. ..  ..@ type       : chr "string"
 .. .. .. @ additional_properties: logi FALSE
 .. .. @ convert    : logi TRUE
 .. .. @ annotations: list()

Next, we store the tool result in user_turn (user turn):

user_turn <- tool_results_as_turn(tool_results)

Output of user_turn shows the tool result:

<Turn: user>
[tool result  (call_0rAWkevHlh4ZXHdfQbob3Aw8)]: 2025-09-05 03:37:59 UTC

After completion of the while loop, we go back to the top of the loop. Since user_turn is not NULL, we run private$submit_turns() again.

The content of the result, which is the assistant’s response, is the following:

$choices[[1]]$delta$content
[1] "Neil Armstrong touched down on the moon on July 20, 1969. As of today (September 6, 2025), it has been 56 years, 1 month, and 17 days since Neil Armstrong first set foot on the lunar surface."

And we insert the result and provider into turn, for assistant turn:

turn <- value_turn(private$provider, result, has_type = !is.null(type))
turn <- match_tools(turn, private$tools)

The assistant turn contains the response of the assistant when the user sent the tool result:

> turn
<Turn: assistant>
Neil Armstrong touched down on the moon on July 20, 1969. As of today (September 6, 2025), it has been 56 years, 1 month, and 17 days since Neil Armstrong first set foot on the lunar surface.

We add these turns to the Chat object:

# self$add_turn(user_turn, turn)
private$.turns[[length(private$.turns) + 1]] <- user_turn
private$.turns[[length(private$.turns) + 1]] <- turn

This concludes private$submit_turns().

assistant_turn <- self$last_turn()
user_turn <- NULL # Important

Since turn_has_tool_request(assistant_turn) is FALSE, that completes the while loop. We go back to the top of the while loop, and exit out since user_turn is NULL. This concludes private$chat_impl(), and hence completes the $chat() method.

Conclusion

In this blog post, we took a closer look at tool calling in ellmer. We traced through the source code of the $chat() method to understand the mechanics of tool calling. The tool calling workflow is as follows: user sends a question along with a tool to the assistant, the assistant responds with a tool request, the user calls the tool and then submits the tool result to the assistant, and finally the assistant responds with its final answer.

More on other capabilities of ellmer, such as structured data, in the next blog post.

Thanks for reading.