Republish: GenServer.reply: Don't Call Us, We'll Call You

Let’s look at how we can use :noreply and GenServer.reply to allow a GenServer to continue working even while its callers wait for the result of long-running operations.

I originally published this article on the Sequin blog. Republishing here with permission.

GenServers are one of the core abstractions provided by the OTP library that both Erlang and Elixir share. A GenServer (generic server) is a separate process that maintains state and provides a way to run code asynchronously.

This article assumes some familiarity with GenServers and how to implement them. If you don’t have that background knowledge, start with the GenServer guide and documentation.

Conceptually, a GenServer process has a “mailbox” – a queue of messages that it needs to process. It processes one message fully before moving on to the next. If the handling of a message does something that takes a long time, the GenServer will not process any further messages until the long operation completes.

Every introductory tutorial about GenServers talks about the two main ways of interacting with a GenServer: call and cast. Briefly: call is used to send a message to a GenServer and wait for its response before moving on. cast is used to send a message to a GenServer without waiting for a reply.

One way to think about this is that, with call, both the caller and the GenServer are “blocked” while the message is being handled. The caller waits for the reply, and the GenServer is busy handing the call and not processing any other messages. With cast, only the GenServer is blocked. The caller continues on, while the GenServer stays busy handling the cast and not processing any other messages.

This might be OK if the GenServer is a “worker” that doesn’t need to respond to additional messages while it’s doing its job. But if the GenServer needs to handle messages from multiple clients, we don’t want it to become unresponsive while the work is happening. If it does, the GenServer can become a bottleneck in the system and cause significant performance problems.

In that case, we want the caller to wait for a reply from the GenServer, but we want the GenServer to be able to process additional messages while a long operation is running. For example, perhaps the GenServer manages a cache of values that take time to compute. If a request comes in for a value that’s not in the cache, the caller should wait for the computation to finish, but other callers should still be able to request already-cached values without having to wait for the unrelated computation to finish.

GenServer has a built-in mechanism to support this pattern. We can return a {:noreply, ...} tuple from its handle_call callback and then later use GenServer.reply to reply to the caller.

Let’s look at an example to see how this works.

Common usage with :reply

We’ll start with a simple GenServer that uses Process.sleep to simulate a slow operation.

defmodule ReplyExample.Server do
  use GenServer

  @timeout :timer.seconds(30)

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def do_the_thing(pid \\ __MODULE__, n) do
    GenServer.call(pid, {:do_the_thing, n}, @timeout)
  end

  @impl GenServer
  def init(_opts) do
    {:ok, %{}}
  end

  @impl GenServer
  def handle_call({:do_the_thing, n}, _from, state) do
    log("Sleeping for 2 seconds...")
    Process.sleep(2000)

    {:reply, n * 1000, state}
  end

  defp log(message) do
    now = DateTime.utc_now() |> DateTime.truncate(:second)
    IO.puts("#{now}: #{message}")
  end

  def test do
    {:ok, pid} = start_link()

    try do
      1..5
      |> Enum.map(fn n -> Task.async(fn -> do_the_thing(n) end) end)
      |> Task.await_many(@timeout)
      |> inspect()
      |> log()
    after
      GenServer.stop(pid)
    end
  end
end

This is a pretty simple GenServer. It has a single function, do_the_thing. When invoked, the caller will wait for a response which will come after a two second delay.

In IEx, I can use the test function to start up five tasks, each calling do_the_thing, and then wait for all of them to complete.

iex(7)> ReplyExample.Server.test()
2023-07-28 16:36:26Z: Sleeping for 2 seconds...
2023-07-28 16:36:28Z: Sleeping for 2 seconds...
2023-07-28 16:36:30Z: Sleeping for 2 seconds...
2023-07-28 16:36:32Z: Sleeping for 2 seconds...
2023-07-28 16:36:34Z: Sleeping for 2 seconds...
2023-07-28 16:36:36Z: [1000, 2000, 3000, 4000, 5000]
:ok

Notice the timestamps. Each request is processed only after the previous one has completed. In order for this to complete successfully, I’ve had to explicitly add a long enough timeout to both the GenServer.call and the Task.await_many calls. Otherwise, one or more of the GenServer.calls would have timed out after the default five seconds.

With this approach, you can imagine a busy GenServer becoming a bottleneck in the system with numerous processes queued up waiting for their turn.

Using :noreply and GenServer.reply

Let’s refactor this code to move the long-running operation into a separate task. We can then use :noreply and GenServer.reply to allow the GenServer to continue processing other messages in the mean time.

defmodule ReplyExample.Server do
  use GenServer

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def do_the_thing(pid \\ __MODULE__, n) do
    GenServer.call(pid, {:do_the_thing, n})
  end

  @impl GenServer
  def init(_opts) do
    {:ok, %{}}
  end

  @impl GenServer
  def handle_call({:do_the_thing, n}, from, state) do
    Task.async(fn ->
      log("Sleeping for 2 seconds...")
      Process.sleep(2000)

      GenServer.reply(from, n * 1000)
    end)

    {:noreply, state}
  end

  @impl GenServer
  def handle_info(_msg, state) do
    {:noreply, state}
  end

  defp log(message) do
    now = DateTime.utc_now() |> DateTime.truncate(:second)
    IO.puts("#{now}: #{message}")
  end

  def test do
    {:ok, pid} = start_link()

    try do
      1..5
      |> Enum.map(fn n -> Task.async(fn -> do_the_thing(n) end) end)
      |> Task.await_many()
      |> inspect()
      |> log()
    after
      GenServer.stop(pid)
    end
  end
end

The significant changes from the previous version are in the handle_call callback, plus the addition of a handle_info callback ¹.

In handle_call, the code is almost identical, but has been moved into an anonymous function that is passed to Task.async. There are other ways to do this, but Task.async is a great fit for this situation ².

After spawning the task, handle_call immediately returns {:noreply, state}. That allows the GenServer to move on to the next message in its mailbox while the caller stays blocked waiting for the reply.

At the end of the task, rather than returning a result, the anonymous function calls GenServer.reply, passing the address of the original caller (the from parameter) and the actual reply. GenServer.reply is what returns the result to the caller, unblocking it.

If you’ve written handle_call callbacks before, you have probably always ignored the from parameter because it’s normally not used. But in this case, from becomes very useful because we can use it to reply to the correct caller.

With these changes, we can again test our GenServer and see the results:

iex(4)> ReplyExample.Server.test()
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:32Z: [1000, 2000, 3000, 4000, 5000]
:ok

Notice the timestamps again. This time, all of the Sleeping for... messages happen within the same second and the final response comes in after 2 seconds. This shows that the GenServer keeps processing messages even though all of the callers are blocked waiting for their replies! And the overall test runs much faster because we’ve been able to run the long operations concurrently.

Because the GenServer stays unblocked, we no longer need to add the long timeouts. Here, I’ve eliminated them entirely, falling back to the default of five seconds.

Wrapping Up

:noreply and GenServer.reply give you a way to keep a GenServer from becoming a bottleneck in your system even when callers need to wait for the result of a slow operation. Taking advantage of these tools requires only a relatively simple change to your code.