As of last week, my time at Mechanical Orchard has come to an end and it’s time to look for what’s next.

I really enjoyed my time there and learned a ton from the great people I got to work with. It’s amazing what you can accomplish when you put together a group of brilliant, empathetic people that know how to work well together and pull in the same direction.

I’m thankful that I had the opportunity to help solve some really hard problems.

What I’m looking for hasn’t changed much since the last time I was on the market. I’ve reproduced the list below. Not much below is a deal-breaker for a new role, but if I was able to write my own ticket, this is what it would look like.

• I do my best work when I’m making life better for other people. That can involve building systems that help people do their jobs more effectively, refactoring to make a codebase easier to work in, mentoring and teaching others, or bridging gaps to resolve conflicts between teams and individuals. Work like this is my “happy place”.

• I want to stay on the technical side. I don’t necessarily need to have my hands on the code all day every day, but I still enjoy building software. I’m not interested in moving into management.

• I’d prefer to have a technical leadership role. I believe everyone can lead from wherever they are (and I do that), but I’ve also found that having the title matters sometimes.

• I’d love to work somewhere that is all-in on Elixir and the BEAM (Erlang virtual machine), or is moving that direction. While I’m open to other technology stacks, I’ve really enjoyed working in Elixir and would love to be able to continue to take advantage of what it offers.

• I’m not focused on any single industry or business domain, but I want to work on things that are designed to serve people and not enslave them.

• I prefer to work remotely. I’m open to some travel for in-person meetings/work.

If you know of any opportunities that match these criteria, please get in touch. You can find more details about my work on my LinkedIn profile.

While I released the initial version of this project almost four years ago, I’ve never written about it or publicized it, other than a podcast appearance.

Background

While doing test-driven development in a number of languages, I’ve come to appreciate a workflow where my tests run automatically every time I save changes to my source code or tests. I even open-sourced an automatic test runner for Smalltalk when I was working in that language.

When I started working in Elixir, I quickly found and adopted Louis Pilfold’s mix test.watch. This was (and still is) a wonderful tool that largely gave me the workflow I’d become used to.

I later started working on some JavaScript projects where we used Jest.

One of Jest’s features is an interactive watch mode, where Jest stays running and re-runs tests on every change, much like mix test.watch. However, it also provides a number of commands that can interactively modify which tests will be run.

I found this feature of Jest very handy, and realized that I often wanted to change the arguments I was passing to mix test.watch. I’d then have to exit it and restart it with the new arguments. Or, I’d run mix test separately in another shell when I needed different arguments.

After I spent some time wishing quietly that someone would build a Jest-like interactive test runner based on mix test.watch, I realized that it probably wasn’t going to happen unless I did it.

So, back in late 2020/early 2021 I started experimenting with the idea. Once I had the basics working, I asked Louis if he’d like me to contribute this back to mix test.watch. He’s been busy building Gleam, so he suggested that I should keep our projects separate. I did just that, releasing mix test.interactive v1.0 in February, 2021.

The Basics

When using mix test.interactive, you’ll run it from a command line, optionally passing some arguments to it. It will stay running until you quit it (with the q command, Ctrl-D, or the ever-popular Ctrl-C Ctrl-C).

While running, it watches for changes to your source code and re-runs the tests when it detects a change. So far, this is the same as mix test.watch.

mix test.interative also allows you to enter commands that change what arguments will be passed to mix test.

Internally, mix test.interactive keeps track of a number of settings which are used to construct a command-line for mix test. These settings are updated every time an interactive-mode command is entered.

Examples

If my test suite is very fast, I’m usually fine with running all of my tests whenever anything changes. This is the default mode of mix test.interactive.

If I want even faster feedback, I might only want to run stale tests; that is, only tests affected by code/test changes made since the tests last passed. For this, I can use the s command to run only the stale tests, which runs mix test --stale under the hood.

In a larger application, I might only want to run tests matching a filename pattern. For example, if I’m working in my application’s Accounts context, I can use the p accounts command to run only those tests whose filenames contain the substring (pattern) accounts.

Perhaps I’ve just made a fundamental change to my application that caused a number of tests to fail. I’d like to keep re-running the failed tests until I can get them all passing again. For that, I can use the f command, which runs mix test --failed under the hood.

I can switch back to running all of the tests again with the a command.

Maybe I want to run a single test at a time, but change which test I’m focused on at any moment. I could add @tag :focus above the test I’m interested in and then use the o focus command, which will run mix test --only focus. I can then move the @tag :focus line to a different test to run that test instead.

If I’m making a number of changes and don’t want to run the tests while making them, I can turn watch mode off with the w command, make the changes, and then press Enter or turn watch mode back on (w command again) to run the tests.

Current Status

As I mentioned, I released the first version of mix test.interactive in February of 2021 and I’ve been enhancing it off and on since then. I use it on every Elixir project I work on. That doesn’t mean it’s perfect, but at this point, it’s got all of the features I can think to add to it.

It can manage the following mix test options via interactive-mode commands:

• --failed (f command)
• --exclude <tag> (x command)
• --include <tag> (i command)
• --max-failures <count> (m command)
• --only <tag> (o command)
• --repeat-until-failure <count> (r command) (Elixir 1.17.0 and later)
• --seed <seed> (d command)
• --stale (s command)
• --trace (t command)
• <filename> or <filename:line_no> arguments (p command)

I’m happy to consider additional feature requests or bug reports.

Wish List

There are two things I’d love to change in mix test.interactive, but haven’t been able to figure out how yet.

Immediate Response

Jest works by immediately processing keystrokes, so there is no need to press Enter after specifying a command.

As near as I can tell, Elixir (and, under the hood, Erlang) do not support immediate input mode out of the box (the equivalent of the C getc/getchar functions).

One suggestion I saw was to use an Elixir wrapper of the ncurses library, but I’m reluctant to add such a dependency because I’m not sure how well it would work on all platforms, including Windows.

If anyone knows how to do this, I’d love some pointers!

Tab-completion for p Command

I’d love to allow tab-completion of arguments to the p command.

From the documentation, I see that Erlang’s :io module has an expand_fun option that can be set that is supposed to be active when calling :io.get_line, but I’ve unable to make it work in mix test.interactive.

Again, any pointers on how to accomplish this would be most welcome!

Wrapping Up

If you think the mix test.interactive approach would work for you, I’d love it if you give this tool a try. It doesn’t interfere with anyone else’s workflow, so it is safe to add to any project for those that want to use it, while staying out of the way of anyone that doesn’t. Check it out on hex.pm!

As I posted last month, Sequin decided to transition to 100% in-person based in San Francisco. I’m not able to make that move, so I started looking for a new position and found it very quickly.

Mechanical Orchard works with large organizations to help modernize their older mainframe applications by replicating them into the cloud. I expect to work on some really interesting problems in domains I haven’t seen before, so I’ll get to learn a ton!

I’m looking forward to getting to know my new co-workers and seeing what we can build together.

Once again, I really appreciate all of you who reached out with opportunities and encouragement. I had a number of interesting positions to consider, and that’s largely thanks to your suggestions.

I originally published this article on the Sequin blog. Republishing here with permission.

GenServers are one of the core abstractions provided by the OTP library that both Erlang and Elixir share. A GenServer (generic server) is a separate process that maintains state and provides a way to run code asynchronously.

This article assumes some familiarity with GenServers and how to implement them. If you don’t have that background knowledge, start with the GenServer guide and documentation.

Conceptually, a GenServer process has a “mailbox” – a queue of messages that it needs to process. It processes one message fully before moving on to the next. If the handling of a message does something that takes a long time, the GenServer will not process any further messages until the long operation completes.

Every introductory tutorial about GenServers talks about the two main ways of interacting with a GenServer: call and cast. Briefly: call is used to send a message to a GenServer and wait for its response before moving on. cast is used to send a message to a GenServer without waiting for a reply.

One way to think about this is that, with call, both the caller and the GenServer are “blocked” while the message is being handled. The caller waits for the reply, and the GenServer is busy handing the call and not processing any other messages. With cast, only the GenServer is blocked. The caller continues on, while the GenServer stays busy handling the cast and not processing any other messages.

This might be OK if the GenServer is a “worker” that doesn’t need to respond to additional messages while it’s doing its job. But if the GenServer needs to handle messages from multiple clients, we don’t want it to become unresponsive while the work is happening. If it does, the GenServer can become a bottleneck in the system and cause significant performance problems.

In that case, we want the caller to wait for a reply from the GenServer, but we want the GenServer to be able to process additional messages while a long operation is running. For example, perhaps the GenServer manages a cache of values that take time to compute. If a request comes in for a value that’s not in the cache, the caller should wait for the computation to finish, but other callers should still be able to request already-cached values without having to wait for the unrelated computation to finish.

GenServer has a built-in mechanism to support this pattern. We can return a {:noreply, ...} tuple from its handle_call callback and then later use GenServer.reply to reply to the caller.

Let’s look at an example to see how this works.

Common usage with :reply

We’ll start with a simple GenServer that uses Process.sleep to simulate a slow operation.

defmodule ReplyExample.Server do
  use GenServer

  @timeout :timer.seconds(30)

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def do_the_thing(pid \\ __MODULE__, n) do
    GenServer.call(pid, {:do_the_thing, n}, @timeout)
  end

  @impl GenServer
  def init(_opts) do
    {:ok, %{}}
  end

  @impl GenServer
  def handle_call({:do_the_thing, n}, _from, state) do
    log("Sleeping for 2 seconds...")
    Process.sleep(2000)

    {:reply, n * 1000, state}
  end

  defp log(message) do
    now = DateTime.utc_now() |> DateTime.truncate(:second)
    IO.puts("#{now}: #{message}")
  end

  def test do
    {:ok, pid} = start_link()

    try do
      1..5
      |> Enum.map(fn n -> Task.async(fn -> do_the_thing(n) end) end)
      |> Task.await_many(@timeout)
      |> inspect()
      |> log()
    after
      GenServer.stop(pid)
    end
  end
end

This is a pretty simple GenServer. It has a single function, do_the_thing. When invoked, the caller will wait for a response which will come after a two second delay.

In IEx, I can use the test function to start up five tasks, each calling do_the_thing, and then wait for all of them to complete.

iex(7)> ReplyExample.Server.test()
2023-07-28 16:36:26Z: Sleeping for 2 seconds...
2023-07-28 16:36:28Z: Sleeping for 2 seconds...
2023-07-28 16:36:30Z: Sleeping for 2 seconds...
2023-07-28 16:36:32Z: Sleeping for 2 seconds...
2023-07-28 16:36:34Z: Sleeping for 2 seconds...
2023-07-28 16:36:36Z: [1000, 2000, 3000, 4000, 5000]
:ok

Notice the timestamps. Each request is processed only after the previous one has completed. In order for this to complete successfully, I’ve had to explicitly add a long enough timeout to both the GenServer.call and the Task.await_many calls. Otherwise, one or more of the GenServer.calls would have timed out after the default five seconds.

With this approach, you can imagine a busy GenServer becoming a bottleneck in the system with numerous processes queued up waiting for their turn.

Using :noreply and GenServer.reply

Let’s refactor this code to move the long-running operation into a separate task. We can then use :noreply and GenServer.reply to allow the GenServer to continue processing other messages in the mean time.

defmodule ReplyExample.Server do
  use GenServer

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def do_the_thing(pid \\ __MODULE__, n) do
    GenServer.call(pid, {:do_the_thing, n})
  end

  @impl GenServer
  def init(_opts) do
    {:ok, %{}}
  end

  @impl GenServer
  def handle_call({:do_the_thing, n}, from, state) do
    Task.async(fn ->
      log("Sleeping for 2 seconds...")
      Process.sleep(2000)

      GenServer.reply(from, n * 1000)
    end)

    {:noreply, state}
  end

  @impl GenServer
  def handle_info(_msg, state) do
    {:noreply, state}
  end

  defp log(message) do
    now = DateTime.utc_now() |> DateTime.truncate(:second)
    IO.puts("#{now}: #{message}")
  end

  def test do
    {:ok, pid} = start_link()

    try do
      1..5
      |> Enum.map(fn n -> Task.async(fn -> do_the_thing(n) end) end)
      |> Task.await_many()
      |> inspect()
      |> log()
    after
      GenServer.stop(pid)
    end
  end
end

The significant changes from the previous version are in the handle_call callback, plus the addition of a handle_info callback ¹.

In handle_call, the code is almost identical, but has been moved into an anonymous function that is passed to Task.async. There are other ways to do this, but Task.async is a great fit for this situation ².

After spawning the task, handle_call immediately returns {:noreply, state}. That allows the GenServer to move on to the next message in its mailbox while the caller stays blocked waiting for the reply.

At the end of the task, rather than returning a result, the anonymous function calls GenServer.reply, passing the address of the original caller (the from parameter) and the actual reply. GenServer.reply is what returns the result to the caller, unblocking it.

If you’ve written handle_call callbacks before, you have probably always ignored the from parameter because it’s normally not used. But in this case, from becomes very useful because we can use it to reply to the correct caller.

With these changes, we can again test our GenServer and see the results:

iex(4)> ReplyExample.Server.test()
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:30Z: Sleeping for 2 seconds...
2023-07-28 16:42:32Z: [1000, 2000, 3000, 4000, 5000]
:ok

Notice the timestamps again. This time, all of the Sleeping for... messages happen within the same second and the final response comes in after 2 seconds. This shows that the GenServer keeps processing messages even though all of the callers are blocked waiting for their replies! And the overall test runs much faster because we’ve been able to run the long operations concurrently.

Because the GenServer stays unblocked, we no longer need to add the long timeouts. Here, I’ve eliminated them entirely, falling back to the default of five seconds.

Wrapping Up

:noreply and GenServer.reply give you a way to keep a GenServer from becoming a bottleneck in your system even when callers need to wait for the result of a slow operation. Taking advantage of these tools requires only a relatively simple change to your code.

I originally published this article on the Sequin blog. Republishing here with permission.

One of Elixir’s best features is its REPL, IEx (interactive Elixir.) The iex command spawns a shell that allows you to run Elixir code interactively.

To exit IEx, you normally press Ctrl-C twice. If you’ve done this before, you may have noticed an interesting menu pop up between the Ctrl-Cs:

BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution

What’s that about? What do all of those options do?

This menu is known as the BREAK menu and it allows you to interact with the running Elixir session in various ways.

The BREAK menu is part of the Erlang Virtual Machine (commonly known as the BEAM), so these options are also available when running Erlang applications and shells.

The BREAK menu is available when:

• Running iex
• Running an application locally (e.g. mix phx.server or iex -S mix)
• Connecting IEx to a remote application (e.g. iex --sname local --remsh app@host)

In the latter case, all of the menu options in the BREAK menu apply only to your local IEx session and not to the remote application. In some cases this is a good thing, as you probably don’t want to accidentally shut down the running application.

Here’s a quick summary of these commands (and some additional undocumented commands). We’ll look at each of these in more detail in the following sections.

If you want to see how these are implemented, you can look at the C source code in the BEAM.

Simple Commands

A number of the BREAK menu options are simple commands for interacting with the running shell.

(a)bort

The a command does the same thing that hitting the second Ctrl-C does: it closes the current shell.

If you’re running IEx normally, it will close the session and any application running in it.

If you’re running your application directly (e.g. mix phx.server), it will shut down your application.

If you’re in a remote IEx session connected to a running application, it only closes your IEx session, but leaves the remote application alone.

(A)bort with dump

This is the same as (a)bort, but also produces a BEAM crash dump file (erl_crash.dump).

Looking through long pages of information in an interactive shell on a remote system can often be quite difficult. In that case, producing a crash dump file that you can examine on your local workstation might make things easier.

See the Erlang documentation for more information on interpreting and working with crash dump files.

(c)ontinue

Exits the BREAK menu and returns to your application or shell session.

This is the command to remember when you hit Ctrl-C by accident or change your mind.

(q)uit

The q command is not listed in the BREAK menu, but is supported. It is exactly the same as the a command.

(v)ersion

Prints the current BEAM version.

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
v
Erlang (BEAM) emulator version 12.3.2.6

This shows you the current Erlang Runtime System (ERTS) version and not the OTP version.

Informational Options

A number of the options in the BREAK menu will provide information about the running Erlang VM.

In general, it’s easier to see this information by running either Observer or observer_cli. That’s what we normally use. But if you don’t have those tools available, the BREAK menu is always available.

As I mentioned above, these commands all print information about your local IEx session even when you’ve opened a remote shell to a running application.

If you’re in a normal IEx session or are running your application locally, that’s perfect.

But for a remote shell to a running application, you’re probably more interested in seeing this information for that application instead. There doesn’t seem to be a way to make the BREAK menu display information for the remote application, so you’ll have to rely on using Observer or observer_cli instead.

Let’s look at the commands that are available.

(D)b-tables

Prints a list of ETS tables along with some information about each one.

This can help you find a particular table, see how it’s been configured, and see how big it is.

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
D
=ets:<0.42.0>
Slot: 4710072768
Table: logger
Name: logger
Buckets: 256
Chain Length Avg: 0.011719
Chain Length Max: 1
Chain Length Min: 0
Chain Length Std Dev: 0.107617
Chain Length Expected Std Dev: 0.108042
Fixed: false
9: [{{'$handler_config$','Elixir.Logger'},#{config=>#{counter=>{atomics,#Ref<0.2287420997.2752643074.39926>},sasl=>false,thresholds=>{20,500},translators=>[{'Elixir.Logger.Translator',translate}],truncate=>8096,utc_log=>false},filter_default=>log,filters=>[],formatter=>{logger_formatter,#{}},id=>'Elixir.Logger',level=>all,module=>'Elixir.Logger.Handler'}}]
84: [{'$primary_config$',#{filter_default=>log,filters=>[{process_level,{fun 'Elixir.Logger.Filter':process_level/2,[]}}],handlers=>['Elixir.Logger'],level=>debug,metadata=>#{}}}]
116: [{'$proxy_config$',#{burst_limit_enable=>false,burst_limit_max_count=>500,burst_limit_window_time=>1000,drop_mode_qlen=>1000,flush_qlen=>5000,overload_kill_enable=>false,overload_kill_mem_size=>3000000,overload_kill_qlen=>20000,overload_kill_restart_after=>5000,sync_mode_qlen=>500}}]
Objects: 3
Words: 7709
Type: set
Protection: protected
Compressed: false
Write Concurrency: true
Read Concurrency: true

...

(d)istribution

Prints information about the current node and other nodes it is connected to.

This command shows you information about all of the nodes in your distributed Elixir/Erlang application, which can be helpful when trying to debug connection problems.

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
d
=node:'nonode@nohost'
=no_distribution

(i)nfo

Prints information about memory usage and allocation.

This command produces a large volume of information about memory usage and can be helpful in figuring out where a memory leak might be helpful.

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
i
=memory
total: 34928033
processes: 5723312
processes_used: 5723312
system: 29204721
atom: 336049
atom_used: 331018
binary: 25520
code: 7495815
ets: 537960
=hash_table:atom_tab
size: 8192
used: 6389
objs: 12508
depth: 9
=index_table:atom_tab
size: 13312
limit: 1048576
entries: 12509
=hash_table:module_code
size: 128
used: 103
objs: 174
depth: 5
=index_table:module_code
size: 1024
limit: 65536
entries: 174

...

(l)oaded

Prints a list of loaded modules and their size.

This can be handy for checking that all of the code is available that you thought should be there.

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
l
Current code: 7737441
Old code: 0
erts_code_purger 13536
erl_init 2520
init 63188
prim_buffer 4984
prim_eval 1352
prim_inet 111589
prim_file 41674
zlib 20584
socket_registry 22408
prim_socket 51960
prim_net 10840
prim_zip 24120
erl_prim_loader 56864
erlang 97012
erts_internal 22920
erl_tracer 1680
erts_literal_area_collector 4928
erts_dirty_process_signal_handler 2352
atomics 3464
counters 4176
persistent_term 1456

...

p(o)rt info

This is not listed in the BREAK menu, but does appear in the source code. It prints a list of open ports (identical to what’s included at the end of the p command’s output).

According to the Erlang documentation, ports provide the basic mechanism for communication with the external world, from Erlang’s point of view. An open port will represent a connection from the Erlang VM to another process running in the host operating system.

Listing open ports can allow you to see what other processes your application is talking to.

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
o
=port:#Port<0.0>
State: CONNECTED
Slot: 0
Connected: <0.0.0>
Port controls forker process: forker
Input: 0
Output: 0
Queue: 0
=port:#Port<0.2>
State: CONNECTED|BINARY_IO
Slot: 16
Connected: <0.62.0>
Links: <0.62.0>
Port is UNIX fd not opened by emulator: 2/2
Input: 0
Output: 0
Queue: 0
=port:#Port<0.4>
State: CONNECTED|SOFT_EOF
Slot: 32
Connected: <0.64.0>
Links: <0.64.0>
Port controls linked-in driver: tty_sl -c -e
Input: 4
Output: 235
Queue: 0

(p)roc info

Prints a list of running processes along with a list of open ports.

Processes are a fundamental building blog of Elixir applications, so seeing what processes are running can give you a lot of insight into what’s going on.

Most Elixir/Erlang applications run a large number of processes, so this list will be quite long.

BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
p
...

=proc:<0.1.0>
State: Waiting
Name: erts_code_purger
Spawned as: erts_code_purger:start/0
Current call: erlang:hibernate/3
Spawned by: <0.0.0>
Message queue length: 0
Number of heap fragments: 0
Heap fragment data: 0
Reductions: 9
Stack+heap: 4
OldHeap: 0
Heap unused: 4
OldHeap unused: 0
BinVHeap: 0
OldBinVHeap: 0
BinVHeap unused: 46422
OldBinVHeap unused: 46422
Memory: 800
Stack dump:
Program counter: 0x00000001059a6008 (unknown function)
arity = 3
   erts_code_purger
   wait_for_request
   []

0x0000000118970220 Return addr 0x00000001059a6088 (<terminate process normally>)
Internal State: ACT_PRIO_HIGH | USR_PRIO_HIGH | PRQ_PRIO_HIGH | OFF_HEAP_MSGQ
=proc:<0.2.0>
State: Waiting
Spawned as: erts_literal_area_collector:start/0
Current call: erlang:hibernate/3
Spawned by: <0.0.0>
Message queue length: 0
Number of heap fragments: 0
Heap fragment data: 0
Reductions: 8
Stack+heap: 4
OldHeap: 0
Heap unused: 4
OldHeap unused: 0
BinVHeap: 0
OldBinVHeap: 0
BinVHeap unused: 46422
OldBinVHeap unused: 46422
Memory: 800
Stack dump:
Program counter: 0x00000001059a6008 (unknown function)
arity = 3
   erts_literal_area_collector
   start
   []

0x0000000107c3a3b8 Return addr 0x00000001059a6088 (<terminate process normally>)
Internal State: ACT_PRIO_HIGH | USR_PRIO_HIGH | PRQ_PRIO_HIGH | OFF_HEAP_MSGQ

...

Interactive Command

One of the commands is an interactive command, entering a new mode with sub-commands.

(k)ill

Prints information about one of the running processes in the system, then prompts for what to do with that process:

• (k)ill: Kill the process, then display the next process
• (n)ext: Move to the next process without killing the current one
• (r)eturn: Exit the sub-menu and return to the BREAK menu

When using the p command to list processes (described above), the list can be quite long and hard to navigate.

This command will show you only a single process at a time, allowing you to decide what to do with it.

If the process is running under a supervisor, killing it will cause the supervisor to respond according to its restart policy, often be recreating the process.

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)>
BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
k


Process Information

--------------------------------------------------
=proc:<0.108.0>
State: Waiting
Spawned as: erlang:apply/2
Spawned by: <0.78.0>
Message queue length: 0
Number of heap fragments: 0
Heap fragment data: 0
Link list: [{to,<0.66.0>,#Ref<0.2287420997.2752512002.40054>}]
Reductions: 4190
Stack+heap: 233
OldHeap: 0
Heap unused: 207
OldHeap unused: 0
BinVHeap: 1
OldBinVHeap: 0
BinVHeap unused: 46421
OldBinVHeap unused: 46422
Memory: 2696
Stack dump:
Program counter: 0x0000000105dac3e8 (io:execute_request/3 + 376)
arity = 0
y(0)     #Ref<0.2287420997.2752512002.40054>
y(1)     error
y(2)     {false,{get_line,unicode,<<"iex(1)> ">>}}
y(3)     <0.66.0>

0x0000000107d82470 Return addr 0x0000000105e20998 ('Elixir.IEx.Server':io_get/4 + 136)
y(0)     <0.108.0>
y(1)     <0.78.0>

0x0000000107d82488 Return addr 0x00000001059a6088 (<terminate process normally>)
Internal State: ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL
(k)ill (n)ext (r)eturn:

Wrapping Up

I wasn’t able to find a place where this information was documented, so I hope this gives you some insight into more of the power tools available in the BEAM.

When you’re running your application in an environment that doesn’t have access to more powerful tools like Observer or observer_cli, the BREAK menu can help you get to the bottom of a problem.

This morning, I accepted an offer for a new position. I’ll start first of the new year and will share more details at that time.

I had so many people reach out with really good and interesting opportunities. I wasn’t sure what to expect in the current tech job market, and I was pleasantly surprised by the choices I was presented with.

So thank you all for your referrals and suggestions! I really appreciate that you were willing to share those opportunities with me.

I wrote the first draft of this post while I was still at Sequin, and it uses examples from their domain. Publishing here with permission.

When implementing a new feature in a codebase, we usually have the whole problem loaded into our brains, and we’re intimately familiar with how all of the relevant pieces work and how they fit together. This is good, as it helps us get things done quickly.

But if we’re not careful, we can use this deep knowledge to accidentally embed implicit assumptions into the code we’re writing. When we come back to it later, we find it harder to understand, more difficult to reuse in a slightly different context, and more susceptible to needing to be refactored the next time we’re in the area.

One of the ways I try to avoid this mess is to play dumb. That is, I try to take off my omniscient creator of all the things hat put myself in the shoes of the code I’m writing. I’ll ask some questions from that perspective:

• What is my job?
• What should I know?
• What do I have no business knowing?

This technique applies at many levels: subsystems, individual services, modules/classes, and functions/methods.

Sandi Metz says it very well in her article, What Does OO Afford?

“OO asks you to blithely trust others to correctly do their bit. It wants you to strive for ignorance to protect yourself from the consequences of distant changes to other objects.”

Let’s look at some examples. The first two are both in Elixir which is a functional language, so I’ll be talking about modules and functions below. But the same principles apply in an object-oriented language where we work with classes, objects, and methods.

Module Level

In the Sequin codebase, a Job contains the state needed to replicate a collection of objects from an upstream API to a customer’s database (e.g. HubSpot contacts). Every time a job runs, it fetches a page of objects and writes them to the database. We then update the job with information needed to fetch the next page as well some other metadata.

We have two general kinds of jobs for each collection: backfill and delta. Backfill jobs are responsible for fetching historical data, and delta jobs are responsible for keeping up with the latest changes made upstream. When a customer starts replicating a new collection, we start with a backfill job. When it is done, we then kick off a delta job.

We keep the latest version of each job in memory for speed, but we also need to persist the jobs so that we don’t have to start over after a system restart. For this, we have a JobStore.

We were recently writing some new code related to job execution and we wrote something like this function:

defp note_job_completed(%State{} = state, table_id, row_count) do
  job =
    state
    |> get_job!(table_id)
    |> Jobs.note_job_completed!(state.resource_id, row_count, nil)

  if job.kind == :backfill do
    JobStore.replace_job(state.resource_id, job, [Map.put(job, :kind, :delta)])
  else
    JobStore.add(state.resource_id, job)
  end
end

defp get_job!(%State{} = state, table_id) do
  # Omitted; uses `JobStore.load()` to load all of the jobs for
  # `state.resource_id` and then finds the one for the collection
  # represented by table_id.
end

This code makes some critical assumptions about how the JobStore is implemented. If we ever chose to change our implementation, this code would break in interesting and confusing ways.

This seemed perfectly reasonable when we first wrote it. But because you don’t know the implementation of the JobStore, you probably noticed some problems right away. Our intimate knowledge of the implementation of the JobStore kept us from seeing these problems at first.

Here are some of the assumptions this code makes:

• The JobStore derives a key for each job and stores the job under that key.
• Calling Job.note_job_completed!/3 doesn’t change the job’s key, so when we call JobStore.replace_job with the updated job, the JobStore can still find the job in order to replace it.
• Calling JobStore.add/2 will actually replace any existing job that has the same key rather than adding a second copy of the job.

Once we recognized that we were making these assumptions, we immediately refactored the code. To do that, we put ourselves in the shoes of this code and answered the questions from above:

• What is my job? To update a job after it completes and persist the updated version back into the JobStore in place of the previous version.
• What should I know? There is a JobStore, and it exposes some public functions for interacting with it.
• What do I have no business knowing? How and where the JobStore stores jobs; what mechanisms it uses to find stored jobs; what parts of the Job are relevant to how jobs are stored.

Given those answers, we were able to temporarily forget what we know about the internals of the JobStore (aka play dumb) and make this code much cleaner and safer:

defp note_job_completed(%State{} = state, job, row_count) do
  next_job =
    job
    |> SyncJobs.note_job_completed!(state.resource_id, row_count,  nil)
    |> Map.put(:kind, :delta)

  JobStore.replace_job(state.resource_id, job, [next_job])
end

# get_job! as before

Here are the relevant changes:

• We now save the next version of the job into a new variable, next_job.
• We no longer call add_job at all; we always call replace_job.
• We call replace_job with the job we originally loaded from the store (job) and replace it with the updated job, next_job.

This will continue to work even if we make significant changes to the underlying implementation of the JobStore.

Function Level

When the shape of the data changes upstream (for example, a new custom property is added to a HubSpot contact), Sequin has to react to those changes by updating the structure of the database we’re replicating to. We call these changes “migrations”, as do many ORM libraries.

Sequin recently released an improvement that allows a replication to continue running while attempting to perform the migration. If the migration fails, it is scheduled to be retried later.

The new code that runs when a replication (sync)process starts up looked something like this:

defp initialize_sync(state) do
  {state, jobs} =
    case run_pending_migrations(state) do
      {:ok, state} ->
        # Do migration follow-up work
        # Create any new jobs required by the migrations
        {state, jobs}
      _error ->
        Process.send_after(self(), :restart_sync, @migration_retry_interval)
        jobs = load_jobs!(state)
        {state, jobs}
    end

  # Start running `jobs` in order continue replicating data
end

def handle_info(:restart_sync, state) do
  stop_pipeline(state, :migration_failed)
end

Again, there were some implicit assumptions built into this code because we knew too much when writing it:

• We know that we’re going to retry running migrations by stopping the replication pipeline and allowing it to restart, where it will try running migrations again.
• We know that this is the only place we’re sending the :restart_sync message, so we assume that the reason for stopping the pipeline is :migration_failed.

These assumptions may not always be true. We might move migrations completely away from the replication process and run them out-of-band. We might find other reasons to stop and restart the sync pipeline. Any of those changes would cause us to have to do more refactoring than if we had played dumb while writing this code.

Once we realized that we were making these assumptions, we again asked ourselves the questions, this time from two perspectives:

From the perspective of initialize_sync:

• What is my job? Schedule a retry of failed migrations.
• What should I know? I can send a message to myself after the retry interval to trigger the retry.
• What do I have no business knowing? How the retry is performed, in particular that it will be performed by restarting the entire replication process.

From the perspective of the handle_info callback:

• What is my job? Retry running migrations after a failure.
• What should I know? That we currently re-run migrations by stopping and restarting the replication pipeline.
• What do I have no business knowing? Who is sending me the message to retry migrations and why they’re sending it.

Given this set of answers, we might refactor to something like this:

defp initialize_sync(state) do
  {state, jobs} =
    case run_pending_migrations(state) do
      {:ok, state} ->
        # Do migration follow-up work
        # Create any new jobs required by the migrations
        {state, jobs}
      _error ->
        Process.send_after(self(), :migration_failed, @migration_retry_interval)
        jobs = load_jobs!(state)
        {state, jobs}
    end

  # Start running `jobs` in order continue replicating data
end

def handle_info(:migration_failed, state) do
  stop_pipeline(state, :migration_failed)
end

All we did is change the the message being sent from :restart_sync to :migration_failed, but with this change, we’re cleanly separating “what happened” (migrations failed) from “what do we do about it” (restart the pipeline to retry).

System Level

I don’t have permission to use the code for this example, but several years ago I was reviewing a pull request where a permission check was removed from part of the code. The authors’ comment was, “We removed the permission check because any user can change the topology, so there’s no need for the permission check.”

The original authors were removing what they saw as dead code. If permission checking was rare in the codebase, that would likely have been a good decision.

However, given that the application consistently checked permissions on various operations, they were inappropriately using their knowledge that this operation isn’t currently restricted, making it harder to change that decision in the future.

By asking the questions, they might have come to a different decision:

What is my job? Change the topology What should I know? We often restrict who can perform potentially dangerous operations; we have a standard mechanism for checking these permissions. What do I have no business knowing? Whether or not this particular operation is restricted.

Keeping the permission check in place made the code more resilient to future changes to permissions.

Wrapping Up

I’ve found the technique of playing dumb to be extremely valuable in writing code that is clean and easy to work with. It helps reduce inappropriate coupling and allows code to be more easily reused without first refactoring it.

Note that I’m not advocating doing extra work. Rather, I’m advocating doing the same amount of work in a better way. We’re still writing the same code, but we’re writing it in different places or using better names.

It’s hard to stop and remind myself that I know too much when I’m writing code. I try to make it a habit to put myself in the position of the code I’m writing and ask those questions:

• What’s my job?
• What should I know?
• What do I have no business knowing?

Every new job starts with a process of “onboarding” into that role, whether formal or informal. How do you know when you’re fully onboarded? When do you feel like you have a good handle on things?

When I started at Sequin, one of the first things I did was write down a list of Onboarding Milestones that I could use to gauge my progress. These milestones gave me a roadmap of where I needed to dig deeper. They gave me a sense of accomplishment to see how far I’d come since I started. They formed the basis of my 30/60/90-day and then quarterly goals.

My initial list was very focused on learning the codebase and its architecture. Here it is:

• [x] Can confidently review a non-trivial PR
• [x] Can ship customer-facing code to production
• [x] Can implement and test a non-trivial feature end-to-end
• [x] Can safely refactor a non-trivial part of the codebase
• [x] Can successfully complete a single day of on-call with no help
• [ ] Can successfully complete a full week of on-call with no help
• [x] Can safely make a small deployment change (e.g. adding a new env var)
• [ ] Can safely make a larger infrastructure change (e.g. CI, CD, deployment)
• [x] Can definitively answer questions about how something works from another team member
• [ ] Can update the CHANGELOG (We decided to make announcements elsewhere, so no longer relevant)
• [x] Can write a big picture/visionary memo about product direction
• [x] Can triage/debug a production issue with no help
• [x] Can recognize when seemingly innocent issues are representative of a larger issue

Over time, I add and remove milestones from the list. I might come to realize that there’s something important that I don’t know much about, so I’ll add milestones to tell me when I’ve learned about it. Something might change that means that a milestone is no longer relevant, so I’ll drop it.

As you can see, I hit most of these milestones in my time at Sequin, but I still had a couple to go.

On one of my on-call weeks, I handled everything on my own until the last half-day of my shift, when a really weird new issue arose that I couldn’t figure out. I asked for help, and the milestone remained incomplete. But that’s OK. I was much more capable at customer support than when I started and I even contributed substantial improvements to the support process. This outstanding milestone helped me to focus on learning what I still needed to learn in order to eventually complete it.

Upon reflection, I’d add more items that are less technically focused. Sequin is a very small startup where we all needed to wear a lot of hats. This list has nothing about some of these hats. Some examples of what I’d add:

• More cross-functional items related to marketing, customer calls/discovery sessions, sales, etc.
• Writing documentation
• Writing blog posts
• Other outside communication
• Leading projects
• Using our own software for something useful (“dog-fooding”)

You don’t need to change companies to get value from Onboarding Milestones. Use them any time you get promoted or take on new responsibilities.

I will definitely employ Onboarding Milestones again when I start my next role. I’ve found them extremely valuable.

Your list of Onboarding Milestones is going to be very different from mine. Here are some things to think about as you start to brainstorm:

• What kinds of things are you expected to accomplish in your role?
• What does success look like for your role?
• How senior is your role? The milestone list for someone new to this career will look very different than the one for a senior person with many years of experience.
• What do you value?
• What can you control and/or influence? What factors are outside of your control?

Here’s a partial list of categories to consider as you develop your Onboarding Milestones:

• Customer interaction
• Representing your company to the outside world
• Customer support
• Troubleshooting/debugging/incidents
• Peer interaction
• Mentoring/being mentored
• Contributions to code/documentation/PR reviews
• Comfort in the codebase: where to find stuff; where the tech debt lives; etc.
• Subject matter expertise
• Able to onboard someone else (aka onboarding buddy)
• Knowing which screw to turn
• Leaving the honeymoon phase: starting to see technical/process/leadership issues

What categories would you add to this list?

Learning is a continuous process. “Onboarded” is not a binary on/off state. You might complete your company’s formal onboarding process, but that has nothing to do with whether you’re really onboarded.

Onboarding is like the sunrise: the darkness slowly gives way to light, but you’re never really sure when that happened. There might be some “aha” moments, like when the sun first peeks over the horizon, but expect this to be a continuous process. You can use your Onboarding Milestones to help you measure your progress.

Thanks to Bucky, Gene, and Jeff for brainstorming the milestone categories with me, to Drew for the sunrise metaphor, and to Anthony for his thorough review of an earlier draft of this post.

Sequin recently decided that they need to transition from 100% remote to 100% in-person based in San Francisco. That’s just not a move I can make at this time.

While I’m disappointed to no longer be working there, I’m really grateful for the time I had, and the great team I got to work with. We built some cool stuff, and I was able to be involved in more facets of the business than I ever have before. All in all, it was was a wonderful learning experience.

Last year when I was in a similar position, I posted a list of what I was looking for. Sequin ticked most of those boxes. I find that my priorities remain largely the same, so I’ll reproduce that list here with an addition and some minor edits:

• I do my best work when I’m making life better for other people. That can involve building systems that help people do their jobs more effectively, refactoring to make a codebase easier to work in, mentoring and teaching others, or bridging gaps to resolve conflicts between teams and individuals. Work like this is my “happy place”.

• I want to stay on the technical side. I don’t necessarily need to have my hands on the code all day every day, but I still enjoy programming. I’m not interested in moving into management.

• I’d prefer to have a technical leadership role. I believe everyone can lead from wherever they are (and I do that), but I’ve also found that having the title matters sometimes.

• I’d love to work somewhere that is all-in on Elixir and the BEAM (Erlang virtual machine), or is moving that direction. While I’m open to other technology stacks, I’ve been really enjoying working in Elixir and would love to be able to continue to take advantage of what it offers.

• I’m not focused on any single industry or business domain, but I want to work on things that are designed to serve people and not enslave them.

• I prefer to work remotely. I’m open to some light travel for in-person meetings/work.

If you know of any opportunities that match these criteria, please get in touch. You can find more details about my work on my LinkedIn profile.

The answer is :noreply and GenServer.reply. See my latest post on the Sequin blog for more details and an example. I have also republished here on my own blog.

The result of my exploration is now up on the Sequin blog and has been republished here on my own blog. Check it out!

As I posted last month, I was laid off from InfluxData and was looking for my next opportunity. I’m happy to say that I found it at Sequin!

Sequin is a small startup that allows integrating external APIs with your database. I’ll be working mostly in Elixir (as I wanted) and I’m really excited about the tools we’re going to build and the people we’ll be able to serve.

I can’t wait to get started and to see what we can accomplish in the coming months and years!

I am beyond grateful to many of you who contacted me with potential opportunities, said nice things to and about me, encouraged me, and served as references. Because of you, I had a lot of great options to consider. Words can’t express how much I appreciate you!

As Dragosh Mocrii discovered, IntelliSense (aka ElixirSense) and the “jump to definition” features won’t work for functions and macros imported with use.

As of this writing, I’m using Erlang 25.1.2 and Elixir 1.14.2, but ElixirLS seems to be using Erlang 22.x and Elixir 1.12.x.

The solution is to recompile ElixirLS with the versions of Elixir and Erlang that your project uses.

Dragosh gives a set of instructions in his blog post. You can also refer to the vscode-elixir-ls README.

I’ve distilled these two sources into a set of instructions that works well for me (at least on MacOS) and posted them to a Gist, which I’ve also included below to save you a click.

# First time only
git clone --recursive git@github.com:elixir-lsp/vscode-elixir-ls.git
cd vscode-elixir-lsp

# Second and subsequent times
cd vscode-elixir-lsp
git pull && git submodule update --recursive

# Always
npm install
cd elixir-ls
asdf local erlang {desired version}
asdf local elixir {desired version}
asdf install # should do nothing, but just to be safe
cd ..
rm *.vsix
npx vsce package
code --install-extension *.vsix --force

I really enjoyed my three years there. I got to work with great people, I learned a ton, and was able to do a lot of work that I’m proud of.

I now find myself looking for what’s next.

I’m open to a lot of possibilities, but here are the things I’m prioritizing:

• I do my best work when I’m making life better for other people. That can involve building systems that help people do their jobs more effectively, refactoring to make a codebase easier to work in, mentoring and teaching others, or bridging gaps to resolve conflicts between teams and individuals. Work like this is my “happy place”.

• I want to stay on the technical side. I don’t necessarily need to have my hands on the code all day every day, but I still enjoy programming. I’m not interested in moving into management.

• I’d prefer to have a technical leadership role. I believe everyone can lead from wherever they are (and I do that), but I’ve also found that having the title matters sometimes.

• I’d love to work somewhere that is all-in on Elixir and the BEAM (Erlang virtual machine), or is moving that direction. While I’m open to other technology stacks, I’ve been really enjoying working in Elixir and would love to be able to take more advantage of what it offers than I’ve been able to so far.

• I’m not focused on any single industry or business domain, but I want to work on things that are designed to serve people and not enslave them.

If you know of any opportunities that match these criteria, please get in touch. You can find more details about my work on my LinkedIn profile.

Check out Episode 147!

The main topic we discussed was the Elixir ConfigCat SDK I worked on last year.

We also talked about mix_test_interactive, a tool I released earlier this year. mix_test_interactive is adapted from Louis Pilfold’s wonderful mix_test_watch, but adds an interactive element that allows you to control which tests you want to run.

We had a fun side discussion on the use of emoji in log messages and for git commits. I mentioned gitmoji, which provides a catalog of emoji to use for various kinds of code changes.

I enjoyed my time on the podcast. Thanks so much to the hosts for inviting me!

I wrote a blog post for InfluxData that was just published on The New Stack.

The SDK itself is on hex.pm.

Zeal

I’ve loved working at Zeal. The people are great, the projects are interesting, and I’ve learned a ton. They’re hiring and I wouldn’t hesitate to recommend working there if it seems like a good fit for you.

As I transitioned from the industrial automation world to the web world, I intentionally chose to work for a consulting company. I felt that it would provide the opportunity to experience many different projects, teams, and environments and give me the breadth to get a sense of which parts of web development I enjoyed most.

Zeal delivered on that goal in a big way. I worked on a dozen client projects and shipped production code in at least eight different languages during that time, several of which I didn’t really know when I started there.

Growth

Last year, I heard an interview with Whitney Johnson on the EntreLeadership podcast where she talked about the idea of “S-curves” as it relates to personal growth and learning.

The basic idea is that when you’re first learning something, you don’t know much; you’re at the bottom of the “S”. As you learn, you start moving up the curve of the “S”. This is where you grow the most. As you reach the top of the “S”, it’s time to use what you’ve learned to help bring others along the curve as well. But if you stay there too long, you start getting bored because there’s not as much learning happening any more. Once you reach that point, you need to start looking for the next “S” curve to move to so you can keep growing and challenging yourself.

This metaphor describes my last five years really well. At Zeal, I came in knowing something about web development, but not a lot. I went through a period of rapid learning and growth as I climbed the “S” curve. And now I feel like I’ve been near the top of the “S” for the last year or so. I’ve decided to make a move so that I can jump to the bottom of the next “S”.

Next?

As I thought about what I wanted to do next, I had a few guiding thoughts:

• I want to stay technical. While I enjoy working with people, I’m not currently interested in moving into the people-management side of things.

• I want to be able to get above the day-to-day process of developing features sometimes. My most fulfilling work has been when I’ve been able to work on the bigger picture: system architecture, design and structure of the codebase, and integrating the various pieces of a complex system together.

• I really enjoy teaching and mentoring, especially as it relates to technical skills.

• I want the opportunity to continue growing and to expand the impact I’m able to have on a team and a company.

• I want to get back to working at a product company again, but this time in the web world. I like the feedback cycle that results from living in the same codebase for a long time. I get to see the payoff from the investments I make in code quality. I also get to learn from the mistakes I make and shortcuts I take.

InfluxData

Enter InfluxData.

Believe it or not, I heard about the opportunity when I received a random e-mail from a recruiter on LinkedIn about it. After looking things over, it seemed interesting enough to pursue, so I responded.

Through the interview process, I met my manager and future teammates and became more and more excited about the opportunity and the people.

It’s a growing company (and also hiring!) with lots of interesting challenges to address and opportunity for growth on a technical career path.

It looks like it’s going to tick all of the boxes I listed above and I can’t wait to get started!

When first working with floating point numbers, many programmers are surprised when two numbers that should be equal are not. For example, in binary, 0.1 + 0.2 does not exactly equal 0.3 because 0.1 does not have a finite represenation in the binary floating point representation that our computers use. This is similar to how 1/3 does not have a finite representation in decimal.

The solution most people come up with is to check whether the absolute difference between the expected and actual values is small enough. If you know that the numbers you’ll be working with are in a certain range, this can be fine. But if you’re working with either very large or very small numbers, this may not work the way you’d expect.

A solution to this problem is to compare the relative difference between the two numbers. This is a bit trickier, though, as you have to worry about what happens when the numbers are close to 0.

Because of these complexities, most unit testing frameworks provide a way of checking that a floating point result is close enough to the expected value.

Here are some examples:

assert_in_delta(0.3, 0.1 + 0.2, 0.0005)
assert_in_epsilon(0.3, 0.1 + 0.2, 0.0001)  # relative difference

expect(0.1 + 0.2).to be_within(0.0005).of(0.3)

assertEquals(0.3, 0.1 + 0.2, 0.0005)

expect(0.1 + 0.2).to.be.closeTo(0.3, 0.0005)

expect(0.1 + 0.2).toBeCloseTo(0.3, 3)  // number of digits, not absolute value

Hopefully you get the idea. This is pretty much what I expect to find when I encounter a new testing framework.

However, I recently worked on a Python project where I used pytest as the test framework.

pytest uses Python’s built-in assert statement in its tests and is able to produce very nice failure messages from that. However, assert by itself doesn’t provide any niceties for comparing floating point numbers.

To solve this problem, pytest includes an approx function that creates an object that knows how to compare itself to a floating point value appropriately.

Here are some examples of how to use it:

assert 0.1 + 0.2 == approx(0.3)
 
assert 0.1 + 0.2 == approx(0.3, rel=1e-3)
 
assert 0.1 + 0.2 == approx(0.3, abs=1e-3)

As you can hopefully see, approx provides a lot of flexibility with a very simple API. If you provide a rel value, it does relative comparison (the default). If you instead provide an abs value, it does absolute comparison. It can even handle comparing arrays or dictionaries containing numbers.

After so many years of using different testing frameworks in different languages, I was genuinely surprised to run into such a different approach to the problem. I think this is a really nice, effective solution and it’s got me thinking about how it might be applied to other languages and testing frameworks.

Background

As a recap, here are my original posts:

• Encapsulating the Redux State Tree
• Redux Reducer/Selector Asymmetry
• Modular Reducers and Selectors

Where Did We Leave Off?

To summarize the earlier posts very briefly, we’ve defined selectors within our Redux modules that work on the module’s local section of the state tree. We use these selectors in each module’s reducer and its tests.

We also need versions of these selectors that work on the global state tree. We’ll use these in containers and thunk action creators both within the module, and potentially in other modules.

In order to get from a local selector to a global selector, we need to know how to get from the root of the state tree to the module’s local section of the tree. The main app reducer knows how to do that. So the main part of the app could be the one to apply that knowledge to create globalized selectors. However, since we need access to those globalized selectors within the module, we end up with a circular dependency:

In Modular Reducers and Selectors, we talked about the “textbook” ways of solving the cyclic dependency problem:

• Apply the Dependency Inversion Principle by extracting something within the module that both the app and the rest of the module depend on; or

• Create a new module that both the app and the module depend on.

We looked at Jack Hsu’s approach, which is to define a constant within the module. This constant is used to globalize the selectors and is also used by the main app reducer to specify where in the state tree the module’s reducer is mounted. This a form of the Dependency Inversion Principle approach.

We also looked at living with a bit of duplication. The app reducer specifies where to mount the module’s reducer in the state tree, and we duplicate that knowledge within the module in order to globalize the selectors. Up until now, that’s been my approach, though I still like Jack’s idea as well.

Code-Splitting

In all of this discussion about modular Redux applications, I didn’t understand one of the biggest advantages of a modular Redux architecture: code-splitting. In a large application, you often want to load only the parts of your code that are needed for a particular section of the application. A good modular structure can make it much easier to define the boundaries of the split.

But how do you code-split a Redux application when there is a single store?

This is where Nicholas Gallagher comes in. His post, Redux modules and code-splitting, tackles this question and comes up with a really nice approach involving a reducer registry.

As a side-effect, this reducer registry is an elegant solution to the circular dependency problem. It takes Jack’s idea of extracting a constant within the module and combines it with the “create a new module that both the app and the module depend on” approach.

What Does It Look Like?

I won’t repeat all of Nicholas’ code here, but the basic idea is that there is no more top-level app reducer that pulls in all of the module reducers. Instead, there is a ReducerRegistry in a separate module.

In the main part of the application, we create the Redux store by combining all of the reducers that the ReducerRegistry knows about.

Each module in turn registers itself with the ReducerRegistry using its own moduleName constant. As a side benefit, this same moduleName constant can be used in other places, such as when creating action type constants for the module.

The dependency structure ends up looking like this:

I tested this approach in a sample application to see what it would look like. It worked great and I’m happy with how it turned out.

I did run into one little gotcha that’s worth noting.

Before, I’d export my module reducer from the module’s index.js file so that it can be imported into the main app reducer:

import reducer from "./reducer";
 
export reducer;

There’s now no need to export the reducer, but we do need to import it somewhere so that our module bundler (e.g. Webpack) knows about it and includes it in the appropriate bundle.

There are a couple of ways to do this.

The reducer can register itself with the ReducerRegistry:

import { combineReducers } from "redux";
import { createReducer } from "zeal-redux-utils";
 
import reducerRegistry from "modules/reducerRegistry";
 
import moduleName from "./constants/moduleName";
 
const reducer = createReducer(0, {
  // ...action handlers go here...
});
 
reducerRegistry.register(moduleName, reducer);
 
export default reducer;

Then, the module’s index.js file can import the reducer without re-exporting it:

import "./reducer";

Another option is to move the registration of the reducer into the module’s index.js file:

import { combineReducers } from "redux";
import { createReducer } from "zeal-redux-utils";
 
export default createReducer(0, {
  // ...action handlers go here...
});

import reducerRegistry from "modules/reducerRegistry";
 
import moduleName from "./constants/moduleName";
import reducer from "./reducer";
 
reducerRegistry.register(moduleName, reducer);

Either way works, but I lean toward the latter approach because it allows the reducer to stand on its own and only code outside of itself determines how to hook it into the main application reducer.

Note that I’ve used createReducer from zeal-redux-utils here, but that’s an implementation detail. You can define your reducers however you want.

While there are often better solutions, Smalltalk provides several methods for introspecting the class of an object or the inheritance hierarchy of a class. These methods include #isKindOf: and #isMemberOf: for instances and #includesBehavior:, #inheritsFrom:, and #isDirectSubclassOf: for classes.

Up until now, the DoubleAgents library didn’t do anything special with these methods. That would have been OK, but because of some internal implementation details it didn’t even allow mocking or stubbing #isKindOf: and #isMemberOf: at all.

I ran into a situation where it made sense to use isKindOf:. I wanted to be able to test the code using DoubleAgents, but found I was unable to.

I decided to make some changes to DoubleAgents to support these methods.

DoubleAgents has two kinds of test doubles:

• Standalone doubles completely replace a collaborator.

• In-place doubles (also known as partial doubles) are “normal” objects that have one or more methods replaced by a mock or a stub for a given test. In-place doubles can be instance-side doubles, class-side doubles, or “any instance” doubles.

With the latest changes to DoubleAgents:

• All in-place doubles now respond properly to #isKindOf: and #isMemberOf: by default. Since they are partial doubles, they should act like the original object or class they are doubling unless told to do otherwise by mocking or stubbing a method.

• In-place class doubles also respond properly to #includesBehavior:, #inheritsFrom:, and #isDirectSubclassOf:.

• Standalone doubles DO NOT pretend to be #isKindOf: or #isMemberOf: the class they are doubling. Standalone doubles only respond to messages that are explicitly mocked or stubbed.

• It is now possible for any kind of double (in-place or standalone) to stub or mock these introspection messages. Previously, it was not possible to mock or stub #isKindOf: and #isMemberOf:.

I have published these changes as version 22 in the Cincom Public Store Repository and put a snapshot of the current version on GitHub.

The latest version has also been included as a contributed package in the about-to-be-released VisualWorks Smalltalk 8.3.1.