I recently finished building my first Elixir application, a command-line application for managing invoices for my side work.

InvoiceTracker is up on GitHub, but I don’t expect it to be terribly useful for anyone but myself.

I learned a lot while working on the project, which is the point of a learning project.

I’ll walk through some of the interesting milestones along the way and then share some thoughts and opinions about Elixir.

The Process

This is a retrospective journey of the project, talking about my approach, some of the problems I had to solve, and some of the tools and libraries I found to make my job easier.

Start With Tooling

I started out by trying to get some basic infrastructure in place. It’s always tempting to jump right in and start coding, and maybe that’s a better approach. But I find that having some good tooling in place early pays off pretty quickly.

In JavaScript, I’ve gotten used to having a good linter, testing with watch mode, and CI, so I started there.

After generating the initial project skeleton with mix new, I immediately added:

  • credo for linting

  • asdf for version management. As a bonus, asdf supports numerous languages, so I now use it for Ruby and JavaScript as well.

  • CircleCI for automated builds. Most open source projects seem to use Travis, but I use CircleCI at work all the time, so I’m more familiar with it.

  • mix_test.watch for automatically running my tests whenever I save changes.

With that, I had a pretty fluid development flow running and I could start adding features.

Command-Line Handling

Since I was writing a command-line application, I started by looking for a library for this. After considering a few options, I settled on ex_cli and it’s been a good choice.

It uses macros for its DSL (domain-specific language), and my limited experience with them caused me to introduce a bug along the way. Overall, though, it’s been a really nice library to work with. It still needs support for showing help for sub-commands; I haven’t made the time to submit a PR for that but for a personal-use application, it’s not a terribly high priority for me.

End-to-end Testing

I wanted to have good end-to-end testing in place from the start, so the first thing I did was write a feature test that compiles my application to an escript and then uses System.cmd to run it and capture the output. That way, I can be pretty confident that everything is working correctly.

I adopted briefly to help with temporary file management for the feature tests.

Data Storage

The first thing I needed to figure out was how I was going to store data. After doing some research, I learned about Erlang’s ETS (Erlang Term Storage) and related DETS (Disk-based Erlang Term Storage). This looked promising so I went with it, using the in-memory ETS for unit tests, and the disk-based DETS for real usage and the feature tests.

I added a Repo module to manage the storage for me, abstracting the implementation away from the rest of the code. That way, I can change my storage approach in the future without having to change anything else. Repo uses an Agent to manage storage in a separate process.

While ETS and DETS have essentially the same API, they’re not truly polymorphic with each other. It took me some time puzzling over obscure error messages to realize that I can’t use DETS functions on an ETS table and vice versa. After a few iterations, I ended up creating a couple of factory functions for starting up either ETS or DETS as appropriate. It passes the correct module (:ets or :dets) along with the actual table into the agent so that the rest of the code is identical.

The other advantage of these factory functions is that they are responsible for creating the ETS/DETS table. That way, the tables can be kept private to the Repo Agent’s process and not exposed to the rest of the application.

Mocking and Stubbing

As I’ve written about before, I tend to use mocks and stubs quite a bit in my tests. I’ve been finding that I need them less in functional code, but there are still places where they’re handy.

I started out with test doubles for the Repo module using the mock library, but quickly found that those weren’t that helpful. I switched to using the real Repo module with an in-memory ETS store instead.

Late in development, I reintroduced mocking in order to test the way that the InvoiceTracker module converts an invoice date into a date range to use when requesting a time summary from the TimeTracker module.

It didn’t seem worth it to extract a separate module for that calculation, so I mocked out TimeTracker to check that it was called with the correct dates.

Type Checking

I started out writing type specs for everything (mostly because credo’s default configuration complained if I didn’t). I messed with Dialyxir a bit, but quickly decided it was mostly getting in my way and not helping.

Towards the end of the project, I decided to give it another try and added type specs back into the code. I’ll have to decide if/when I do more development on this project whether I find the specs helpful.

It’s definitely important to ensure that the results of the first run of Dialyxir are cached properly on CI, or builds start taking a really long time.

I also ran into some issues with warnings around protocol implementations. I ended up adding a dialyzer.ignore-warnings file to silence them.

Table Formatting

A lot of the output of this application looks best as a nicely-formatted table. I found table_rex which does (almost) everything I need.

The one thing table_rex doesn’t support is a separator line before a footer in the table. I ended up adding this myself by following an approach suggested by Darian Moody.

Helpful Defaulting

I wanted the application to be smart about defaulting options and parameters, especially dates. As much as possible, I wanted to be able to specify only the bare minimum information and have the app figure out the rest.

I’ve succeeded with that. My workflow involves running invoice g -s to generate and save a new invoice, invoice p to record a payment, and invoice s to generate a status report. I don’t need to specify dates, invoice numbers, or payment amounts anywhere.

In order to implement proper defaulting for dates, I had to be able to do some date math. I found Timex for this, and it has everything I needed. It’s a really nice library.

Timex has one issue when using it from an escript, as in my case. The latest versions of the tzdata library (which Timex depends on) attempt to download and cache timezone data. But in an escript, this caching is not allowed. I used the workaround outlined in that issue, downgrading tzdata to the last version before it started auto-updating.

Calling External APIs

One of the features I added was the ability to retrieve time tracking records from my time-tracking app (Toggl) and use that to produce the data I need to create an invoice as well as a summary of what I worked on for that time period.

I looked at several options and settled on Tesla. It’s not the most popular Elixir library for this purpose, but I feel like it should be. I like its API a lot and found it easy to use for my purposes.

I haven’t yet figured out the best way to test the API calls, especially via my end-to-end tests. I’m aware of exvcr for the unit tests, but I think I’m going to have to run a fake server for the end-to-end tests since they run the application in a separate system process.

In the end, I kept the API call layer very thin and extracted the processing of responses into its own module where it was easier to test. This isn’t perfect, and it’s something I’ll probably spend more time on at some point.

Rounding

The biggest problem I needed to solve for this application was the handling of rounding. I bill and report in six minute increments (tenths of an hour), and I round times to the nearest tenth rather than always rounding up.

For the overall invoice amount, the rounded time and total invoice amount are always correct, but I also report a breakdown by project and produce an activity report where each project is broken down into a number of entries.

With rounding at these lower levels, it’s often the case that the rounded times, when added up, don’t match the total time at the next highest level. For example, if I spend 15 minutes on one task and 16 minutes on another task, the total is 31 minutes, or 0.5 hours when rounded. However, if I round the two individual times, they both come out to 0.3 hours and added together, they total 0.6 hours. Now the report looks incorrect, even though the unrounded numbers are right.

I ended up developing an algorithm that finds the entries that were closest to rounding up or closest to rounding down, depending on the direction of the adjustment. I then round those entries up or down to make the sums match the totals I was reporting.

There was nothing Elixir-specific in this part, but it was an interesting problem to solve.

Giving Back

Along the way, I ran into a few issues or missing features with the libraries I was using. I took some time out from developing my application to contribute back to these projects.

  • I found that Tesla, with the default httpc adapter, wasn’t automatically starting the Erlang :ssl app. As a result, my API calls were failing when I ran the application. Interestingly, if I ran the app in iex, everything worked fine. Apparently, something in iex was starting the :ssl app for me. I submitted a pull request to have Tesla start the application. It’s been merged and released in v0.7.1.

  • I wanted the ability to define short aliases for my ex_cli commands (for example p or pay for payment), but that feature wasn’t supported. After checking in with the maintainer, I implemented support for aliases in ex_cli.

  • I also found that the help command wasn’t formatting the command list correctly, so I also fixed that. These two changes have been merged and released as v0.1.2.

  • When adding typespecs, I found that some of the functions in Timex had incorrect typespecs themselves, which was causing dialyxir to fail on my code. I submitted a pull request to fix the type specs and its been merged and released as v3.1.16.

Opinions

There’s a lot about Elixir that I enjoyed, but I’m not yet sure I liked it enough to make it my go-to language of choice. I plan to next work on a web-based project that uses Elixir and Phoenix on the back-end and then see what I think.

Overall, I’m pretty happy with the code I ended up with. I’m open to comments and suggestions for improving it and making it more Elixir-like.

Here are a few of the thoughts I have at this stage of the game.

Functions and Pipelines

I absolutely love the pipe operator (|>). It’s a really clean, concise way to express data transformations.

With my Ramda experience, I still find it strange that it’s the first parameter that is implicit in Elixir rather than the last, but I’m getting used to that.

I use compose (and/or pipe) in Ramda quite a bit, so I was already used to thinking the way the pipe operator requires. I find the pipe operator syntax cleaner than using compose, which is nice.

What I really miss, though is Ramda’s automatically-curried functions and support for partial application. There are ways to achieve these things in Elixir, but they’re not built in, and they’re not the way most people write Elixir.

Managing State

I feel like I’m still struggling a bit with how to manage state in an Elixir application and how to divide responsibilities into modules. I’m sure this will come with more time and effort, but there were definitely times when I wasn’t sure about how I should structure things.

Ecosystem

I was able to find libraries that helped me get my application working, but it doesn’t seem like there aren’t as many choices as available as there are in more mature languages like Ruby and JavaScript.

A lot of the available choices are Erlang libraries, which makes sense because Erlang has been around longer. But that means that I had to get comfortable reading documentation and examples written in Erlang, and then figure out how to translate that into Elixir. I’m not good enough at this part, yet, so more learning for me!

CircleCI doesn’t have first-class support for Elixir yet, so I found I had to mostly configure that myself. I haven’t tried CircleCI 2.0 yet but at first glance, it looks like it will be a better fit. I also haven’t looked at what TravisCI offers in this area.

Documentation and Type-Checking

Documentation is definitely a first-class citizen in the Elixir world. This is really nice when learning how to use a new library, but it’s still easy for a library author to do the bare minimum necessary on documentation.

My biggest problem is that I really don’t like having all of those comments in my source files when I’m trying to write code. They get in the way and make it harder to find what I’m looking for.

I feel mostly the same way about type specs. I’m still not convinced of the value they provide. I see advantages and disadvantages of both sides of the type-checking debate, but I still tend to fall more on the dynamic-language side of that spectrum.

Formatting Code

I’m still learning what best practices are for formatting Elixir code. I expect that will come as I read more of other people’s code.

What I really miss is an opinionated auto-formatter like Prettier (for JavaScript). I am amazed at how quickly I get annoyed with languages that don’t have auto-formatting like that.

There has been some discussion about something like go fmt for Elixir, but there seems to be a fair bit of opposition to the idea.

Conclusion

I learned a lot while working on this project, but I’m not yet sold on Elixir as my language of choice. I want to do another project with it before I decide that for sure.