profile picture

Use Justfiles to Work with MongoDB

May 21, 2021 - development mongodb beginner

I have started to write Justfiles for all my new projects. There's a good chance you haven't heard of Just or Justfiles. They're very similar to Make and Makefiles. Whereas Make is designed to build artifacts from source files, and combine them in potentially complex ways, Just is "just" a task runner.

Just can be installed with a bunch of different package managers, or you can just download a prebuilt binary.

A short Justfile, containing a single recipe to import some data into a MongoDB cluster looks like this:

import:
    mongoimport $MDB_URI --collection recipes recipe_data.json

The file is literally called Justfile, with no file extension. You would usually put this file at the root of your project directory. This will all seem familiar, if you've ever worked with Makefiles!

This specifies a recipe called "import". When I ask Just to run the import target, it will run the mongoimport command I've written on the line below. The environment variable $MDB_URI specifies the cluster (and database) that the data stored in recipe_data.json will be loaded into. As long as I've set the $MDB_URI environment variable correctly, then if I go to the directory that holds the Justfile and run the following, it will run the mongoimport command and load the data into my MongoDB database.

just import

Back to that $MDB_URI variable. Just has this great feature – it automatically loads environment variables from a .env file you put in the same directory. So if I create a file called .env containing the following, then when I run the just command, I'll be able to refer to the $MDB_URI variable, like I did in the Justfile above.

# .env

MDB_URI=mongodb+srv://username:password@timeseries.abcde.mongodb.com/cocktails?retryWrites=true&w=majority

The idea in this case is that local configuration, like the connection string for connecting to a MongoDB cluster, is stored in the .env file. The .env file is not committed to revision control, whereas the Justfile can be committed and then reused by other team members.

I'll often have a handful of targets for setting up my database in any MongoDB project that I'm working on:

# Justfile

# Recreate the database with sample_data.
all: init import

# Drop all the collections, and recreate them with associated indexes.
init:
    mongosh $MDB_URI init_database.js

# Import recipe and review data into the cluster at $MDB_URI.
import:
    # Load recipe_data.json into the recipes collection:
    mongoimport $MDB_URI --collection recipes recipe_data.json
    # Load review_data.json into the reviews collection:
    mongoimport $MDB_URI --collection reviews review_data.json

Let's break this down:

So with this combination of Justfile and .env files, I've gained two things:

  1. I've got a quick and easy way to run commands related to my project.
  2. I've got documented commands related to my project, for easy reference.

Speaking of documentation, now that I added some comments to my Justfile, if I run just --list I get the following output:

❯ just --list
Available recipes:
    all    # Recreate the database with sample_data.
    import # Import recipe and review data into the cluster at $MDB_URI.
    init   # Drop all the collections, and recreate them with associated indexes.

It prints out each target, along with the comment above each one. Self-documenting code! I ❤ it.

Why Don't You Use Make?

I used to use Makefiles for this kind of thing in the past. I continued to use Make even after I became aware of Just; I couldn't quite see the benefit of using a less ubiquitous tool. But since I switched I've become a total convert, mostly for the following reasons:

Some Useful Snippets

Here are a few useful targets that I've found myself using across different projects:

Connect To Your Database

I have the following short recipe in every Justfile! It runs mongosh against the MDB_URI that's stored in the local .env configuration, so you'll get an interactive connection to your database you can use to run ad-hoc commands against your data.

# Connect to the configured database
connect:
    mongosh $MDB_URI

Initialize Your Database For Development

I usually have a JavaScript file that can be run against a new database, with mongosh, to initialize collections and indexes in a reproduceable way. An example init script looks like this:

// init.js

// Drop the collection if it already exists:
db.scores.drop();
// Create a unique index on "username":
db.scores.createIndex({ "username": 1 }, { unique: true });

// Drop the collection if it already exists:
db.stock_exchange_data.drop();
// Time series collections need to be created ahead-of-time.
// Have you tried MongoDB's new time series collections? They're awesome!
db.createCollection("stock_exchange_data", {
    timeseries: {
        timeField: "ts",
        metaField: "source",
        granularity: "hours"
    }
});

Then I'll set up an "init" recipe to run it:

# Initialize the configured database
init:
    mongosh $MDB_URI init.js

Now I can run just init to run the "init.js" script against my database.

A Parameterized Target To Run Mongosh Scripts

If I have a bunch of scripts I may want to run against my database, I'll add a "run" recipe that can run any of the scripts.

The following target makes use of recipe parameters. In the example below, "script" is a variable that is passed on the command-line, and then referred to using {{script}}

# Run a script on the configured database
run script:
    mongosh $MDB_URI {{script}}

With the recipe above in your Justfile, you can execute a mongosh script called "create_view.js" like this:

just run create_view.js

Other Features

The documentation for Just is very long – a good indication of quite how many features it has! Nevertheless, it remains relatively straightforward to get up and running. Some features I've found useful:

There are a bunch more features. I recommend you scan the documentation to get a better picture of all it can do.

In Conclusion

I've seen a bunch of benefits since I started to add Justfiles to all my projects. It speeds up my day-to-day operations when I'm working with MongoDB (and other tools). When I come back to a project after a while, I can see the commands needed to create and work with the database, by running just --list or just scanning the Justfile. The ability to document each recipe and easily list them is super-helpful, and it also provides a way to help others work with the projects I've worked on.

The examples in this post are relatively straightforward, but Just really comes into its own when you want a recipe to run multiple commands, each with their own list of flags that you would otherwise have to remember. They're an excellent alternative to writing shell scripts!