In the world of software development, having a reliable, efficient and reproducible local development environment is crucial for productivity and collaboration. Different projects often require specific versions of dependencies, and conflicts often arise when multiple projects share the same system-wide configuration. Resolving these conflicts can be frustrating and can have severe impacts on productivity. There are many solutions to this problem, but this blog post will focus on the Nix ecosystem.
What is Nix?
The Nix Expression Language is a declarative language designed specifically for building and managing packages using the Nix Package Manager.
The Nix Package Manager is a powerful and purely functional package manager designed to create and manage software environments. It provides a declarative and reproducible approach to package management, allowing users to define and install packages along with their dependencies in a self-contained and isolated manner. Nix treats packages as immutable and ensures that installations are atomic, meaning that they do not interfere with each other or the system-wide configuration. This approach enables developers to create consistent and reliable development environments across different machines, making it easier to reproduce builds and share environments with others.
The Nix ecosystem is much more than this, but the Nix Package Manager, in combination with the nix-shell command, is what we will use to create isolated development environments.
Why do I need a local development environment?
Local development environments offer numerous benefits. They enable developers to work on projects with specific dependencies without interfering with system-wide configurations. This isolation prevents conflicts and makes it easier to reproduce builds on different machines. Moreover, local environments allow developers to experiment with different versions of dependencies and switch between them effortlessly. While cloud platforms offer convenience and scalability, local development environments complement them by providing offline capabilities, speed, cost-effectiveness, customization, reproducibility, and more. A combination of local and cloud-based development approaches can empower developers to work efficiently and effectively throughout the software development lifecycle.
Benefits of building development environments from code
Building development environments from code offers several benefits, including reproducibility and collaboration. By defining the development environment in code, developers can ensure reproducibility by capturing all the necessary dependencies, configurations, and tooling as code. This allows for consistent and predictable environments across different machines, making it easier to reproduce and debug issues. Moreover, code-based environments promote collaboration as developers can version control and share the environment setup. Team members can easily sync their environments, reducing setup time and improving collaboration efficiency. Code-based development environments foster a streamlined and unified development experience, enhancing productivity and enabling seamless collaboration among developers.
Building a local devenv with Nix
Although there are other Nix-based solutions that don't require knowing the Nix language, understanding the core concepts lays a solid foundation for using the development environments that are built on Nix.
Install Nix:
Nix is supported on Linux & macOS. If you're running Windows, Nix will run in WSL. For both Linux & macOS It is recommended to install Nix in multi-user mode by running the following command:
After installation completes, close and reopen your terminal window.
Ad-hoc shell environments
Now that you have Nix installed, you can install & remove packages without cluttering your system. The nix-shell command makes it easy to install any package, making it available only in the Nix shell via the use of symlinks and path updates. Once you exit the shell, the package remains on your system for future use but is no longer in the path. If you decide that you won't want to reuse the package, you can free up disk space by running nix-collect-garbage or nix-store --gc to completely remove all unreferenced packages.
Every package that is downloaded by Nix is written to a directory name under /nix/store/, including a cryptographic hash of the package. This ensures that every version of a package is in its own unique path.
Setup a sample Nix development environment
For this example, say you have 2 projects that require different versions of nodejs. I will also add a specific version of Python to project2 to show how it overrides the system-installed version.
Starting with a folder structure like this,
In the following configuration files, the buildInputs list contains the packages that we'd like to install in the shell. The shellHook parameter is a string that will be executed once the shell has been setup. In this example, it will output the versions of node & python3 available to the shell.
Add the following config to project1/shell.nix:
And the following to project2/shell.nix:
Running nix-shell in either of these directories should now spawn a bash shell:
The shell failed to open because these versions of node are EOL and Nix has marked them as insecure so they must be explicitly allowed. Thankfully, Nix outputs a useful error message including multiple solutions to the problem. Following the recommendations and running nix-shell again returned the same error but for the openssl dependency which has also been marked as insecure. Adding all required insecure packages to ~/.config/nixpkgs/config.nix ends up looking like this:
Now nix-shell works and nodejs v16.20.0 is available in the project1 directory. The python version shown is my system-wide installation under /usr/local/bin rather than the /nix/store path:
Switching to the project2 directory we see nodejs v14.21.3 and the system-wide Python installation has been overridden by Python 3.10.11:
Manually running nix-shell:
How the direnv tool can enhance the experience:
While Nix helps manage packages and environments, the direnv tool takes the experience a step further. Direnv integrates with your shell and automatically loads environment variables when you enter a directory. It can also be configured to load the nix shell defined in shell.nix. This feature allows you to switch between Nix-managed development environments as easily as changing directories.
Setup direnv
Install direnv using a package manager like Homebrew:
or Nix:Enable direnv by adding the following line to your shell's initialization file (e.g., .bashrc, .zshrc):
Navigate to your project directory and create a .envrc file:
Run direnv allow to allow the .envrc file to be sourced on entering the directory.
Now, whenever you enter the project directory, direnv will automatically activate the Nix environment, providing you with the necessary dependencies and configurations. The .envrc can also contain environment variables specific to the project, but these would not be loaded when running a nix-shell manually without allowing direnv to source the .envrc. Keeping all of the configuration within the shell.nix file allows the user to access the same environment without requiring the use of direnv.
Install nix-direnv
Direnv is convenient but can become annoying when it blocks for several seconds every time you enter an allowed directory. The nix-direnv project replaces some of direnv's internal functions, implementing a cache of the Nix shell. This won't have any effect on the initial shell creation, but subsequent directory traversals will load the shell from the cache, resulting in a significantly improved experience.
As a non-root user, install the nix-direnv package:
Then add nix-direnv to $HOME/.config/direnv/direnvrc:
Also add the following configuration to your /etc/nix/nix.conf:
Automatically running nix-shell with nix-direnv:
Alternative Solutions:
While nix-shell and direnv provide a simple but powerful combination for managing local development environments, there are many alternatives, including:
devenv.sh
The devenv CLI tool, built on-top of nix-shell and direnv, provides commands to simplify the creation and management of Nix development environments. It also manages direnv configurations and provides a command to run environments in containers. It offers simplicity and ease of use by adding a layer of abstraction above Nix.
Devbox
Another solution based on the Nix ecosystem but configured and managed using the devbox CLI tool. Devbox also supports devcontainers for integration with VSCode.
Dev Containers
Developed by Microsoft, devcontainers are based on Visual Studio Code's Remote Development extension. They allow you to define development environments using Docker containers and provide a consistent development experience across different machines.
devcontainers are well-suited for teams using VSCode as their primary IDE, providing a seamless integration with familiar workflows.
DevPod
DevPod is a desktop GUI app that creates isolated development environments based on the Dev Container spec. It automates the setup of dependencies and configurations, making it easy to share and reproduce development environments across teams. However, compared to Nix, DevPod introduces an additional layer of complexity with container management and may require more resources.
Overall, the choice between Nix's nix-shell and the alternative solutions depends on various factors such as project complexity, platform requirements, and personal preferences. Nix's nix-shell and devenv.sh excel in providing lightweight, reproducible, and fine-grained control over dependencies using the Nix package manager. On the other hand, alternative solutions like DevPod, and devcontainers provide better isolation by use of containerization, and provide IDE integration.
What's Next?
I've only just begun my Nix journey, starting with building development environments using "classic" Nix. In a future post I'll cover the new experimental features of Nix, specifically using Flakes to improve reproducibility.
Further Reading
- nix.dev: home of official documentation for the Nix ecosystem
- Nix Pills: a tutorial introduction into the Nix package manager and Nixpkgs package collection
- NixOS Wiki: the unofficial user's wiki
- Find nix packages: search more than 80,000 packages in the Nix Channels
- Nix Package Version: an unoffical project to search for all versions of packages across all Nix Channels
- Awesome Nix: A curated list of the best resources in the Nix community.
