# Nix(OS) - Revolutionizing packaging and configuration management! ![NixOS logo](img/nixos-white.svg) The Purely Functional Linux Distribution
# Before we begin (FYI) - Ask questions at any time - Please ask lots of questions :) - The slides contain some redundancy - There are a few optional slides at the end - Please give me feedback (louder, faster/slower, more/less details, etc.) Notes: - Try to ask longer questions after the presentation - I might postpone some questions until the corresponding slide - All questions are awesome! (even if I forget to mention this)
# About me - Michael Weiss aka. primeos - Computer science student at the University of Tübingen - I love free software, etc. - First `nixpkgs` commit: 2016-10-05 - I maintain ~44 packages and ~4 modules (2018-09-18) - Email: [dev.primeos@gmail.com](mailto:dev.primeos@gmail.com) Notes: - First commit: 8268f0a759597bfdf07f2cc48b265215040b5c5e (pgpdump: init at v0.31) - 2018-06-08 - Maintaining ~41 packages - Wrote ~4 modules (2 active, 1 WIP, 1 disabled) - Maintaining 3 modules
# Main components - Nix (package manager) - Nixpkgs (Nix packages collection) - NixOS (operating system) - NixOps (DevOps / cloud deployment tool) Notes: - Entries correspond to GH repos - Only read out slide (details will follow later, e.g. on the next slide)
# Nix* ISO/OSI model ![Nix* ISO/OSI model](img/nix-iso-osi.svg) Notes: - Nix ~ dpkg, pacman, portage - Nixpkgs ~ apt/deb sources, PKGBUILD - NixOS ~ Arch, Ubuntu, Gentoo - NixOps ~ Puppet, Ansible, Chef - Configuration Management - Infrastructure as Code - DevOps - Derivations and store paths
# Other tools - Hydra (Nix based continuous build system) - Disnix (distributed services deployment) - PatchELF (change dynamic linker and RPATH) - {cabal,go,node,pip,python,pypi,composer,hex, bower,vim,...}2nix Notes: - Disnix: Deploying (micro)services - nixcrates, cargo2nix, elm2nix, ...
# History - Started as a **research project** (with funding) - First paper in 2004 (many will follow) - Nix package manager developed by **Eelco Dolstra** as part of his PhD research (~2003) - First NixOS prototype developed by Armijn Hemel as his master's thesis project - Hydra developed as part of the LaQuSo Buildfarm project
# Timeline ![Nix project logo](img/nix-snowflake.svg) - 2003: init (research begins) - 2007: NixOS becomes usable + x86_64 support - 2008: Website moved to nixos.org - 2009: Nix logo + Nix(OS) build on Hydra - 2011: Migration from Subversion to Git(Hub) - 2013: Switch from Upstart to systemd + NixOps 1.0 release - 2015: NixOS Foundation + First NixCon (Berlin) - 2017: Second NixCon (Munich) - 2018: Third NixCon (London) Notes: - https://nixos.org/news.html - 2007: NixOS is not an [April Fools’ Joke](https://www.osnews.com/comments/17601) - www.osnews.com post on 1st April 2017 (introduces NixOS, focus on purely functional) - Nix logo: - Nix is Latin for snow (-> snowflake) - Was initially created for the Haskell logo competition
# Repology (2018-06-08) ![Repology statistics](img/map_repo_size_fresh-2018-06-08.svg) Notes: - Is Nix production ready? - Yes (at least definitely beyond a PoC) - Don't trust these numbers too much
# Problems of classical package managers - Upgrades/configuration changes destructively update the system state (overwriting files in sequence -> temporary inconsistency) - State -> nondeterministic builds -> not reproducible - Different versions of a binary - Package conflicts - No rollbacks - No configuration management Notes: - Apps might have an undefined behaviour or crash when launched during an upgrade + possibility of permanent damage when interrupted (e.g. even unbootable!). - Incomplete dependency specifications (no isolated build environment) - Nominal dependency specifications (only name and optionally version) - Configuration files not controlled by the package manager. - Upgrading modified configuration files is problematic. - Binaries = global variables (e.g. /bin/bash)
# Nix(OS) - Atomic upgrades/rollbacks (software & configuration) - Multiple versions of a package (side-by-side, e.g. testing a new Apache version) - Deterministic & Reproducible builds - Reliable upgrades (and rollbacks - configuration bound to correct software version + service reloads/restarts) - Reliable channel upgrades/rollbacks (e.g. 17.03 -> 17.09) - Unprivileged users can securely install software Notes: - Next slide: Declarative configuration management
# Being functional - Classically: Imperative configuration - Stateful changes (-> dependency hell, inconsistent states, etc.) - NixOS: Declarative configuration - Packages/Configuration = immutable values - (Complete) rebuilds instead of destructive updates - Referential transparency (~an expression always evaluates to the same result) Notes: - Referential transparency: An expression always evaluates to the same result in any contex (no side effects like (uncontrolled) imperative updates).
# Problems - Lacking manpower/workforce (e.g. for better testing/security/documentation) - Not all packages are reproducible (2016: 12.8%) - Running pre-compiled binaries - Scripts with hard-coded paths don't work - No GUI for package/configuration management - No LTS releases or super stable (i.e. old :P) branches - Not all use-cases or configuration options supported - Some tricks available + PRs welcome ;) Notes: - 13 Dec 2016: "Currently, out of 36550 build steps in a full NixOS evaluation, only 4699 (12.8%) were detected as non-deterministic. (Note that Hydra can only detect the presence of non-determinism, not its absence)." - Eelco
# Nix - A purely functional package manager (transparent source/binary deployment) - Secure multi-user support - Stores packages in the Nix store (`/nix/store` by default) - Each package has it's own unique identifier/directory - E.g. `qn96dbgqdryaw38zw6v08da34q5v4qz3-git-repo-1.12.37` (cryptographic hash, name, version) - Enables multiple versions & binary substitutes - "Forces" complete dependencies Notes: - Hash captures all dependencies (160-bit truncations of SHA-256 encoded in base-32 (i.e. 32 characters)) - src/libstore/store-api.cc Store::makeStorePath + comment above - Version: Digit after a dash - Garbage collection (nix-collect-garbage) - Similar to Content Addressed Storage (CAS) - Nothing is ever overwritten!
# Nix expressions / Nix expression language - A DSL (not a GPL!) - Describes graphs of build actions ("derivations") - Packages, compositions of packages, configurations, ... - Dynamically typed ~~("Nix won't be complete until it has static typing." @edolstra)~~ - https://typing-nix.regnat.ovh/ - Lazy (a very important feature!) - Purely functional (no side-effects, immutable store) - Turing complete (e.g. Dhall is not -> [dhall-nix](https://github.com/dhall-lang/dhall-nix)) Notes: - DSL vs GPL: MD5 collisions by @Azlig (Chromium updater) :D - "Config files shouldn't be Turing complete" (-> always terminate) - Dhall is explicitly typed, evaluated, and configuration files can be reduced to a normal form - TODO: - Bad evaluation errors? :D - Missing typing - Error in anonymous function at unknown file? - Evaluated vs interpreted? - Bugs? :D (e.g. a-b)
# nix-repl - **Demo**
# Nixpkgs (the Nix packages collection) - Main GitHub repository (permissive MIT/X11 license) - Contains definitions of packages (Nix) and modules (NixOS) - Also contains tests, library functions, etc. - Different branches (rolling: master, stable: release-YY.MM) - Build and tested by Hydra (+ uploaded to binary cache) - Distributed through (nixpkgs-)channels (nixpkgs-unstable, nixos-unstable(-small), nixos-YY.MM(-small)) Notes: - Stable: MM = 03 or 09 (end of month, branch off one month before) - Talk about codenames, release schedule, freezing, etc.?
# An example Nix package (pgpdump) ```nix pgpdump = callPackage ../tools/security/pgpdump { }; ``` ```nix { stdenv, fetchFromGitHub , supportCompressedPackets ? true, zlib, bzip2 }: stdenv.mkDerivation rec { name = "pgpdump-${version}"; version = "0.32"; src = fetchFromGitHub { owner = "kazu-yamamoto"; repo = "pgpdump"; rev = "v${version}"; sha256 = "1ip7q5sgh3nwdqbrzpp6sllkls5kma98kns53yspw1830xi1n8xc"; }; buildInputs = stdenv.lib.optionals supportCompressedPackets [ zlib bzip2 ]; meta = with stdenv.lib; { description = "A PGP packet visualizer"; longDescription = '' pgpdump is a PGP packet visualizer which displays the packet format of OpenPGP (RFC 4880) and PGP version 2 (RFC 1991). ''; homepage = http://www.mew.org/~kazu/proj/pgpdump/en/; license = licenses.bsd3; platforms = platforms.linux; maintainers = with maintainers; [ primeos ]; }; } ```
# Dependency graphs pgpdump's runtime dependencies: ```bash nix-store -q --graph $(nix-store --realise $(nix-instantiate -A pgpdump)) | dot -Tpdf | zathura - ``` ![Pgpdump runtime dependencies](img/pgpdump-rdeps.svg) Notes: - Build time dependencies available as well (but graph way too large) - E.g. CVE patches as build inputs
# nix-env (manipulate or query Nix user environments) ![User environments](img/user-environments.png) https://nixos.org/nix/manual/figures/user-environments.png
# nix-shell - **Demo**
# NixOS - Implements a declarative and purely functional system configuration model - Based on Nix (package + configuration management) - NixOS modules (separation of concerns) - Form the full "system configuration" ```nix { config, pkgs, ... }: { options = { nested attribute set of option declarations using mkOption }; config = { nested attribute set of option definitions }; } ``` Notes: - modules = functions - { config } -> config = full system configuration
# An example NixOS module ```nix { config, lib, pkgs, ... }: with lib; let cfg = config.programs.vim; in { options.programs.vim = { defaultEditor = mkOption { type = types.bool; default = false; description = '' When enabled, installs vim and configures vim to be the default editor using the EDITOR environment variable. ''; }; }; config = mkIf cfg.defaultEditor { environment.systemPackages = [ pkgs.vim ]; environment.variables = { EDITOR = mkOverride 900 "vim"; }; }; } ```
# Another example NixOS module ```nix { config, lib, pkgs, ... }: with lib; let cfg = config.services.monetdb; in { meta.maintainers = with maintainers; [ StillerHarpo primeos ]; ###### interface options = { services.monetdb = { enable = mkEnableOption "the MonetDB database server"; package = mkOption { type = types.package; default = pkgs.monetdb; defaultText = "pkgs.monetdb"; description = "MonetDB package to use."; }; user = mkOption { type = types.str; default = "monetdb"; description = "User account under which MonetDB runs."; }; group = mkOption { type = types.str; default = "monetdb"; description = "Group under which MonetDB runs."; }; dataDir = mkOption { type = types.path; default = "/var/lib/monetdb"; description = "Data directory for the dbfarm."; }; port = mkOption { type = types.ints.u16; default = 50000; description = "Port to listen on."; }; listenAddress = mkOption { type = types.str; default = "127.0.0.1"; example = "0.0.0.0"; description = "Address to listen on."; }; }; }; ###### implementation config = mkIf cfg.enable { users.users.monetdb = mkIf (cfg.user == "monetdb") { uid = config.ids.uids.monetdb; group = cfg.group; description = "MonetDB user"; home = cfg.dataDir; createHome = true; }; users.groups.monetdb = mkIf (cfg.group == "monetdb") { gid = config.ids.gids.monetdb; members = [ cfg.user ]; }; environment.systemPackages = [ cfg.package ]; systemd.services.monetdb = { description = "MonetDB database server"; wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; path = [ cfg.package ]; unitConfig.RequiresMountsFor = "${cfg.dataDir}"; serviceConfig = { User = cfg.user; Group = cfg.group; ExecStart = "${cfg.package}/bin/monetdbd start -n ${cfg.dataDir}"; ExecStop = "${cfg.package}/bin/monetdbd stop ${cfg.dataDir}"; }; preStart = '' if [ ! -e ${cfg.dataDir}/.merovingian_properties ]; then # Create the dbfarm (as cfg.user) ${cfg.package}/bin/monetdbd create ${cfg.dataDir} fi # Update the properties ${cfg.package}/bin/monetdbd set port=${toString cfg.port} ${cfg.dataDir} ${cfg.package}/bin/monetdbd set listenaddr=${cfg.listenAddress} ${cfg.dataDir} ''; }; }; } ```
# An example NixOS configuration ```nix { config, pkgs, ... }: { system.nixos.stateVersion = "18.03"; nix.useSandbox = true; boot.kernelPackages = pkgs.linuxPackages_latest; i18n = { consoleFont = "Lat2-Terminus16"; consoleKeyMap = "de"; defaultLocale = "en_US.UTF-8"; }; time.timeZone = "Europe/Berlin"; environment = { systemPackages = with pkgs; [ # Core vim_configurable gitAndTools.gitFull wget # Monitoring htop python3Packages.glances # Tools time file tree lsof pciutils hdparm zip unzip # Network inetutils bind.dnsutils tcpdump ]; variables = { EDITOR = "vim"; }; }; programs.bash.enableCompletion = true; } ```
# Nixpkgs overlays ```nix self: super: # self: Final package set / fixed-point result (use as dependencies) # super: Previous evaluation result { nix = super.nix.override { storeDir = "${
}/store"; stateDir = "${
}/var"; }; boost = super.boost.override { python = self.python3; }; rr = super.callPackage ./pkgs/rr { stdenv = self.stdenv_32bit; }; } ```
# NixUP & co. - Nix User Profile - Manage $HOME
# Community - A great & kind community ([overview](https://nixos.org/nixos/community.html)) - [nix-devel mailing list](https://groups.google.com/forum/#!forum/nix-devel) - Discourse (Forum): [discourse.nixos.org](https://discourse.nixos.org/) - Bugs and PRs via GitHub ([Nixpkgs](https://github.com/NixOS/nixpkgs)) - [#nixos](irc://irc.freenode.net/#nixos) on irc.freenode.net - Blogs ([NixOS planet](https://planet.nixos.org/)) - Local meetups (e.g. in [Stuttgart](https://hackmd.shackspace.de/NixOS-meetup)) - NixCon - Commercial support via consulting companies
# Learning - [Learn X in Y minutes, where X=nix](https://learnxinyminutes.com/docs/nix/) - [A tour of Nix](https://nixcloud.io/tour/) - By Joachim Schiele & Paul Seitz from Tübingen ;) - [Unofficial user's wiki](https://nixos.wiki/wiki/Main_Page) - Manuals ([Nix](https://nixos.org/nix/manual/), [Nixpkgs](https://nixos.org/nixpkgs/manual/), [NixOS](https://nixos.org/nixos/manual/), and [NixOps](https://nixos.org/nixops/manual/))
# Nix Pills - A ported version of the Nix Pills (a series of blog posts) - A tutorial introduction into the Nix package manager and Nixpkgs package collection - In the form of short chapters called 'pills' - https://nixos.org/nixos/nix-pills/
# Trying out Nix* - Use Nix side-by-side with your regular package manager: ```bash curl https://nixos.org/nix/install | sh ``` - Experiment with nix-env, nix-shell, nix-repl, etc. - Try out NixOS (e.g. VirtualBox demo appliance) - Install NixOS
# Thank you :) ![Nix project logo](img/nix-snowflake.svg) - Questions? - Feedback - Discussion Notes: - Feedback - Possible improvements? - Do or do not, there is no try. - Q&A - Hacking - Summary
# License - For the content of these slides - Public domain - CC0 1.0 Universal (CC0-1.0) - Doesn't apply to external sources like images - Some quotes, notes, etc. are from other sources (this can hopefully be considered fair use) - Framework: reveal.js - MIT (see https://github.com/hakimel/reveal.js)