About

Trustix - build transparency reference implementation

Agree on inputs; agree on outputs

Trustix works by associating a hash of the inputs to a build with a hash of the resulting output. Reproducibility means that if multiple builders started with the same inputs, they should produce the same output. By comparing across builders, we can detect problems such as non-reproducibility and trojan viruses. By tracking the history of a build, we can ascertain security compromises, track build quality, and help build trust over time for a given system and provider.

Binary planting protection

When we download a program, we typically trust that the binaries correspond to the program that we want. But how can we be sure that what we get is actually what we want, and not something malicious?

The trust we have in what we download is generally a proxy for the trust we have in the provider we download it from. This provider can put in place various measures to protect against attacks such as man-in-the-middle attacks, usually by using cryptographic hashes or digital signatures. These would protect against tampering in transit, but would not protect us if the binary was already compromised.

For example, the build infrastructure of the provider might be compromised, such that even if good source code goes in, bad binaries come out. This turns the provider into a single point of failure for trust.

Trustix makes it easy to distribute this responsibility among a number of non-authoritative peers. By comparing the build results between various providers, we can ascertain whether one or more might have been compromised.

Tamper-evident history

Even if a provider's private key is compromised and an attacker gains unrestricted access, Trustix's immutable build log ensures that any modification to old entries can be detected and flagged. Any entries known to be from before the attack can still be trusted.

Since the build log provided by trustix is additive, we can ascertain when exactly a provider might have been compromised. Any binaries from before this point can still be trusted. Any modification of the log history is easily detected by peers, making history tampering impossible.

Decide your level of trust

There is no universal notion of "verified"; you decide which providers to trust and how much. Your personal security needs may be low, in which case you can trust any binary cache on your list. In your high-security environment, however, you may require every provider to agree on the same output in order to trust any of them.

Analyze package reproducibility

In the real world, builds are often non-reproducible without being maliciously modified. The build output might contain the time or date, or a reference to a local folder used during the build process. While most of these cases are benign, we can't really tell. The answer is to measure and track reproducibility. By comparing what different providers build on different machines and in different circumstances, we can track which packages are reproducible, and thus more trustworthy.

Trustix - A new model for Nix binary substitutions

Trustix is a tool that compares build outputs for a given build input across a group of independent providers to establish trust in software binaries.

Overview

We often use pre-built software binaries and trust that they correspond to the program we want. But nothing assures that these binaries were really built from the program's sources and reasonable built instructions. Common, costly supply chain attacks exploit this to distribute malicious software, which is one reason why most software is delivered through centralized, highly secured providers. Trustix, a tool developed via an NGI0 PET grant, establishes trust in binaries in a different, decentralized manner. This increases security, and paves the way for an internet where small providers can deliver safe code, ultimately with a safer and larger offer for the user.

Trustix is developed for the Nix ecosystem.

How does this translate to Nix?

In the Nix ecosystem, pre-built binaries are distributed through so-called binary substituters. Similar to other centralized caching systems, they are a single point of failure in the chain of trust when delivering a package to a user. This is problematic for several reasons:

First, if anyone manages to compromise the NixOS Hydra build machines and its keys, they could upload backdoored builds to users. In the Nix ecosystem, a compromised key is even more dangerous because https://cache.nixos.org can't use a rolling key because of the way it is set up. This means that a compromised key would realistically mean that all packages in the cache are compromised. They would have to be rebuilt or garbage collected which is very costly.

Second, the NixOS Hydra hardware, on which the binaries are built, may also be compromised and not considered trustworthy by more security conscious users.

Trustix design

Trustix aims to solve this problem via distributed trust & trust agility. Essentially it compares build outputs across a group of independent builders that log and exchange hashes of build input/output pairs. This is achieved through the following methodology:

• Each builder is associated with a public-private key pair
• In a post-build hook the output hash (NAR hash) of the build is uploaded to a ledger

This allows a user to trust binary substitutions based on an M-of-N vote among the participating builders.

Here is an example: Let's say we have 4 builders configured: Alice, Bob, Chuck & Dan. We have configured Trustix to require a 3/4 majority for a build to be trusted. Alice, Bob, Dan and Chuck all claim to have built the hello derivation. All builders participate in the Trustix network and communicate precisely what they have built with a hash that describes the build inputs of hello, and what have obtained as output with another hash. For the same input, the first 3 builders have arrived at the same output hash but Chuck has obtained something different.

This information can now be used by a Trustix user to:

• track build reproducability across a large number of builders.
• trust only builds that have been confirmed by a majority of selected builders.
• automatically identify and exclude misbehaving builders such as Chuck in above's example.

Trustix - End to end Nix howto's

Up until now we have talked about components in isolation, let's go through some practical examples of how to deploy your own Trustix nodes.

Trustix - Usage via Nix

The easiest way to use Trustix is via the NixOS modules, though even they require some manual preparation in terms of generating keys.

This document will guide you through the very basic NixOS setup required both by log clients and log publishers.

How to actually publish/subscribe are laid out in other documents.

Requisites

• A NixOS installation using Flakes

Create keys

All Trustix build logs are first and foremost identified by their key pair, which will be the first thing we have to generate.

Let's start by generating a key pair for our log:

$ mkdir secrets
$ nix run github:nix-community/trustix#trustix -- generate-key --privkey secrets/log-priv --pubkey secrets/log-pub

Additionally logs are identified not just by their key, but how that key is used. If a key is used for multiple protocols (not just Nix) those logs will have a different ID. This ID is what subscribers use to indicate what they want to subscribe to.

To find out the log ID for the key pair you just generated: $ nix run github:nix-community/trustix#trustix -- print-log-id --protocol nix --pubkey $(cat secrets/log-pub)

Flakes

• flake.nix

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

    trustix = {
      url = "github:nix-community/trustix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  outputs = { nixpkgs, flake-utils, trustix, ... }: {
    nixosConfigurations.trustix-example = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules =
        [ ({ pkgs, ... }: {
            # import trustix modules
            imports = [
              trustix.nixosModules.trustix
              ./configuration.nix
            ];
          })
        ];
    };

  };
}

• configuration.nix:

{ config, pkgs, lib, ... }:
{
  services.trustix = {
    enable = true;

    # Trustix differentiates between the concepts of a "signer" and a "publisher".
    # A signer refers to a private key implementation.
    # These can be file based or use hardware tokens.
    signers.my-signer = {
      type = "ed25519";
      # Configuring the private key like this by path is bad practice because the key ends up world readable in /nix/store.
      # You should either:
      # - Put the key in a persistent path and reference it like: `ed25519.private-key-path = "/path/to/key"`
      # - Use a secrets management solution like sops-nix or agenix.
      ed25519.private-key-path = ./secrets/log-priv;
    };

    publishers = [
      {
        # Use the key configured above
        signer = "my-signer";

        # Trustix is built first and foremost for Nix, but could also be used for verifying other package ecosystems.
        protocol = "nix";

        # An arbitrary (string -> string) attrset with metadata about this log.
        # This isn't used by the Trustix logs but is used to inform the Nix binary cache proxy about possible substitution sources.
        meta = {
          upstream = "https://cache.nixos.org";
        };

        # The public key identifying this log.
        publicKey = {
          type = "ed25519";
          key = builtins.readFile ./secrets/log-pub;
        };
      }
    ];
  };
}

Effect

This will set up an instance of Trustix on your system. In the next chapter we will look at using the post build hook to publish results to our local log.

Trustix - Usage via Nix

In the previous chapter we set up the main Trustix daemon. It's now time to actually start using it to publish build results.

Requisites

• A NixOS installation using Flakes
• The basic setup from the previous chapter

Setup

• configuration.nix:

{ config, pkgs, lib, ... }:
{
  # Our basic Trustix configuration from before
  services.trustix = {
    enable = true;

    signers.my-signer = {
      type = "ed25519";
      ed25519.private-key-path = ./secrets/log-priv;
    };

    publishers = [
      {
        signer = "my-signer";
        protocol = "nix";
        meta.upstream = "https://cache.nixos.org";
        publicKey = {
          type = "ed25519";
          key = builtins.readFile ./secrets/log-pub;
        };
      }
    ];
  };

  # Enable the post build hook to push builds to the main Trustix daemon
  services.trustix-nix-build-hook = {
    enable = true;
    # Log id as returned by `trustix print-log-id --protocol nix --pubkey $(cat secrets/log-pub)`
    # This is your logs globally unique identifier and what clients will use to subscribe to your build results.
    logID = "0c7942343fa91b610704d531f552f3e785705dbd7d22c965bc0d58fa3ff2c87c";
  };
}

Effect

This sets up Nix with a post build hook that publishes any builds performed locally to your locally running log.

Trustix - Subscribing

This document walks you through how to subscribe to an already published binary cache.

Requisites

• A local Trustix instance
• A remote log's metadata
  • Public key
  • URL

Configuring

• Add log(s) to your configuration.nix

{ pkgs, config, ... }:
{

  services.trustix = {
    enable = true;

    subscribers = [
      {
        protocol = "nix";
        publicKey = {
          type = "ed25519";
          key = "2uy8gNIOYEewTiV7iB7cUxBGpXxQtdlFepFoRvJTCJo=";
        };
      }
    ];

    # A remote can expose many logs and they are not neccesarily created by the remote in question
    remotes = [
      "https://demo.trustix.dev"
    ];

  };

}

Trustix - Binary cache setup

The easiest way to use Trustix is via the NixOS modules, though even they require some manual preparation in terms of generating keys.

This document walks you through how to configure your local system as a binary cache.

Requisites

We are assuming you have already followed the steps to set up one or more subscribers to your local Trustix instance.

• Generate a public/private keypair to use with your local binary cache.

$ nix-store --generate-binary-cache-key binarycache.example.com cache-priv-key.pem cache-pub-key.pem

• Move the keys somewhere persistent and safe Of course having keys around readable by anyone on the system is not a good idea, so we will move these somewhere safe. In this tutorial we are using /var/trustix/keys but you are free to use whatever you wish. A deployment tool like Colmena, Morph or NixOps is recommended to deal with secrets.

$ mv cache-priv-key.pem /var/trustix/keys/cache-priv-key.pem

Configuring

• Add the binary cache to your configuration.nix

{ pkgs, config, ... }:
{

  # Enable the local binary cache server
  services.trustix-nix-cache = {
    enable = true;
    private-key = "/var/trustix/keys/cache-priv-key.pem";
    port = 9001;
  };

  # Configure Nix to use it
  nix = {
    binaryCaches = [
      "http://localhost:9001"
    ];
    binaryCachePublicKeys = [
      "binarycache.example.com://06YZJreoL8n9IdDlhnA3t7uJmHUI/rIIy3uO4FHRY="
    ];
  };

  # Configure your Trustix daemon with a decision making process on how
  # to determine if a build is trustworthy or not.
  #
  # In this case we configure it to have at least 2/3 majority to be substituted.
  #
  # Note that this configuration is incomplete and assumes you have already set up a subscriber.
  services.trustix = {
    deciders.nix = [
      {
        engine = "percentage";
        percentage.minimum = 66;
      }
    ];
  };

}

You are now all set up to use Trustix as a substitution method!

Project structure

Trustix is structured as a monorepo consisting of many subpackages:

• trustix

The main package with all log functionality. This component is generic and doesn't know anything about any Nix or other package manager specifics.

• trustix-doc

The main documentation package that aggregates documentation from the various subpackages.

• trustix-nix

This is a supplemental daemon to the main Trustix daemon that layers some knowledge about Nix on top of the generic log functionality. It contains a post-build hook used to submit newly built packages to the logs, a binary cache HTTP interface and a development tool to submit already built closures.

• trustix-nix-r13y

This package is an implementation of a reproducibility tracker backed by logs.

• trustix-proto

Trustix-proto contains all shared protobuf definitions shared by various components, as well as generated Go libraries to interact with Trustix over it's RPC mechanism (gRPC).

Globally installed tooling

Trustix doesn't depend on much in the way of globally installed tools.

We do make two assumptions in regards to tooling managed outside of the repository though:

• Nix

If you've read this far you likely already know Nix and what it is, so we won't go into any detail about this.

• direnv

A shell extension to load directory local environments in a currently running shell and/or editor. This will load a present shell.nix/default.nix when used with the direnv rule use nix, which is the mode of operation we are using direnv in.

Getting started

All subpackages have their own shell environments which all needs to be explicitly whitelisted to be loaded. For convenience we have a Makefile target in the root of the project called direnv-allow. To whitelist all subpackages run:

$ make direnv-allow

Makefile structure

All components are using Makefile's as their development entry points for ease of use.

All standard Make targets are always implemented, even though they are no-ops in some cases where they don't make sense. For example a build step doesn't make sense for most Python code.

These are all standard make targets you can expect to find for any given package:

• build

Builds the package.

• test

Runs the tests for a given package.

• lint

This target runs all configured linter steps.

• format

This target checks the formatting of a given package.

• develop

This target runs the package in development (watch) mode.

• doc

This target builds documentation. This is mostly outputing markdown files in the relevant location for the trustix-doc package to compose.

Running the whole setup

To run individual components change directory to the relevant package and run:

$ make develop

This also works from the project root where it will start all packages in watch mode.

Quickly runing all tests

From the root directory run:

$ make all

Notes

Cryptographic keys for development is checked in to the repository for ease of use and a very quick getting started experience.

Protocol Documentation

Table of Contents

• api/api.proto

  • GetLogAuditProofRequest

  • GetLogConsistencyProofRequest

  • GetLogEntriesRequest

  • GetMapValueRequest

  • KeyValuePair

  • Log

  • Log.MetaEntry

  • LogEntriesResponse

  • LogHeadRequest

  • LogSigner

  • LogsRequest

  • LogsResponse

  • MapValueResponse

  • ProofResponse

  • SparseCompactMerkleProof

  • ValueRequest

  • ValueResponse

  • Log.LogModes

  • LogSigner.KeyTypes

  • LogAPI

  • NodeAPI

• rpc/rpc.proto

  • DecideRequest

  • DecisionResponse

  • EntriesResponse

  • EntriesResponse.EntriesEntry

  • FlushRequest

  • FlushResponse

  • LogValueDecision

  • LogValueResponse

  • SubmitRequest

  • SubmitResponse

  • SubmitResponse.Status

  • LogRPC

  • RPCApi

• schema/loghead.proto

  • LogHead

• schema/logleaf.proto

  • LogLeaf

• schema/mapentry.proto

  • MapEntry

• schema/queue.proto

  • SubmitQueue

• Scalar Value Types

Top

api/api.proto

GetLogAuditProofRequest

Get log audit proof for a given tree

GetLogConsistencyProofRequest

Get a consistency proof between two given log sizes

GetLogEntriesRequest

GetMapValueRequest

KeyValuePair

Log

Log.MetaEntry

LogEntriesResponse

LogHeadRequest

Request a signed head for a given log

LogSigner

LogsRequest

LogsResponse

MapValueResponse

ProofResponse

SparseCompactMerkleProof

Sparse merkle tree proof

ValueRequest

ValueResponse

Log.LogModes

LogSigner.KeyTypes

LogAPI

LogAPI is a logical grouping for RPC methods that are specific to a given log.

NodeAPI

NodeAPI is a logical grouping for RPC methods that are for the entire node rather than individual logs.

Top

rpc/rpc.proto

DecideRequest

DecisionResponse

EntriesResponse

EntriesResponse.EntriesEntry

FlushRequest

FlushResponse

LogValueDecision

LogValueResponse

SubmitRequest

SubmitResponse

SubmitResponse.Status

LogRPC

RPCApi are "private" rpc methods for an instance related to a specific log. This should only be available to trusted parties.

RPCApi

RPCApi are "private" rpc methods for an instance. This should only be available to trusted parties.

Top

schema/loghead.proto

LogHead

Log

Top

schema/logleaf.proto

LogLeaf

Leaf value of a merkle tree

Top

schema/mapentry.proto

MapEntry

Top

schema/queue.proto

SubmitQueue

This type is internal only and not guaranteed stable

Scalar Value Types