About

Trustix - build transparency reference implementation

asciicast

Agree on inputs; agree on outputs

Trustix works by associating a hash of the inputs to a build with a hash of the resulting output. Reproducibility means that if multiple builders started with the same inputs, they should produce the same output. By comparing across builders, we can detect problems such as non-reproducibility and trojan viruses. By tracking the history of a build, we can ascertain security compromises, track build quality, and help build trust over time for a given system and provider.

Binary planting protection

When we download a program, we typically trust that the binaries correspond to the program that we want. But how can we be sure that what we get is actually what we want, and not something malicious?

The trust we have in what we download is generally a proxy for the trust we have in the provider we download it from. This provider can put in place various measures to protect against attacks such as man-in-the-middle attacks, usually by using cryptographic hashes or digital signatures. These would protect against tampering in transit, but would not protect us if the binary was already compromised.

For example, the build infrastructure of the provider might be compromised, such that even if good source code goes in, bad binaries come out. This turns the provider into a single point of failure for trust.

Trustix makes it easy to distribute this responsibility among a number of non-authoritative peers. By comparing the build results between various providers, we can ascertain whether one or more might have been compromised.

Tamper-evident history

Even if a provider's private key is compromised and an attacker gains unrestricted access, Trustix's immutable build log ensures that any modification to old entries can be detected and flagged. Any entries known to be from before the attack can still be trusted.

Since the build log provided by trustix is additive, we can ascertain when exactly a provider might have been compromised. Any binaries from before this point can still be trusted. Any modification of the log history is easily detected by peers, making history tampering impossible.

Decide your level of trust

There is no universal notion of "verified"; you decide which providers to trust and how much. Your personal security needs may be low, in which case you can trust any binary cache on your list. In your high-security environment, however, you may require every provider to agree on the same output in order to trust any of them.

Analyze package reproducibility

In the real world, builds are often non-reproducible without being maliciously modified. The build output might contain the time or date, or a reference to a local folder used during the build process. While most of these cases are benign, we can't really tell. The answer is to measure and track reproducibility. By comparing what different providers build on different machines and in different circumstances, we can track which packages are reproducible, and thus more trustworthy.

Trustix - A new model for Nix binary substitutions

Trustix is a tool that compares build outputs for a given build input across a group of independent providers to establish trust in software binaries.

Overview

We often use pre-built software binaries and trust that they correspond to the program we want. But nothing assures that these binaries were really built from the program's sources and reasonable built instructions. Common, costly supply chain attacks exploit this to distribute malicious software, which is one reason why most software is delivered through centralized, highly secured providers. Trustix, a tool developed via an NGI0 PET grant, establishes trust in binaries in a different, decentralized manner. This increases security, and paves the way for an internet where small providers can deliver safe code, ultimately with a safer and larger offer for the user.

Trustix is developed for the Nix ecosystem.

How does this translate to Nix?

In the Nix ecosystem, pre-built binaries are distributed through so-called binary substituters. Similar to other centralized caching systems, they are a single point of failure in the chain of trust when delivering a package to a user. This is problematic for several reasons:

First, if anyone manages to compromise the NixOS Hydra build machines and its keys, they could upload backdoored builds to users. In the Nix ecosystem, a compromised key is even more dangerous because https://cache.nixos.org can't use a rolling key because of the way it is set up. This means that a compromised key would realistically mean that all packages in the cache are compromised. They would have to be rebuilt or garbage collected which is very costly.

Second, the NixOS Hydra hardware, on which the binaries are built, may also be compromised and not considered trustworthy by more security conscious users.

Trustix design

Trustix aims to solve this problem via distributed trust & trust agility. Essentially it compares build outputs across a group of independent builders that log and exchange hashes of build input/output pairs. This is achieved through the following methodology:

  • Each builder is associated with a public-private key pair
  • In a post-build hook the output hash (NAR hash) of the build is uploaded to a ledger

This allows a user to trust binary substitutions based on an M-of-N vote among the participating builders.

Here is an example: Let's say we have 4 builders configured: Alice, Bob, Chuck & Dan. We have configured Trustix to require a 3/4 majority for a build to be trusted. Alice, Bob, Dan and Chuck all claim to have built the hello derivation. All builders participate in the Trustix network and communicate precisely what they have built with a hash that describes the build inputs of hello, and what have obtained as output with another hash. For the same input, the first 3 builders have arrived at the same output hash but Chuck has obtained something different.

This information can now be used by a Trustix user to:

  • track build reproducability across a large number of builders.
  • trust only builds that have been confirmed by a majority of selected builders.
  • automatically identify and exclude misbehaving builders such as Chuck in above's example.

Trustix - End to end Nix howto's

Up until now we have talked about components in isolation, let's go through some practical examples of how to deploy your own Trustix nodes.

Trustix - Usage via Nix

The easiest way to use Trustix is via the NixOS modules, though even they require some manual preparation in terms of generating keys.

This document will guide you through the very basic NixOS setup required both by log clients and log publishers.

How to actually publish/subscribe are laid out in other documents.

Requisites

  • A NixOS installation (flakes based optional)

Strategies

Classical Nix

It's highly recommended to use some automated tool like niv to ensure you are up to date with your external dependencies, here we'll show you how to integrate Trustix in your NixOS configuration manually using no external tooling.

From within your configuration directory, clone Trustix:

$ git clone https://github.com/nix-community/trustix.git

And add it to your NixOS configuration like:

{ config, pkgs, lib, ... }:
{
  imports = [ ./trustix/nixos ];
}

Flakes

This is a minimal flake.nix for using Trustix with Flakes:


{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";

    trustix = {
      url = "github:nix-community/trustix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  outputs = { nixpkgs, flake-utils, trustix, ... }: {

    nixosConfigurations.trustix-example = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules =
        [ ({ pkgs, ... }: {
            # import trustix modules
            imports = [ trustix.nixosModules.trustix ];
          })
        ];
    };

  };
}

Effect

This will add all relevant services to your system (but not enable them) and adds packages to the pkgs set via an overlay.

Trustix - Builder setup with Colmena (flake based)

Up until now we have talked about components in isolation, let's take a stab at a real deployment using Colmena. Once we're done we will have:

  • A Trustix instance that others can subscribe to
  • A local post build hook that publishes builds

Most of the contents in this article will be applicable to other deployment systems too with only minimal changes, the biggest difference should be how they deal with key material (secrets).

Requisites / assumptions

  • You need a recent Nix with Flakes enabled
  • We start off in an empty git repository
  • Our domain name is demo.trustix.dev

Create keys

All Trustix build logs are first and foremost identified by their key pair, which will be the first thing we have to generate.

Let's start by generating a key pair for our log:

$ mkdir secrets
$ nix run github:nix-community/trustix#trustix -- generate-key --privkey secrets/log-priv --pubkey secrets/log-pub

Additionally logs are identified not just by their key, but how that key is used. If a key is used for multiple protocols (not just Nix) those logs will have a different ID. This ID is what subscribers use to indicate what they want to subscribe to.

To find out the log ID for the key pair you just generated: $ nix run github:nix-community/trustix#trustix -- print-log-id --protocol nix --pubkey $(cat secrets/log-pub)

Create a deployment

In your flake.nix put:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";

    trustix = {
      url = "github:nix-community/trustix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };
  outputs = { nixpkgs, flake-utils, trustix, ... }:
    let
      hostName = "demo.trustix.dev";
    in

    # Provide colmena
    (flake-utils.lib.eachDefaultSystem
      (system:
        let
          pkgs = nixpkgs.legacyPackages.${system};
        in
        {
          devShells.default = pkgs.mkShell {
            packages = [ pkgs.colmena ];
          };
        })
    ) // {

      # Our actual deployment
      colmena = {
        meta = {
          nixpkgs = import nixpkgs {
            system = "x86_64-linux";
            # Note that the overlay has to be applied manually when using Colmena
            overlays = [ trustix.overlays.default ];
          };
        };

        # Import the Trustix NixOS modules on all machines
        defaults = { pkgs, ... }: {
          imports = [
            trustix.nixosModules.trustix
          ];
        };

        "${hostName}" = {

          # Main Trustix daemon configuration
          services.trustix = {
            enable = true;

            # Signers & publishers are separate concepts as the same key
            # could potentially be used to publish multiple logs under different Trustix subprotocols.
            #
            # If you don't know what this means: While Trustix is build for Nix first, the core can be used for other ecosystems too, not just Nix.
            signers.my-signer = {
              type = "ed25519";
              ed25519.private-key-path = "/var/lib/my-trustix-key";
            };

            publishers = [
              {
                signer = "my-signer"; # Use the key configured above
                protocol = "nix"; # This publisher is using the Nix subprotocol

                # An arbitrary (string -> string) attrset with metadata about this log
                meta = {
                  upstream = "https://cache.nixos.org";
                };

                publicKey = {
                  type = "ed25519";
                  key = builtins.readFile ./secrets/log-pub;
                };
              }
            ];

          };

          # Enable the post build hook to push builds to the main Trustix daemon
          services.trustix-nix-build-hook = {
            enable = true;
            # The logID we got earlier
            publisher = "453016597475f45532e0a22a448ea7e0fb915e950d3c8930bfd23d962d73f9c1";
          };

          deployment = {
            targetHost = hostName;
            targetUser = "root";

            # We are using the Colmena secrets facility to upload the keys to the remote
            # without ending up world readable in the Nix store.
            keys = {
              my-trustix-key = {
                keyFile = ./secrets/log-priv;

                # This mode is too open for a real world deployment but we don't want to deal
                # with the complexities of secrets management here.
                permissions = "0644";
                destDir = "/var/lib";
              };
            };

          };
        };
      };
    };
}

Enter the development shell and deploy:

$ nix develop  # Pulls in Colmena via Flakes devShells
$ colmena apply  # Deploy

Spread your log

In the next chapter we will go over how to use this log from clients. The most important thing right now is to make a note of your public key and your domain name.

Trustix - Subscribing

This document walks you through how to subscribe to an already published binary cache.

Requisites

  • A local Trustix instance
  • A remote log's metadata
    • Public key
    • URL

Configuring

  • Add log(s) to your configuration.nix
{ pkgs, config, ... }:
{

  services.trustix = {
    enable = true;

    subscribers = [
      {
        protocol = "nix";
        publicKey = {
          type = "ed25519";
          key = "2uy8gNIOYEewTiV7iB7cUxBGpXxQtdlFepFoRvJTCJo=";
        };
      }
    ];

    # A remote can expose many logs and they are not neccesarily created by the remote in question
    remotes = [
      "https://demo.trustix.dev"
    ];

  };

}

Trustix - Binary cache setup

The easiest way to use Trustix is via the NixOS modules, though even they require some manual preparation in terms of generating keys.

This document walks you through how to configure your local system as a binary cache.

Requisites

We are assuming you have already followed the steps to set up one or more subscribers to your local Trustix instance.

  • Generate a public/private keypair to use with your local binary cache.
$ nix-store --generate-binary-cache-key binarycache.example.com cache-priv-key.pem cache-pub-key.pem
  • Move the keys somewhere persistent and safe Of course having keys around readable by anyone on the system is not a good idea, so we will move these somewhere safe. In this tutorial we are using /var/trustix/keys but you are free to use whatever you wish. A deployment tool like Colmena, Morph or NixOps is recommended to deal with secrets.

$ mv cache-priv-key.pem /var/trustix/keys/cache-priv-key.pem

Configuring

  • Add the binary cache to your configuration.nix
{ pkgs, config, ... }:
{

  # Enable the local binary cache server
  services.trustix-nix-cache = {
    enable = true;
    private-key = "/var/trustix/keys/cache-priv-key.pem";
    port = 9001;
  };

  # Configure Nix to use it
  nix = {
    binaryCaches = [
      "http://localhost:9001"
    ];
    binaryCachePublicKeys = [
      "binarycache.example.com://06YZJreoL8n9IdDlhnA3t7uJmHUI/rIIy3uO4FHRY="
    ];
  };

  # Configure your Trustix daemon with a decision making process on how
  # to determine if a build is trustworthy or not.
  #
  # In this case we configure it to have at least 2/3 majority to be substituted.
  #
  # Note that this configuration is incomplete and assumes you have already set up a subscriber.
  services.trustix = {
    deciders.nix = [
      {
        engine = "percentage";
        percentage.minimum = 66;
      }
    ];
  };

}

You are now all set up to use Trustix as a substitution method!

Project structure

Trustix is structured as a monorepo consisting of many subpackages:

The main package with all log functionality. This component is generic and doesn't know anything about any Nix or other package manager specifics.

The main documentation package that aggregates documentation from the various subpackages.

This is a supplemental daemon to the main Trustix daemon that layers some knowledge about Nix on top of the generic log functionality. It contains a post-build hook used to submit newly built packages to the logs, a binary cache HTTP interface and a development tool to submit already built closures.

This package is an implementation of a reproducibility tracker backed by logs.

Trustix-proto contains all shared protobuf definitions shared by various components, as well as generated Go libraries to interact with Trustix over it's RPC mechanism (gRPC).

Globally installed tooling

Trustix doesn't depend on much in the way of globally installed tools.

We do make two assumptions in regards to tooling managed outside of the repository though:

If you've read this far you likely already know Nix and what it is, so we won't go into any detail about this.

A shell extension to load directory local environments in a currently running shell and/or editor. This will load a present shell.nix/default.nix when used with the direnv rule use nix, which is the mode of operation we are using direnv in.

Getting started

All subpackages have their own shell environments which all needs to be explicitly whitelisted to be loaded. For convenience we have a Makefile target in the root of the project called direnv-allow. To whitelist all subpackages run:

$ make direnv-allow

Makefile structure

All components are using Makefile's as their development entry points for ease of use.

All standard Make targets are always implemented, even though they are no-ops in some cases where they don't make sense. For example a build step doesn't make sense for most Python code.

These are all standard make targets you can expect to find for any given package:

  • build

Builds the package.

  • test

Runs the tests for a given package.

  • lint

This target runs all configured linter steps.

  • format

This target checks the formatting of a given package.

  • develop

This target runs the package in development (watch) mode.

  • doc

This target builds documentation. This is mostly outputing markdown files in the relevant location for the trustix-doc package to compose.

Running the whole setup

To run individual components change directory to the relevant package and run:

$ make develop

This also works from the project root where it will start all packages in watch mode.

Quickly runing all tests

From the root directory run:

$ make all

Notes

Cryptographic keys for development is checked in to the repository for ease of use and a very quick getting started experience.

Protocol Documentation

Table of Contents

Top

api/api.proto

GetLogAuditProofRequest

Get log audit proof for a given tree

FieldTypeLabelDescription
LogIDstringrequiredLog identifier
Indexuint64requiredTree node index
TreeSizeuint64requiredTree size (proof reference)

GetLogConsistencyProofRequest

Get a consistency proof between two given log sizes

FieldTypeLabelDescription
LogIDstringrequiredLog identifier
FirstSizeuint64requiredFrom tree size
SecondSizeuint64requiredTo tree size

GetLogEntriesRequest

FieldTypeLabelDescription
LogIDstringrequiredLog identifier
Startuint64requiredGet entries from
Finishuint64requiredGet entries to

GetMapValueRequest

FieldTypeLabelDescription
LogIDstringrequiredLog identifier
KeybytesrequiredMap key
MapRootbytesrequiredMap root hash to derive proof from

KeyValuePair

FieldTypeLabelDescription
KeybytesrequiredMap key
ValuebytesrequiredMap value

Log

FieldTypeLabelDescription
LogIDstringrequired
ModeLog.LogModesrequired
Protocolstringrequired
SignerLogSignerrequired
MetaLog.MetaEntryrepeated

Log.MetaEntry

FieldTypeLabelDescription
keystringoptional
valuestringoptional

LogEntriesResponse

FieldTypeLabelDescription
Leavestrustix_schema.v1.LogLeafrepeated

LogHeadRequest

Request a signed head for a given log

FieldTypeLabelDescription
LogIDstringrequiredLog identifier

LogSigner

FieldTypeLabelDescription
KeyTypeLogSigner.KeyTypesrequired
Publicstringrequired

LogsRequest

FieldTypeLabelDescription
ProtocolsstringrepeatedAllow to filter logs response based on the protocol identifier

LogsResponse

FieldTypeLabelDescription
LogsLogrepeated

MapValueResponse

FieldTypeLabelDescription
ValuebytesrequiredNote that the Value field is actually a MapEntry but we need to return the marshaled version as that's what the proof is created from
ProofSparseCompactMerkleProofrequired

ProofResponse

FieldTypeLabelDescription
Proofbytesrepeated

SparseCompactMerkleProof

Sparse merkle tree proof

FieldTypeLabelDescription
SideNodesbytesrepeated
NonMembershipLeafDatabytesoptional
BitMaskbytesrequired
NumSideNodesuint64required

ValueRequest

FieldTypeLabelDescription
Digestbytesrequired

ValueResponse

FieldTypeLabelDescription
Valuebytesrequired

Log.LogModes

NameNumberDescription
Log0

LogSigner.KeyTypes

NameNumberDescription
ed255190

LogAPI

LogAPI is a logical grouping for RPC methods that are specific to a given log.

NodeAPI

NodeAPI is a logical grouping for RPC methods that are for the entire node rather than individual logs.

Method NameRequest TypeResponse TypeDescription
LogsLogsRequestLogsResponseGet a list of all logs published by this node
GetValueValueRequestValueResponseGet values by their content-address

Top

rpc/rpc.proto

DecideRequest

FieldTypeLabelDescription
Keybytesrequired
Protocolstringrequired

DecisionResponse

FieldTypeLabelDescription
DecisionLogValueDecisionrequired
MismatchesLogValueResponserepeatedNon-matches (hash mismatch)
MissesstringrepeatedFull misses (log ids missing log entry entirely)

EntriesResponse

FieldTypeLabelDescription
Keybytesrequired
EntriesEntriesResponse.EntriesEntryrepeated

EntriesResponse.EntriesEntry

FieldTypeLabelDescription
keystringoptional
valuetrustix_schema.v1.MapEntryoptional

FlushRequest

FieldTypeLabelDescription
LogIDstringrequired

FlushResponse

LogValueDecision

FieldTypeLabelDescription
LogIDsstringrepeated
Digestbytesrequired
Confidenceint32required
Valuebytesrequired

LogValueResponse

FieldTypeLabelDescription
LogIDstringrequired
Digestbytesrequired

SubmitRequest

FieldTypeLabelDescription
LogIDstringrequired
Itemstrustix_api.v1.KeyValuePairrepeated

SubmitResponse

FieldTypeLabelDescription
statusSubmitResponse.Statusrequired

SubmitResponse.Status

NameNumberDescription
OK0

LogRPC

RPCApi are "private" rpc methods for an instance related to a specific log. This should only be available to trusted parties.

RPCApi

RPCApi are "private" rpc methods for an instance. This should only be available to trusted parties.

Method NameRequest TypeResponse TypeDescription
Logs.trustix_api.v1.LogsRequest.trustix_api.v1.LogsResponseGet a list of all logs published/subscribed by this node
DecideDecideRequestDecisionResponseDecide on an output for key based on the configured decision method
GetValue.trustix_api.v1.ValueRequest.trustix_api.v1.ValueResponseGet values by their content-address

Top

schema/loghead.proto

LogHead

Log

FieldTypeLabelDescription
LogRootbytesrequired
TreeSizeuint64required
MapRootbytesrequired
MHRootbytesrequired
MHTreeSizeuint64required
SignaturebytesrequiredAggregate signature

Top

schema/logleaf.proto

LogLeaf

Leaf value of a merkle tree

FieldTypeLabelDescription
Keybytesoptional
ValueDigestbytesoptional
LeafDigestbytesrequired

Top

schema/mapentry.proto

MapEntry

FieldTypeLabelDescription
DigestbytesrequiredValue digest of tree node
Indexuint64requiredIndex of value in log

Top

schema/queue.proto

SubmitQueue

This type is internal only and not guaranteed stable

FieldTypeLabelDescription
Minuint64requiredMin is the current (last popped) ID
Maxuint64requiredMax is the last written item

Scalar Value Types

.proto TypeNotesC++JavaPythonGoC#PHPRuby
doubledoubledoublefloatfloat64doublefloatFloat
floatfloatfloatfloatfloat32floatfloatFloat
int32Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.int32intintint32intintegerBignum or Fixnum (as required)
int64Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.int64longint/longint64longinteger/stringBignum
uint32Uses variable-length encoding.uint32intint/longuint32uintintegerBignum or Fixnum (as required)
uint64Uses variable-length encoding.uint64longint/longuint64ulonginteger/stringBignum or Fixnum (as required)
sint32Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.int32intintint32intintegerBignum or Fixnum (as required)
sint64Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.int64longint/longint64longinteger/stringBignum
fixed32Always four bytes. More efficient than uint32 if values are often greater than 2^28.uint32intintuint32uintintegerBignum or Fixnum (as required)
fixed64Always eight bytes. More efficient than uint64 if values are often greater than 2^56.uint64longint/longuint64ulonginteger/stringBignum
sfixed32Always four bytes.int32intintint32intintegerBignum or Fixnum (as required)
sfixed64Always eight bytes.int64longint/longint64longinteger/stringBignum
boolboolbooleanbooleanboolboolbooleanTrueClass/FalseClass
stringA string must always contain UTF-8 encoded or 7-bit ASCII text.stringStringstr/unicodestringstringstringString (UTF-8)
bytesMay contain any arbitrary sequence of bytes.stringByteStringstr[]byteByteStringstringString (ASCII-8BIT)