Welcome

Welcome to Mainmatter’s Advanced Rust workshop!

You know Rust. You have shipped something real, and cargo clippy rarely surprises you any more. This course is about the next step: designing Rust APIs that other people, including future you, cannot get wrong. Lifetimes are assumed, but elision, non-lexical borrows and higher-ranked bounds are covered where they come up, because API design keeps running into them.

The premise is simple. Every rule your domain has is enforced somewhere: in a comment, in a code review, in a runtime check, or in the type system. The further left you push it, the cheaper it gets. Rust gives you an unusually powerful set of tools for pushing rules all the way into the compiler, and most Rust code uses a small fraction of them.

What you will build

Rather than a series of unrelated puzzles, you will build one library from nothing: minidb, a small embedded key-value store, in the spirit of redb or sled.

It starts as the kind of code anyone would write in an afternoon: HashMaps, &str parameters, and Option everywhere. By the end of the day, it will be a library where forgetting to commit a transaction is a compile-time error, where a key from one store cannot be used with another, and where the only way to hold a value you should not have is to write unsafe.

Each exercise is a complete, standalone copy of the library at that point in its evolution. You never have to carry a broken state forward.

Methodology

This is a hands-on workshop. Expect to spend at least half the day writing code.

Exercises are test-driven: each one ships a set of tests that describe the API you are supposed to build, and your job is to make them pass. Some tests are compile_fail doctests, which assert that certain code must not compile. Those are the interesting ones: in this course, a compiler error is frequently the feature.

Some exercises hand you a todo!() to replace. In others there is nothing to replace: the exercise text tells you what to add, and the tests show you its shape. Sometimes a single line is enough, sometimes you will need to reshape a whole type.

⚠️ Do not modify the tests. They are the specification. Change the code under test, not the test.

If you get stuck for more than ten minutes, grab a trainer. We are here to help. You can also find solutions to all exercises in the solutions branch of this repository.

Setup

You need a recent stable Rust toolchain:

rustup update stable

Clone the repository and create a branch to work on:

git clone https://github.com/mainmatter/advanced-rust-workshop
cd advanced-rust-workshop
git checkout -b my-solutions

Then install the workshop runner, the tool that walks you through the exercises:

cargo install --locked workshop-runner

The workflow

From the root of the repository, run:

wr

wr finds the first exercise you have not solved yet, compiles it, runs its tests, and either congratulates you or shows you what went wrong. It will not let you move on until the current exercise compiles and passes. Solve it, run wr again, and it opens the next one.

That is the whole loop. Let’s make sure it works.

Exercise

The exercise for this section is located in 00_intro/00_welcome

Names and docs

Before any of the clever type-level machinery, an API is a list of names and a page of prose. Both are read far more often than the code behind them, and both are the parts nobody budgets time for.

This chapter is short and deliberately unglamorous. It is here because the rest of the day assumes it: a Key type is only an improvement if it is called Key, and a parse method that returns Result is only usable if its failure modes are written down somewhere.

The starting point

#![allow(unused)]
fn main() {
store.set_value("users", "42", "Alice");
store.get_value("users", "42");
store.is_has_bucket("users");
store.get_count();
store.as_map();
}

Every one of those works. Every one of them is called something you would have to look up, because each name was invented in isolation rather than borrowed from the vocabulary Rust programmers already know.

Two ideas

Everything in this chapter follows from two claims.

A name is a promise about cost and ownership. as_bytes promises a cheap borrow. to_owned admits an allocation. into_bytes warns that the receiver is consumed. A Rust programmer reading as_map() has already concluded that it is free, and will call it in a loop.

A doc comment is for the caller, and the caller cannot see the body. So it documents what the function is for, what it promises, and what it does to them on a bad day. Not how it works: they can read that, and if they cannot, the comment will be wrong within a release anyway.

Exercise

The exercise for this section is located in 01_api_design/00_intro

Naming conventions

Rust’s naming conventions are not a style preference. They are a compression scheme: a caller who knows them can predict the cost, the ownership and the failure mode of a method from its name alone, without opening the documentation.

The Rust API Guidelines list them all. What follows is the subset that carries real information.

Cost and ownership: as_, to_, into_

The most valuable convention in the language, and the most frequently broken.

A method called as_map that clones a HashMap is not a naming quibble. It is a lie about performance, and it will be called inside a loop by someone who trusted it.

Constructors

• new is the obvious constructor, and takes no options nobody would expect.
• with_capacity, with_config: a constructor with one salient parameter.
• from_* for conversions that cannot fail, try_from/parse for conversions that can.
• Default::default when “empty” is meaningful, and prefer it to new() taking no arguments only when a default genuinely exists.

Borrow the standard library’s vocabulary

Collections have already settled this, and every Rust programmer has already learned it:

set_value, delete, size and has_key all work and all cost the reader a lookup. Reach for the word the standard library already uses, even when your own word is marginally more accurate.

Prefixes that carry nothing

• get_ is noise. store.get_count() says “get” twice, once in the verb and once in the fact that it is a method returning a value. The standard library uses get alone, for lookups that can fail.
• is_ is for adjectives: is_empty, is_ascii. Possession is contains_ or has_. is_has_bucket is the sound of two conventions colliding.
• _mut pairs with a borrowed getter of the same name: get/get_mut, iter/iter_mut. If you have a _mut method with no partner, one of the two is misnamed.

Where there is a len, there is an is_empty

A small one, and Clippy will tell you off for it. If your type has a len, callers will write x.len() == 0, which is both noisier and, for lazy or computed collections, potentially slower. Give them is_empty.

The test

For each name, ask: could a competent Rust programmer who has not read this file guess what it returns, whether it allocates, and whether it takes ownership? If the answer is no for any of the three, the name is doing less work than it could.

Exercise

The exercise for this section is located in 01_api_design/01_naming

Doc comments

A doc comment is written for someone who can see your signature and cannot see your body. That single constraint decides almost everything about what belongs in one.

Anatomy

#![allow(unused)]
fn main() {
/// Inserts a value, returning the value it replaced, if any.
///
/// Values are stored per bucket, so the same key in two buckets is two values.
///
/// # Errors
///
/// Returns [`NameError`] if the key is empty, over 64 bytes, or contains
/// characters outside `[A-Za-z0-9._/-]`.
///
/// # Examples
///
/// ```
/// let mut store = Store::new();
/// assert_eq!(store.insert("users", "42", "Alice"), None);
/// ```
pub fn insert(&mut self, bucket: &str, key: &str, value: &str) -> Option<String>
}

• The summary line is one sentence, in the third person, ending with a full stop. It shows up in the type’s method list, so it has to stand alone: rustdoc will show it next to forty others.
• The body is for what the caller cannot infer: the surprising bit, the invariant, the relationship to the neighbouring method.
• # Errors lists the conditions, not just the type. “Returns Err” tells a caller nothing they could not read off the signature.
• # Panics is the one section people skip and the one that costs them. If your function can panic, that is part of its contract, and a caller who does not know cannot defend against it.
• # Examples comes last, and is the only part of the whole comment that cannot silently rot.

What and why, not how

The comment that hurts is the one that restates the body:

#![allow(unused)]
fn main() {
/// Loops over the buckets and sums their lengths.
pub fn len(&self) -> usize
}

The caller does not care, and the sentence becomes false the moment someone caches the count. Write instead what it is for and what it costs:

#![allow(unused)]
fn main() {
/// Returns the number of values across every bucket.
///
/// This walks every bucket, so it is O(number of buckets), not O(1).
pub fn len(&self) -> usize
}

Examples are tests

cargo test compiles and runs every example in every doc comment. This is worth more than it sounds:

• an example that no longer compiles is a failing test, so your documentation cannot drift out of sync with your API without someone noticing;
• an example is the only part of the docs that is proven to be true;
• writing one is the fastest way to discover that your own API is annoying to call.

The corollary is that a rotted example is worse than no example, because it is a test that nobody ran. If a doc example does not survive a rename, either the rename or the example was wrong.

Hidden lines starting with # let you keep an example short without making it a lie:

#![allow(unused)]
fn main() {
/// ```
/// # use minidb::Store;
/// let mut store = Store::new();
/// store.insert("users", "42", "Alice");
/// ```
}

The use runs, so the example is real, but it does not clutter the rendered page.

Link to your own types

Square brackets make intra-doc links: [NameError] resolves to the type, and rustdoc will warn you if it stops resolving. #![deny(rustdoc::broken_intra_doc_links)] turns that warning into a build failure, which is how you keep the cross-references honest.

How much is enough

Not every method deserves four sections. A reasonable floor:

• every public item has a summary line, and #![deny(missing_docs)] is how you make that true rather than aspirational;
• anything returning Result has # Errors;
• anything that can panic has # Panics;
• anything whose use is non-obvious has an example.

Everything beyond that is a judgement call, and the failure mode is not “too little documentation” but prose that repeats the signature in longer words.

Exercise

The exercise for this section is located in 01_api_design/02_doc_comments

The newtype pattern

Here is the store from the previous chapter, trimmed to the four operations we will spend the rest of the day evolving:

#![allow(unused)]
fn main() {
pub struct Store {
    buckets: HashMap<String, HashMap<String, String>>,
}

impl Store {
    pub fn insert(&mut self, bucket: &str, key: &str, value: &str) -> Option<String> { /* ... */ }
    pub fn get(&self, bucket: &str, key: &str) -> Option<&str> { /* ... */ }
    pub fn remove(&mut self, bucket: &str, key: &str) -> Option<String> { /* ... */ }
}
}

There is nothing wrong with this code. It compiles, it is easy to read, it does what it says. It is also the version of the library that will generate support tickets for the next two years.

Three strings walk into a function

insert takes three &str parameters. To the compiler they are interchangeable. To the domain they are nothing of the sort: the first names a partition, the second names a value inside it, the third is the value. Get the order wrong and the compiler waves you through:

#![allow(unused)]
fn main() {
let mut store = Store::new();
store.insert("users", "42", "Alice");

// Later, in a different file, written by a different person, at 17:45 on a Friday:
store.get("42", "users")  // => None
}

No panic. No error. Just None, which the caller will dutifully interpret as “no such user”, because that is what None means everywhere else in this API.

The information needed to catch this exists. It is in the parameter names, in the doc comment, and in the head of whoever wrote insert. The only place it is not is in the type system, which is the one place the compiler can read.

What this chapter is about

Three steps, each one a small, unglamorous change:

1. Give distinct things distinct types. The compiler cannot help you tell bucket from key until they stop being the same type.
2. Parse, don’t validate. A type that can only be built from valid input turns “is this key legal?” from a question you keep asking into a question you answered once.
3. Close the back door. An invariant enforced by a constructor that anyone can bypass is a convention, not an invariant.

None of this is clever. That is the point: it is the cheapest correctness you will ever buy in Rust, and most codebases leave it on the table.

Exercise

The exercise for this section is located in 02_newtype/00_intro

Semantic confusion

A newtype is a tuple struct with exactly one field:

#![allow(unused)]
fn main() {
pub struct Bucket(pub String);
pub struct Key(pub String);
}

That is the whole pattern. Bucket and Key hold the same data as before and behave the same at runtime, but they are now different types, and Store::get(&self, bucket: &Bucket, key: &Key) can no longer be called with its arguments the wrong way round.

The bug from the previous section stops being a bug you find in production and becomes a bug you find while typing.

It really is free

A newtype with a single field has the same size, alignment and representation as the field itself. There is no wrapper object, no indirection, no allocation. After monomorphisation and inlining, code that passes a Key around compiles to exactly the code that passed a String around.

You pay in source code, not in cycles: the wrapping, the unwrapping, and the trait impls you now have to write yourself. That last cost is real, and we will spend the next chapter on it.

A type alias is not a newtype

This is the tempting shortcut, and it does nothing:

#![allow(unused)]
fn main() {
type Key = String;
type Bucket = String;
}

A type alias introduces a new name, not a new type. Key and Bucket are both still String, they are still interchangeable, and the swapped-argument bug still compiles. Aliases are for shortening Result<T, std::io::Error>, not for encoding meaning.

The same goes for the other near miss:

#![allow(unused)]
fn main() {
pub struct Key(pub String);

fn get(bucket: &str, key: &str)  // still takes two `&str`
}

Defining the type is only half the work. The type has to reach the signature.

When not to do it

Newtypes have a cost at every boundary they cross, so they are not free in a codebase, only on the CPU. A rough rule:

• Wrap it when the value has domain meaning that the underlying type does not capture (Key, UserId, Celsius, Bytes), when confusing it with its neighbours is plausible, or when it will grow an invariant later. Whether it will grow an invariant later is easier to predict than you think.
• Leave it alone when the underlying type already says everything (fn len(&self) -> usize), or when the value is genuinely just data passing through.

The signal to watch for is two or more parameters of the same type sitting next to each other in a signature. It is not a proof of a problem, but it is where the problems live.

Exercise

The exercise for this section is located in 02_newtype/01_semantic_confusion

Parse, don’t validate

Key is a distinct type now, but it still accepts anything a String accepts: the empty string, a megabyte of user-supplied bytes, a newline, a null byte. Somewhere downstream, something will care.

The usual answer is a validation function:

#![allow(unused)]
fn main() {
fn is_valid_key(raw: &str) -> bool { /* ... */ }
}

and a rule that everyone calls it before doing anything interesting. This has a specific failure mode: is_valid_key returns a bool, and a bool is forgotten the instant it goes out of scope. The compiler has no memory that you checked. Three functions later, someone checks again, defensively, because they cannot tell whether anyone already did. Six functions later, nobody checks at all.

The alternative

Move the check into the only path that can produce the type:

#![allow(unused)]
fn main() {
impl Key {
    pub fn parse(raw: &str) -> Result<Self, NameError> { /* ... */ }
}
}

Now the validation returns evidence rather than a verdict. A Key value is proof that the check ran and passed, and that proof travels with the value, through function calls, into structs, across threads. Downstream code does not re-check, because there is nothing left to check.

This is the difference between validating (asking a question and throwing away the answer) and parsing (turning a weakly typed input into a strongly typed output, once, at the edge).

The shape generalises far beyond newtypes:

Where the boundary goes

Parse once, at the edge: where bytes arrive from a socket, a config file, a CLI argument, a database row. Everything inside the edge speaks in domain types and never sees a raw &str again.

Put differently: the raw type should have the shortest possible lifetime in your program. &str comes in, Key comes out, and the &str is gone.

Designing the error

parse returns a Result, so you need an error type, and you may as well make it a good one:

#![allow(unused)]
fn main() {
pub enum NameError {
    Empty,
    TooLong { length: usize },
    InvalidCharacter { character: char, index: usize },
}
}

Three things worth copying here:

• One variant per way of failing. A single NameError::Invalid(String) would compile, but callers could not distinguish “too long” from “contains a slash” without parsing your prose. Variants are matchable, strings are not.
• Carry the context the caller needs to act. TooLong without a length forces the reader to go find the limit. InvalidCharacter without an index forces them to hunt for the offending byte.
• Do not carry the input. The caller has it: they passed it to you. Cloning it into the error is a needless allocation on the error path, and a way to leak user data into logs.

An error is an API, and it is the part of the API your users meet on their worst day.

Costs, honestly

The type only helps if it is expensive to bypass and cheap to use. That means parse should be the only way in, which is the next section, and it means the type will eventually need Display, Debug, PartialEq, Hash and friends before anyone can comfortably put it in a HashMap or an error message, which is the next chapter.

Exercise

The exercise for this section is located in 02_newtype/02_parse_dont_validate

Is it encapsulated?

Key::parse rejects invalid keys. This still compiles:

#![allow(unused)]
fn main() {
let key = Key(String::new());
}

And so does this:

#![allow(unused)]
fn main() {
let mut key = Key::parse("users/42").unwrap();
key.0.push('\n');
}

The invariant lasted exactly as long as it took someone to reach around it. An invariant enforced by a constructor that anyone can bypass is a convention, and conventions do not survive contact with a deadline.

The rule

An invariant holds only if every path that can construct or mutate the value goes through code that checks it. In Rust that means the field is private, and the module boundary does the rest:

#![allow(unused)]
fn main() {
pub struct Key(String);

impl Key {
    pub fn parse(raw: &str) -> Result<Self, NameError> { /* ... */ }

    pub fn as_str(&self) -> &str {
        &self.0
    }

    pub fn into_inner(self) -> String {
        self.0
    }
}
}

Note what changed and what did not. Inside the defining module, self.0 still works, which is why Store needs no rewrite. Outside it, Key has exactly one door, and parse is standing in it.

Giving data back

Two accessors, and the difference between them is worth being deliberate about:

• as_str(&self) -> &str hands out a read-only view. The caller can look, cannot touch, and the Key stays valid.
• into_inner(self) -> String hands over the data and consumes the Key. The caller can now do anything at all to that String, and it does not matter: it is not a Key any more. The only way back is parse.

The one you should not write is as_mut(&mut self) -> &mut String. It hands out unrestricted mutation of a value that is still a Key, which is the same hole as a public field with more ceremony. If callers need to modify a key, give them an operation that preserves the invariant, or make them go through parse again.

Do not reach for Deref

Sooner or later somebody suggests this:

#![allow(unused)]
fn main() {
impl Deref for Key {
    type Target = String;
    fn deref(&self) -> &String { &self.0 }
}
}

It is seductive: every String method appears on Key for free, and the wrapping stops feeling like work. It is also a mistake for a newtype like this one:

• it re-exports the entire String API as though it were Key’s API, including methods that make no sense for a key and methods that will be added to String in future releases;
• with DerefMut, it hands back exactly the mutation hole you just closed;
• deref coercion is implicit, so Key starts silently coercing to String in ways that undo the type distinction you built the newtype for.

Deref is for smart pointers, types whose whole purpose is to stand in for something else: Box<T>, Rc<T>, MutexGuard<T>. A newtype is the opposite. Its purpose is to not be the thing it wraps.

Implement the handful of methods your callers actually need. It is more typing and a smaller API, and a smaller API is the product.

The hole nobody notices

Once your newtype has an invariant, every trait that can construct it from the outside is a new door. The most common one is serde:

#![allow(unused)]
fn main() {
#[derive(Deserialize)]  // reads the raw string straight into the field
pub struct Key(String);
}

#[derive(Deserialize)] bypasses parse entirely, which means a JSON payload can hand you a Key that parse would have rejected. If a type has an invariant, its Deserialize impl has to go through the same door as everyone else, using #[serde(try_from = "String")] or a hand-written impl.

The same question is worth asking of any trait you derive on a type with an invariant: can this construct a value, or mutate one, without passing my check? Default frequently can. So can a careless From.

Exercise

The exercise for this section is located in 02_newtype/03_encapsulation

Common traits

The last chapter took a String and wrapped it. Wrapping does not only add: it takes away.

A String can be printed, compared, sorted, cloned and used as a key in a HashMap. Key(String) can do none of those, because a newtype starts life with no traits at all. Three lines that used to work:

#![allow(unused)]
fn main() {
println!("{key:?}");                          // Key does not implement Debug
let same = a == b;                            // no PartialEq
map.insert(key, "Alice");                     // no Hash, no Eq
}

This is the bill for the newtype pattern, and it is the reason people abandon it halfway. It arrives as a compiler error at the exact moment you are trying to do something else.

The good news

Most of the bill is paid with one line:

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct Key(String);
}

So this chapter is short. It is about the four decisions in it that derive cannot make for you:

• Debug on a type that holds user data. Deriving it is a decision about what ends up in your logs.
• Hash and Eq together. They have a contract, and hand-writing one of the two breaks it silently.
• Clone versus Copy. One of them changes how your API feels to use.
• From and TryFrom. The generic entry points to the parsing you already wrote.

Everything else on the list, Display, PartialOrd, Ord, is either a derive or a five-line impl, and we will not spend the morning on it.

Exercise

The exercise for this section is located in 03_common_traits/00_intro

Debug and Display

Two traits, two audiences, and the distinction is worth getting right because it decides which one you reach for at three in the morning.

Debug is for programmers. It is {:?}, it shows structure, it is allowed to be ugly, and it should round-trip your mental model of the value. Derive it on virtually everything.

Display is for the people using your program. It is {}, there is no derive, and the absence of a derive is the point: a human-facing rendering is a decision, not a projection of your field names.

A rule of thumb that survives contact with reality: if you cannot say who reads the output, you want Debug.

#![allow(unused)]
fn main() {
use std::fmt::{self, Display, Formatter};

#[derive(Debug)]
pub struct Key(String);          // Key("users/42")

impl Display for NameError {     // "key is empty"
    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result { /* ... */ }
}
}

Debug is a security surface

Here is the part people learn the hard way.

Debug output does not stay where you put it. It ends up in log lines, in panic messages, in unwrap() failures, in test output pasted into a ticket, in an error report shipped to a third-party service. Anything reachable by Debug from a struct you log is, effectively, logged.

So for a type holding data you do not own, deriving Debug is a decision:

#![allow(unused)]
fn main() {
pub struct Value(String);

impl Debug for Value {
    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
        write!(f, "Value(<redacted, {} bytes>)", self.0.len())
    }
}
}

The '_ in Formatter<'_> is the anonymous lifetime: there is a borrow inside that type, and naming it would buy nothing. Chapter 4 says what it is short for.

The length is not an accident. A redaction that shows nothing at all makes debugging genuinely harder, and people respond by removing it. Showing the length distinguishes an empty value from a truncated one and from a value that is there but wrong, which covers most of what you actually need, and tells a reader of your logs nothing they can use.

This is the same reasoning behind secrecy’s Secret<T>, and behind std’s decision that OsStr and Path print quoted and escaped rather than raw.

The failure mode to watch for is indirect: a type with a careful Debug impl held inside a struct that derives Debug is safe, because the derive calls your impl. A type with a careful impl whose data is also reachable through some other public accessor that gets logged is not. Redaction protects a field, not a value.

Writing the impls

For the common cases you rarely need to touch a Formatter directly:

#![allow(unused)]
fn main() {
impl Debug for Config {
    fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
        f.debug_struct("Config")
            .field("url", &self.url)
            .field("token", &"<redacted>")
            .finish_non_exhaustive()
    }
}
}

debug_struct, debug_tuple, debug_list and debug_map handle the formatting flags for you, so {:#?} still pretty-prints. finish_non_exhaustive renders the .. that tells a reader you left something out on purpose.

One rule for Display: never write \n into it. The caller decides about layout, and a Display impl that emits a newline is unusable inside a larger message.

Exercise

The exercise for this section is located in 03_common_traits/01_debug

Eq, Hash and conversions

The contract

Hash and Eq are not independent traits. They come with a promise:

If a == b, then hash(a) == hash(b).

HashMap relies on it completely. A lookup hashes the key to find a bucket, then compares for equality inside that bucket. If two equal values hash differently, the second one lands in a different bucket and the map simply does not find it. There is no panic and no warning: entries go missing, and the bug reproduces on one machine in ten because it depends on the hasher’s random seed.

The classic way to break it is to make equality cleverer than hashing:

#![allow(unused)]
fn main() {
#[derive(Hash)]                                   // hashes the exact bytes
pub struct Key(String);

impl PartialEq for Key {                          // compares case-insensitively
    fn eq(&self, other: &Self) -> bool {
        self.0.eq_ignore_ascii_case(&other.0)
    }
}
}

Key("A") == Key("a") is now true, and their hashes differ. Every HashMap<Key, _> in the program is quietly broken.

The rule that follows is blunt and worth following: derive Hash and PartialEq together, or write both by hand. Never one of each. If you want case-insensitive keys, normalise in parse so that the stored bytes are already canonical, and let the derives stay honest.

Eq on top of PartialEq is a marker: it promises that equality is reflexive, so a == a always holds. Floats do not qualify, which is why f64 has PartialEq and not Eq, and why a struct containing one cannot be a HashMap key.

Clone and Copy

Clone is an explicit, possibly expensive duplicate. Copy says the type is duplicated implicitly on every assignment, which is only appropriate for small, plain data that has no invariant about uniqueness.

Key holds a String, so Copy is not even an option. But the interesting question is the one you face when it is possible: adding Copy to a public type is a permanent commitment, because removing it later breaks every caller who relied on a value still being usable after they passed it somewhere. Clone can be added and removed with far less drama.

From, TryFrom and parse

You already wrote the conversion in chapter 2. TryFrom is how you make it discoverable:

#![allow(unused)]
fn main() {
impl TryFrom<&str> for Key {
    type Error = NameError;

    fn try_from(raw: &str) -> Result<Self, Self::Error> {
        Self::parse(raw)
    }
}
}

Two things follow for free, which is the whole reason to bother:

• TryInto comes with it. A blanket impl in std gives you raw.try_into() on the calling side, and it is the form that works when the target type is inferred rather than named.
• Generic code can call it. A function taking T: TryFrom<&str> works with your type without knowing it exists. This is how clap, serde and friends accept domain types.

Keep the inherent parse as well. It is discoverable in the rustdoc method list, it does not need the trait in scope, and it gives a better error message when someone gets the types wrong.

From is the infallible sibling, and there is one rule about it: From must never panic. If the conversion can fail, it is TryFrom. An impl From<&str> for Key that unwraps internally is a landmine, because callers reasonably assume .into() cannot blow up.

The other direction is free and worth adding: impl From<Key> for String gives you .into() where into_inner() reads awkwardly, and costs nothing since you have the method already.

The rest of the list

For completeness, and none of it deserves more than a line here:

• Display: covered in the previous section. No derive, and that is deliberate.
• PartialOrd and Ord: derive them if the field order happens to be the order you want, which for a single-field newtype it always is. Same contract trap as Hash: if you hand-write Ord, make it consistent with Eq.
• Default: only if “empty” is a meaningful value. For a type with an invariant it usually is not, and #[derive(Default)] on a validated newtype is another door around parse.
• Serialize and Deserialize: covered in chapter 2. The derive on Deserialize bypasses your constructor, so use #[serde(try_from = "String")].

Exercise

The exercise for this section is located in 03_common_traits/02_hash_eq

Ownership, borrowing and lifetimes

Most Rust programmers meet the borrow checker as an obstacle: the thing that rejects the code they wanted to write. This chapter is about the other half of the deal, which is that the same machinery enforces your rules, for free, if you shape your signatures to ask for it. Before that pays off it is worth being exact about what the rules are, because the next chapter is built on them.

Two kinds of reference

#![allow(unused)]
fn main() {
&Store        // shared
&mut Store    // exclusive
}

A shared reference may be one of many. An exclusive reference is the only reference to that value for as long as it exists.

Calling them “immutable” and “mutable” is the usual shorthand, and it is wrong in a way that matters: &Cell<T>, &Mutex<T> and &AtomicUsize all let you change the value through a shared reference. What &mut promises is not the right to write, it is the absence of anybody else. Permission to write is what that promise buys.

One rule

Any number of shared references, or exactly one exclusive reference. Never both.

Aliasing XOR mutation, and almost every borrow error is this rule saying no. Holding a value borrowed out of the store and then changing the store asks for both at once:

#![allow(unused)]
fn main() {
let alice = store.get(&users, &id);

store.insert(&users, &id, Value::new("Bob"));   // error[E0502]

println!("{alice:?}");
}

Two exclusive references are the same rule from the other side, and give E0499.

The rule is not bureaucracy. It is what makes iterator invalidation impossible, what makes data races impossible without unsafe, and what lets the compiler assume that a value behind &mut cannot change underneath it.

A borrow lasts a region

Every reference is valid over a region of the code, and a lifetime is the name of that region. The region is not the enclosing block: it ends at the reference’s last use. Moving one line makes the example above compile:

#![allow(unused)]
fn main() {
let alice = store.get(&users, &id).map(Value::as_str);
assert_eq!(alice, Some("Alice"));       // last use of the borrow

store.insert(&users, &id, Value::new("Bob"));   // fine
}

Lexical, and then not

That is younger than the language. Before Rust 2018 a borrow really did run to the end of its block, and the version above had to be written with an extra scope around the read purely to end the borrow early. Non-lexical lifetimes replaced the block with the actual span of use, and a large class of “the borrow checker is wrong” complaints disappeared with it.

One wrinkle is worth carrying into the next chapter. Every value is dropped at the end of its scope, Drop impl or not; the drop point is the same either way. What a Drop impl changes is that the compiler counts that drop as a use, because drop takes &mut self and could read whatever the value borrows. So the borrow cannot end early, and runs to the drop point instead. A type with no Drop impl of its own inherits this from any field that has one. That is exactly what a guard wants.

The ones you never write

You seldom spell a lifetime out, because most are inferred. Store::get is declared

#![allow(unused)]
fn main() {
pub fn get(&self, bucket: &Bucket, key: &Key) -> Option<&Value>
}

and means

#![allow(unused)]
fn main() {
pub fn get<'s, 'b, 'k>(&'s self, bucket: &'b Bucket, key: &'k Key) -> Option<&'s Value>
}

Three rules produce that, and they are the whole of lifetime elision for functions:

1. every elided input lifetime becomes its own parameter;
2. if there is exactly one input lifetime, every elided output gets it;
3. if one of the inputs is &self or &mut self, every elided output gets self’s lifetime instead.

There is a third spelling, '_, the anonymous lifetime: there is a borrow in this type and it is not worth naming. You have written one already, in Formatter<'_> in chapter 3.

None of these rules apply to a struct that holds a reference. There you write the lifetime yourself, which is the first thing chapter 5 does.

The receiver is the API

Every method makes a claim about what the caller may do afterwards, and the receiver is where the claim is written down:

Reaching for self is the move people forget. Any time an operation genuinely ends the life of a thing (commit, close, finish, build, into_inner), taking self turns “please do not use this afterwards” from a doc comment into a compiler error.

Where this chapter goes

Two exercises, and they are the two halves of the same coin.

1. Aliasing XOR mutability, from the inside: the rule above is the same rule that stops you mutating a collection while walking it. Meeting it head-on and learning the standard ways through is most of what “fighting the borrow checker” turns out to be.
2. Ownership in signatures: minidb currently borrows things it immediately clones. That is a cost the caller pays and cannot see, and the fix is to say what you mean.

Exercise

The exercise for this section is located in 04_borrowing/00_intro

Aliasing XOR mutability

The rule from the previous section, once more, because this is where you meet it from the inside:

At any moment, a value may have either any number of shared references or exactly one exclusive reference. Never both.

Reading is not what the rule is about, and neither is writing. It is about how many ways there are to reach the value at once. Everything the next chapter gets for free rests on this, and so does the error you are about to provoke.

The same rule is also the one that stops you doing this:

#![allow(unused)]
fn main() {
for (bucket, values) in &self.buckets {
    for (key, value) in values {
        if !predicate(bucket, key, value) {
            self.remove(bucket, key);        // error[E0502]
        }
    }
}
}

Why this is not the compiler being difficult

Removing an entry can make a HashMap reallocate its table, which moves every entry. The iterator is holding a pointer into the old table. In C++ this is undefined behaviour with a name, iterator invalidation, and it is a reliable source of exploitable bugs. In Java and Python it is a runtime exception, checked with a modification counter on every step. In Rust it is a compile error and costs nothing at runtime.

The point worth taking away is that this is not a special case about collections. It is one rule, applied uniformly, and it is the same rule that gave minidb transaction isolation.

The three ways through

Two passes. Collect the decisions first, then act on them. Always works, costs an allocation, and is the answer when the logic is complicated:

#![allow(unused)]
fn main() {
let doomed = self.buckets.iter()
    .flat_map(|(bucket, values)| values.keys().map(move |key| (bucket.clone(), key.clone())))
    .filter(|(bucket, key)| !predicate(bucket, key, /* ... */))
    .collect::<Vec<_>>();

for (bucket, key) in doomed {
    self.remove(&bucket, &key);
}
}

The method written from the inside. retain can do in one pass what you cannot do from outside, because inside the implementation the borrow is not a problem:

#![allow(unused)]
fn main() {
self.buckets.retain(|bucket, values| {
    values.retain(|key, value| predicate(bucket, key, value));
    !values.is_empty()
});
}

Worth internalising as a habit: when the borrow checker rejects a loop that mutates, check whether the collection already has a method for exactly that shape. retain, retain_mut, drain, extract_if, entry and iter_mut between them cover most of it.

Indices instead of references. An index is not a borrow, so the borrow ends between iterations:

#![allow(unused)]
fn main() {
for i in 0..items.len() {
    if !keep(&items[i]) {
        items.remove(i);      // careful: indices shift
    }
}
}

This is the escape hatch that scales to graphs, where nodes refer to each other by usize into an arena rather than by reference. It also gives up everything the borrow checker was doing for you: an index into the wrong collection is a logic bug the compiler cannot see.

When you genuinely need two exclusive borrows

Sometimes the requirement is real: two exclusive references into the same collection, at once, to different elements. The compiler cannot prove the elements are distinct, so it refuses.

The standard library solves this by providing the operations from the inside, where unsafe can be used once and reviewed carefully:

#![allow(unused)]
fn main() {
let (left, right) = slice.split_at_mut(mid);
let [a, b] = map.get_disjoint_mut(["x", "y"]);
}

RefCell is the other answer: move the check to runtime, and accept a panic if you get it wrong. That is a real trade rather than a defeat, but reach for it after the two above.

The meta-lesson: the borrow checker is deliberately conservative, so the seams where a safe API is built on unsafe are exactly the places where a rule that is true cannot be proved. split_at_mut exists because that proof is impossible in general and trivial in that one case.

Exercise

The exercise for this section is located in 04_borrowing/01_retain

Ownership in signatures

Here is minidb’s insert, as you have been using it all day:

#![allow(unused)]
fn main() {
pub fn insert(&mut self, bucket: &Bucket, key: &Key, value: Value) -> Option<Value> {
    self.buckets
        .entry(bucket.clone())
        .or_default()
        .insert(key.clone(), value)
}
}

It borrows, and then immediately clones, because a map has to own its keys. The signature says “lend me these”; the body says “mine now”. Two allocations per insert that the caller cannot see in the signature, cannot avoid, and is not told about.

This is the pattern to learn to spot: a &T parameter that is cloned in the body is a hidden cost. It looks polite and it is not.

Say what you need

The rule is boring and worth following: if the function needs ownership, ask for ownership.

#![allow(unused)]
fn main() {
pub fn insert(&mut self, bucket: Bucket, key: Key, value: Value) -> Option<Value>
}

Three things improve at once. The cost moves to the call site, where the caller can see the clone and decide whether they needed it. A caller who already has an owned value stops paying for a copy they did not need. And the signature stops lying.

The standard library is consistent about this, and it is worth reading the asymmetry deliberately:

#![allow(unused)]
fn main() {
impl<K, V> HashMap<K, V> {
    pub fn insert(&mut self, k: K, v: V) -> Option<V>;          // stores it: takes it
    pub fn get<Q>(&self, k: &Q) -> Option<&V>;                  // looks at it: borrows it
    pub fn remove<Q>(&mut self, k: &Q) -> Option<V>;            // looks it up, hands back the value
}
}

Nothing there is an accident, and the shape of each signature tells you what happens to your data.

The middle ground

Taking ownership pushes clone() calls onto callers, which is honest but can get noisy. Two ways to soften it, both with real costs:

impl Into<T> lets a caller pass whatever they have, and the conversion happens inside:

#![allow(unused)]
fn main() {
pub fn insert(&mut self, bucket: impl Into<Bucket>, ...)
}

The cost is not zero: it is still a conversion, it just moved. It also makes the signature harder to read and it does not work for a type with a validating constructor, because From must not fail.

Cow<'a, T> lets one function serve both callers, borrowing when it can and owning when it must. It is the right answer for a parser or normaliser that usually passes data through unchanged, and it is overkill almost everywhere else. Reach for it when profiling says so, not before.

Owned and view types

The larger version of this idea is that many domains want a pair of types: one that owns and one that borrows.

The convention that falls out of it, and it is one of the most reliable rules in Rust API design:

Take the view type as a parameter, return the owned type.

A function taking &str can be called by anyone holding a String, a &str, or a string literal. A function taking &String can only be called by someone who happens to have a String, and gains nothing for the restriction. The same argument applies to &[T] over &Vec<T>, and to &Path over &PathBuf.

minidb does not need a separate KeyRef type, because &Key already does the job: a Key is a thin wrapper and a shared reference to it is a perfectly good view. The pattern earns its keep when the owned type has structure the view does not need, which is exactly why Path exists and &PathBuf is a code smell.

Exercise

The exercise for this section is located in 04_borrowing/02_ownership

RAII

minidb needs transactions: a group of changes that take effect together and can be taken back together if something goes wrong halfway.

A transaction has to reach the store somehow, and with what the course has covered so far there is exactly one way to arrange that. Give it the store:

#![allow(unused)]
fn main() {
pub struct Transaction {
    store: Store,
    undo: Vec<Undo>,
}

pub fn begin(self) -> Transaction
pub fn commit(self) -> Store
pub fn rollback(self) -> Store
}

Each change is applied to the store as it happens and the transaction records how to undo it, so rollback can put everything back and commit keeps it. It works, and the first exercise is four passing tests that prove it works and explain why nobody would ship it.

What is wrong with it

The store is inside the transaction. Nothing outside can reach it, so Transaction has to grow its own copy of every Store method a caller might want while a transaction is open. get is the first one. It would not be the last.

Every call site has to catch the store on the way out.

#![allow(unused)]
fn main() {
let store = tx.commit();
}

every time, and the variable you started with is gone.

The return types are bookkeeping. commit and rollback both hand back a Store, which says nothing about committing or rolling back. It is there because the ownership has to go somewhere.

An early return loses everything.

#![allow(unused)]
fn main() {
fn write_both(store: Store) -> Result<Store, Error> {
    let mut tx = store.begin();
    tx.insert(users.clone(), alice, Value::new("Alice"));

    let value = fetch_the_other_value()?;   // returns early

    tx.insert(users, bob, value);
    Ok(tx.commit())
}
}

The ? drops the transaction, and the store is inside it, so the caller does not get half a database back. It gets none.

What we want instead

The store should stay where it is and lend itself to the transaction for a while, exclusively, so that nothing else can touch it until the transaction finishes. That is &mut Store, and a struct that keeps one needs a lifetime, which is the one piece of chapter 4 that chapter 4 had no reason to show you.

Where this chapter goes

Five steps, and the last one is the interesting one:

1. A borrowed store. The transaction stops owning the store and borrows it instead, which is where a lifetime first goes on a struct.
2. A drop guard. Make the safe outcome automatic: an abandoned transaction rolls itself back.
3. A drop bomb. Safe is not the same as correct. Make the forgotten commit say so.
4. The limits. Drop is a strong default, not a guarantee. It is worth knowing exactly how strong.
5. A closure API. Stop asking callers to remember anything at all.

The pattern to watch for: each step moves the mistake earlier, from “silent corruption in production” to “loud failure in a test” to “the code that could make the mistake does not exist”.

Exercise

The exercise for this section is located in 05_raii/00_intro

A borrowed store

#![allow(unused)]
fn main() {
pub struct Transaction<'store> {
    store: &'store mut Store,
    undo: Vec<Undo>,
}

pub fn begin(&mut self) -> Transaction<'_>
pub fn commit(self)
pub fn rollback(self)
}

Every complaint from the previous section goes away at once. The store stays where the caller put it, commit and rollback have nothing to hand back, and Transaction::get can be deleted because nothing was taken away in the first place.

The lifetime goes on the struct

'store is a lifetime parameter, and it is the first one in this course you have to write yourself. Elision covers functions; it has nothing to say about structs, because there is no call site to infer from. A struct that holds a reference must declare the region that reference is valid over, and then the struct itself is only valid over that region.

Which is exactly the guarantee we want:

#![allow(unused)]
fn main() {
let tx = {
    let mut store = Store::new();
    store.begin()          // error[E0597]: `store` does not live long enough
};
}

A Transaction<'store> can never outlive the Store it came from, and nobody had to write a check.

'_ everywhere else

begin needs no name for it:

#![allow(unused)]
fn main() {
pub fn begin(&mut self) -> Transaction<'_>
}

&mut self is the only input lifetime, so elision rule three gives it to the elided output. The '_ is not doing the inference, it is announcing it: there is a borrow inside this type, and I am not naming it. Leaving it out entirely still compiles, and the compiler will tell you off:

warning: hiding a lifetime that's elided elsewhere is confusing

The impl header takes the same '_, because none of the methods care which region it is:

#![allow(unused)]
fn main() {
impl Transaction<'_> { .. }
}

Three guarantees nobody paid for

Look at what those two signatures have quietly bought:

A database usually buys the first two with a lock and pays for them at runtime, forever. Here they cost nothing at runtime, cannot be forgotten, and cannot be worked around without changing the signatures.

The third is worth a second look. commit(self) does not merely discourage a double commit: it makes the second call impossible, because the value is gone. That is a single-use value, also called a linear type, and it is the cheapest state machine in Rust.

Nobody wrote a line of code about transaction isolation. It fell out of choosing &mut self over self, and self over &self, one line each.

Exercise

The exercise for this section is located in 05_raii/01_borrow

Drop guards

Now that the store survives its transaction, there is one obvious way to get this wrong, and everybody does:

#![allow(unused)]
fn main() {
fn write_both(store: &mut Store) -> Result<(), Error> {
    let mut tx = store.begin();
    tx.insert(users.clone(), alice, Value::new("Alice"));

    let value = fetch_the_other_value()?;   // returns early

    tx.insert(users, bob, value);
    tx.commit();
    Ok(())
}
}

The early return skips the commit, so the transaction is dropped where it stands and half the work is now permanent. No error, no warning: the type system watched the whole thing happen and said nothing.

Note how little the mistake looks like a mistake. There is no forgotten close(), no unbalanced unlock(), just a ? in the middle of a function, which is the most ordinary thing in Rust.

The idea

Resource Acquisition Is Initialisation is a terrible name for a good idea. The idea is:

Tie the cleanup to a value, and let the compiler run it when the value goes away.

You do not have to remember. You cannot forget on the error path, because the error path drops your values too. You cannot forget on the panic path either, because unwinding drops them as well.

Rust leans on this harder than any mainstream language, because ownership tells the compiler exactly when each value dies. Box frees, File closes, MutexGuard unlocks, JoinHandle waits. None of those need a finally block, and none of them can be skipped by an early return.

The guard

A drop guard is a value whose only job is to do something when it goes out of scope. MutexGuard is the one everybody has met: it exists so that unlock cannot be forgotten, and it has no other purpose.

Our transaction becomes one by implementing Drop:

#![allow(unused)]
fn main() {
impl Drop for Transaction<'_> {
    fn drop(&mut self) {
        self.undo_everything();
    }
}
}

undo_everything is a &mut self method holding the loop that rollback used to run inline. Both paths need it and drop cannot call rollback, which consumes self, so the shared work has to live somewhere both can reach.

That is the entire safety improvement. An early return, a ?, a panic, an unhandled branch: all of them now put the store back the way they found it.

When drop runs

Worth being precise, because two of these surprise people:

• at the end of the scope where the value lives, in reverse declaration order;
• when a value is reassigned, on the old value;
• during unwinding, for everything on the stack between the panic and the catch_unwind (or the top of the thread);
• not when the value is moved: the new owner drops it instead.

The reverse order matters when guards depend on each other. Declare the outer resource first and the inner one second, and they release in the order you would have written by hand.

The &mut self problem

Drop::drop takes &mut self, never self. There is no way around it: the value has to be dropped again after your code runs, so Rust cannot let you consume it.

Which means you cannot move fields out:

#![allow(unused)]
fn main() {
impl Drop for Transaction<'_> {
    fn drop(&mut self) {
        for undo in self.undo.into_iter() {   // error[E0507]: cannot move out of `self.undo`
}

The rule is broader than drop itself. A type that implements Drop cannot have its fields moved out anywhere, because every value of it still has to be dropped afterwards, whole. So writing the impl above breaks rollback, which consumes self and was moving the undo log out quite legally a moment earlier:

#![allow(unused)]
fn main() {
pub fn rollback(self) {
    for undo in self.undo.into_iter().rev() {   // error[E0509]: cannot move out of type
                                                // `Transaction<'_>`, which implements `Drop`
}

Worth knowing before you add a destructor to a type that already has users: it is not a purely additive change.

The two standard ways out are both “leave something valid behind”:

#![allow(unused)]
fn main() {
let undo = mem::take(&mut self.undo);        // leaves an empty Vec, needs Default
let value = self.value.take();               // Option::take, leaves None
}

Option<T> as a field purely so that Drop can take the T is a common shape. It is slightly annoying to read, and it is the price of a destructor that owns something.

Disarming

Here is the subtlety that catches people, and it is the second half of the exercise.

commit consumes self. So the transaction is dropped the moment commit returns, and the Drop impl runs immediately afterwards, undoing everything that was just committed.

The fix is to make Drop able to tell that a decision was already made:

#![allow(unused)]
fn main() {
pub fn commit(mut self) {
    self.undo.clear();          // nothing left to undo, so drop is a no-op
}
}

Emptying the undo log is the cheapest version. A bool flag is the general one, and we will need it in the next section. Either way the principle is the same: a destructor that does real work needs a way to be told the work is already done.

The heavy-handed alternative is mem::forget(self), which drops the value without running its destructor. It works, and it is a bad habit: it skips the destructors of the fields too, which for a type holding a Vec means leaking memory. ManuallyDrop is the honest version of that idea when you genuinely need it.

Guards in the standard library

Once you know the shape you see it everywhere, and reading these is the fastest way to get a feel for it:

That last row is the one to remember when you write your own. Drop::drop returns (), so a destructor cannot report a failure. BufWriter flushes on drop and swallows any error, which is why its documentation tells you to call flush() yourself if you care whether the bytes arrived.

A destructor is not a place to do fallible work. If the cleanup can fail in a way the caller should know about, give them an explicit method that returns a Result, and keep the destructor as the fallback that stops things being worse.

Exercise

The exercise for this section is located in 05_raii/02_drop_guard

Drop bombs and the limits of Drop

The guard made the safe outcome automatic. It also made a bug invisible: a transaction somebody forgot to commit now behaves exactly like one they meant to abandon. The data is safe and the program is still wrong.

A drop bomb is a guard that panics when it is dropped without an explicit decision:

#![allow(unused)]
fn main() {
impl Drop for Transaction<'_> {
    fn drop(&mut self) {
        if self.finished {
            return;
        }

        self.undo_everything();

        if !thread::panicking() {
            panic!("transaction dropped while neither committed nor rolled back");
        }
    }
}
}

The bomb turns “silent corruption in production” into “loud failure in the test suite”, which is the trade every time you can get it.

The rule you cannot skip

A panic during unwinding aborts the process. Not a nicer panic, not a caught panic: abort, with no unwinding, no destructors, and no message beyond a terse note about panicking in a destructor.

That is why the thread::panicking() check is there. Without it, any real failure inside a transaction scope, an assert! in a test, a genuine bug, gets replaced by a hard abort that tells you nothing about the original problem. You lose the actual error and gain a dead process.

So the rule for any destructor that can panic:

#![allow(unused)]
fn main() {
if !thread::panicking() {
    panic!("...");
}
}

Some crates go further and only arm the bomb in debug builds, on the grounds that aborting a production process over a bookkeeping mistake is worse than the mistake. That is a judgement call about who is running the code.

When not to arm it

sqlx is the counterexample worth knowing, because it is the same domain. Its Transaction implements Drop and rolls back silently: if neither commit nor rollback is called before the transaction goes out of scope, the changes are undone and nothing is said. A plain guard, and a deliberate refusal to take the next step.

The reason is that dropping a transaction means something there. Abandoning one on the error path is the idiom: you write ? after each query, and the abandonment is the abort. A bomb would fire on correct code. Async sharpens the same point well beyond databases, because dropping a future is an ordinary event. A timeout fires, a select! branch loses, a request is cancelled, and everything in that future’s state is dropped without anyone having done anything wrong. A destructor cannot tell cancellation from forgetting, so an armed bomb would turn every timeout into a process abort.

sqlx does not even roll back in drop, for that matter. Drop is synchronous and a rollback needs I/O, so it marks the connection and the ROLLBACK goes out when that connection is next used or returned to the pool. A guard that is already best-effort and deferred is the wrong place to assert anything.

So the test is: arm the bomb only where dropping without a decision has no legitimate meaning.

rust-analyzer’s parser passes it. Its Marker must be completed or abandoned, forgetting is unambiguously a bug, it is all one codebase, and the panic lands in a developer’s own test run rather than in a user’s server. That is what matklad’s drop_bomb crate is for, and it is where the name comes from.

minidb passes it too, but only because of where this chapter is going. Arming the bomb changes what the ? in write_both does: the early return we opened with now panics instead of quietly rolling back. If begin were the interface we shipped, that would be a hard sell. It is not. The next section puts a closure API in front of it that makes the decision itself, which leaves the bomb armed for the callers who need the raw form and off the path everybody else takes.

Drop is a strong default, not a guarantee

It is genuinely possible for a destructor never to run, and leaking is safe in Rust: it is not unsafe, and no rule in the language promises drop will happen.

The ways it does not run:

• mem::forget(value), which exists precisely to skip it, and ManuallyDrop<T>, its typed form.
• Box::leak, and anything else that deliberately gives up ownership for a 'static reference.
• Reference cycles. Two Rcs pointing at each other keep the count above zero forever. Nothing in the language stops you.
• process::exit and abort, which do not unwind at all.
• panic = "abort", a profile setting that turns every panic into an abort and skips every destructor on the way out.
• Leaked threads. A detached thread’s stack is never unwound if the process ends first.

This was settled deliberately, in a long argument that has its own name: leaking was going to be possible via Rc cycles no matter what the API said, so mem::forget became safe rather than pretending otherwise.

The practical consequence: you cannot use Drop to enforce a safety invariant. It is a convenience and a very good default, not a proof. If correctness depends on cleanup happening, the cleanup has to be on the only path that reaches the result, which is what the next section is about.

Exercise

The exercise for this section is located in 05_raii/03_drop_bomb

Closure APIs

Look at what the last two sections actually achieved. A forgotten commit now rolls back and panics loudly instead of corrupting data quietly. That is a large improvement, and the mistake is still there to be made.

There is a move available that the guard cannot make: stop handing out the thing that can be misused.

#![allow(unused)]
fn main() {
impl Store {
    pub fn transaction<F, T, E>(&mut self, changes: F) -> Result<T, E>
    where
        F: FnOnce(&mut Transaction<'_>) -> Result<T, E>,
    {
        let mut tx = self.begin();

        let result = changes(&mut tx);

        if result.is_ok() {
            tx.commit();
        } else {
            tx.rollback();
        }

        result
    }
}
}

The caller never holds a Transaction they are responsible for:

#![allow(unused)]
fn main() {
store.transaction(|tx| {
    tx.insert(&users, &alice, Value::new("Alice"));
    let value = fetch_the_other_value()?;      // early return rolls back
    tx.insert(&users, &bob, value);
    Ok(())
})?;
}

The ? still returns early. It now returns early from the closure, which is a value transaction receives and acts on. The decision moved from the caller’s discipline into your code, where it is written once and tested once.

Notice that the drop bomb never fires through this path. The only code that could forget to commit is the six lines you just wrote.

The general shape

This is the same trick as thread::scope, Vec::retain_mut, HashMap::entry and every with_something function you have ever called:

Instead of giving the caller a resource and a rule, take a closure and apply the rule yourself.

It is the strongest of the three levels, and worth naming them explicitly:

The lifetime you did not write

There is an elided lifetime in that bound, and it does not behave like the ones in chapter 4:

#![allow(unused)]
fn main() {
F: FnOnce(&mut Transaction<'_>) -> Result<T, E>
}

In a function signature an elided lifetime becomes a parameter of the function. In an Fn bound it does not: it gets its own binder, and the bound above means

#![allow(unused)]
fn main() {
F: for<'tx, 'store> FnOnce(&'tx mut Transaction<'store>) -> Result<T, E>
}

That is a higher-ranked bound, and it has to be one. transaction creates the Transaction itself, inside its own body, so its lifetime is not something any caller could name. Try to make it a parameter of the function and the compiler says so:

#![allow(unused)]
fn main() {
pub fn transaction<'tx, F, T, E>(&mut self, changes: F) -> Result<T, E>
where
    F: FnOnce(&'tx mut Transaction<'_>) -> Result<T, E>,
//  error[E0597]: `tx` does not live long enough
}

for<'tx, 'store> says the opposite of a parameter, and the opposite is what we mean: the callee picks, and the closure has to cope with whatever it picks.

One trap, in case anyone tries to tidy it up. Collapsing the two binders into one type-checks as a bound and then fails inside the body:

#![allow(unused)]
fn main() {
F: for<'tx> FnOnce(&'tx mut Transaction<'tx>) -> Result<T, E>,
//  error[E0505]: cannot move out of `tx` because it is borrowed
}

Tying the borrow of tx to tx’s own store lifetime keeps it borrowed for as long as it exists, so it can never be moved into commit. &mut T is invariant in T, so the compiler cannot shrink its way out. The two lifetimes have to stay separate, which is exactly what '_ gives you for free.

What it costs

That last column is not a rounding error, and the honest version of this advice includes it.

• Borrows cannot escape. The closure’s return value cannot borrow from the transaction, because the transaction is gone by the time transaction returns. If a caller wants a reference to something inside, they have to clone it or restructure.
• Control flow gets awkward. break and continue do not cross a closure boundary, and return inside the closure returns from the closure. ? works, which covers most real cases, but a loop that wants to abandon its transaction mid-iteration is now fighting you.
• Composition suffers. Two resources means two nested closures, and the rightward drift is real.
• Async makes it worse. A closure API that must accept an async block needs the higher-ranked bounds that Rust is still not good at expressing, and the error messages are terrible.

So the honest recommendation is both, which is what minidb now has: a closure API as the front door for the ninety percent, and begin still there, still guarded, still armed, for the callers whose control flow does not fit. std does the same thing: thread::scope for the common case, thread::spawn when you need a handle.

What you should not do is offer only the raw version and document the rule. That is the level where the mistake is silent, and this whole chapter is about not being there.

Exercise

The exercise for this section is located in 05_raii/04_closure_api

Typestate

A new requirement: reporting code should be able to open a transaction that is guaranteed not to write. Here is the implementation almost every codebase ships:

#![allow(unused)]
fn main() {
pub struct Transaction<'store> {
    // ...
    read_only: bool,
}

pub fn insert(&mut self, bucket: Bucket, key: Key, value: Value) {
    assert!(!self.read_only, "cannot write through a read-only transaction");
    // ...
}
}

It is correct. Every rule is enforced. Count what it costs:

• a bool in every transaction, including the ones that will never be read-only;
• a branch on every insert, forever;
• a panic that reaches production if any code path was not covered by a test;
• a # Panics section that a caller has to read, believe and remember.

We have spent the whole day moving errors from runtime to compile time. This is a regression, and it is worth noticing how natural it felt to write.

The move

Typestate means putting the state of a value into its type, so that the operations which are illegal in that state do not exist.

#![allow(unused)]
fn main() {
pub struct Transaction<'store, A> { /* ... */ }

pub struct ReadOnly;
pub struct ReadWrite;

impl<A> Transaction<'_, A> { pub fn get(&self, ..) -> Option<&Value>; }   // both modes

impl Transaction<'_, ReadWrite> { pub fn insert(&mut self, ..); }         // one mode
}

Transaction<'_, ReadOnly> has no insert. Not a private one, not one that panics: there is no method to call, so the mistake is a compile error at the call site, with a message pointing at the line that made it.

The bool is gone, the branch is gone, and ReadOnly and ReadWrite are zero-sized, so the struct does not grow. This is the same trick as the newtype from chapter 2, applied to the state of a value rather than to its meaning.

The two shapes

Typestate comes in two flavours, and this chapter does one of each.

States that never change. A transaction is born read-only or read-write and stays that way. The type parameter is a permanent label, and the value never transitions. This is capability narrowing: the type says what the holder is allowed to do.

States that change. A document writer is at the top level, then inside a bucket, then back at the top level. Each step consumes the value and returns a different type, so the previous state is gone and cannot be used again. This is a state machine, checked by the compiler, with no runtime representation at all.

The second shape is where typestate earns its reputation, and it depends entirely on something you already have: a method taking self leaves the caller with nothing.

Exercise

The exercise for this section is located in 06_typestate/00_intro

States as capabilities

The mechanics are small. Two pieces.

Marker types. Zero-sized structs whose only job is to be distinct:

#![allow(unused)]
fn main() {
pub struct ReadOnly;
pub struct ReadWrite;
}

PhantomData to use a type parameter you store no value of:

#![allow(unused)]
fn main() {
pub struct Transaction<'store, A> {
    store: &'store mut Store,
    undo: Vec<Undo>,
    finished: bool,
    _access: PhantomData<A>,
}
}

Rust requires every type parameter to appear in the body, and PhantomData<A> is how you satisfy that without storing anything. It is zero-sized: size_of::<Transaction<ReadWrite>>() and size_of::<Transaction<ReadOnly>>() are the same, and both are the size of the fields you actually have.

Then split the methods:

#![allow(unused)]
fn main() {
impl<A> Transaction<'_, A> {
    pub fn get(&self, ..) -> Option<&Value> { .. }      // every mode
}

impl Transaction<'_, ReadWrite> {
    pub fn insert(&mut self, ..) { .. }                 // one mode
    pub fn commit(self) { .. }
}
}

Two things that will catch you

Drop must match the struct exactly. A Drop impl has to repeat its struct’s bounds, and it cannot add one the struct does not have. Our struct has none, so neither may the destructor:

#![allow(unused)]
fn main() {
impl<A> Drop for Transaction<'_, A> { .. }      // add `where A: ..` here and you get E0367
}

Chapter 8 adds a bound to the struct, and all three sites, struct, shared impl and Drop, have to gain it together.

This is also why you cannot write a Drop impl for only one mode. If the two modes need different destructor behaviour, the difference has to live in a field, which is exactly what finished does here: a read-only transaction is born finished, so the drop bomb never arms for it.

Nothing yet says what A may be. Transaction<'_, u32> is a nameable type, and it has no methods and no way to be constructed, so it is a curiosity rather than a hole. Writing the set down takes a trait, and deciding who may add to it takes sealing. Both are chapter 8.

Capabilities and tokens

Transaction<'_, ReadWrite> is doing something more general than tracking a state: holding it is proof that you are allowed to write. A function taking one does not need to check anything, because possession is the check.

That idea has its own name, the permission token or capability, and once you see it you find it everywhere:

• MutexGuard<T> is a token that proves the lock is held, and it carries the data so that you cannot reach the data without it. Token plus payload is the most useful form.

• Embedded HALs hand out a Peripherals struct exactly once per program, via take(). Owning the Pin<Output> is proof that nobody else has configured that pin.

• Zero-sized proof tokens are the pure form:

  #![allow(unused)]
  fn main() {
  pub struct Authenticated(());        // private field: only this module can build one
  
  pub fn authenticate(creds: &Credentials) -> Option<Authenticated>;
  pub fn delete_everything(_: &Authenticated);
  }

  delete_everything cannot be called without an Authenticated, and an Authenticated cannot be built without going through authenticate. The private () field is what closes the door, exactly as it did for Key in chapter 2.

The token pattern and typestate are the same idea seen from two angles: a type that exists only to carry a fact the compiler can check.

When not to do this

Typestate is not free, and the costs land on your users:

• Error messages get worse. “no method named insert found for struct Transaction<'_, ReadOnly>” is good. The equivalent in a five-parameter generic builder is not.
• The type parameter is contagious. Every function that takes your type either fixes the state or becomes generic over it, and that spreads.
• dyn becomes awkward. Box<dyn Transaction> does not exist any more, because there is no one type. If callers need to store your value in a collection alongside other states, typestate fights them.
• Two states are sometimes just two types. If ReadOnly and ReadWrite shared no methods at all, two separate structs would be simpler and clearer than a type parameter.

The rule of thumb: reach for typestate when the states share most of their behaviour, when the illegal operations are genuinely illegal rather than merely unusual, and when the value is used directly rather than through a trait object.

Exercise

The exercise for this section is located in 06_typestate/01_transaction

States that move

ReadOnly and ReadWrite never change: a transaction is born in one and dies in it. The other half of the pattern is a value that walks through states, where each step changes the type.

The export format has a rule that no signature has expressed so far:

[users]
42 = Alice
43 = Bob

Entries belong to a bucket. Writing an entry before opening one is nonsense, and so is finishing the document while a bucket is still open. Both are easy mistakes and both are, so far, only caught by reading the output.

Consuming self is the transition

#![allow(unused)]
fn main() {
impl Writer<Root> {
    pub fn bucket(self, bucket: &Bucket) -> Writer<InBucket>;
    pub fn finish(self) -> String;
}

impl Writer<InBucket> {
    pub fn entry(self, key: &Key, value: &Value) -> Self;
    pub fn end(self) -> Writer<Root>;
}
}

Every method takes self and returns the next state. That is the single-use value from chapter 5, applied once per step: after bucket() the Writer<Root> is gone, moved into a Writer<InBucket>, so there is nothing left to call finish on.

The result is a state machine with no runtime representation whatsoever. No enum, no discriminant, no match. The transitions happen at compile time and the generated code is a String and some pushes.

Reading the chain out loud is the fastest way to see what has been bought:

#![allow(unused)]
fn main() {
Writer::new()
    .bucket(&users)                        // Root      -> InBucket
    .entry(&alice, &Value::new("Alice"))   // InBucket  -> InBucket
    .end()                                 // InBucket  -> Root
    .finish()                              // Root      -> String
}

Any other order is a compile error, and the compiler names the state you were in when you got it wrong.

Where you have already met this

• serde’s serializer. serialize_struct returns a SerializeStruct, whose end() returns to the outer serializer. The nesting rules of the data format are enforced by types, which is how serde can be format-agnostic and still not let you emit a malformed document.
• Builders that require fields. Builder<Missing, Missing> gaining parameters as setters are called, with build() only implemented on Builder<Set, Set>. This is how a builder gets compile-time required fields instead of an Option and a runtime check.
• std::process::Command is the counter-example worth noting: it does not use typestate, because every ordering is legal. Typestate would be pure cost there.

The costs, again

The same warnings as the previous section, plus one specific to transitions:

Loops need care. A for loop that calls a transitioning method has to thread the value through:

#![allow(unused)]
fn main() {
let mut open = writer.bucket(bucket);

for (key, value) in entries {
    open = open.entry(key, value);        // reassign, because entry consumed it
}

writer = open.end();
}

That reassignment is the price of the guarantee, and it is the point where people ask whether it was worth it. For a serializer, where a malformed document is a bug that reaches a customer, it usually is. For a fluent builder where every order is fine, it is not.

Conditional transitions are painful. if condition { w.bucket(b) } else { w } does not compile, because the two branches have different types. When you need that, you are back to an enum and a runtime check, and that is the honest signal that the state does not belong in the type.

Exercise

The exercise for this section is located in 06_typestate/02_writer

Extension traits

Every ergonomic improvement so far has been a method on a type we own. Chapter 2 gave us Key and Bucket, and every call site since has spelled the constructor out:

#![allow(unused)]
fn main() {
let users = Bucket::parse("users")?;
let id = Key::parse("42")?;
}

What we want is the method on the string:

#![allow(unused)]
fn main() {
let users = "users".to_bucket()?;
let id = "42".to_key()?;
}

This is the first wish in the course that cannot be granted by adding a method, because str belongs to the standard library. Two separate rules stand in the way, and only the second one is the orphan rule.

The rule you hit first

#![allow(unused)]
fn main() {
impl str {
    fn to_key(&self) -> Result<Key, NameError> { .. }   // error[E0390]
}
}

Inherent impls may only be written in the crate that defines the type: E0116 in general, and E0390 for a primitive like str. No coherence argument is involved here. The rule is only about where a type’s own methods live.

The orphan rule

The rule that shapes the rest of the chapter is the other one. You may implement a trait for a type only if you own the trait or you own the type. Both foreign is E0117:

#![allow(unused)]
fn main() {
impl Display for Option<String> { .. }   // error[E0117]
}

The reason is coherence. If two crates could both impl Display for Vec<u8>, then a program depending on both would have two implementations for the same call and no principled way to choose. Rust’s answer is to make the situation impossible rather than to define a tie-break, which is also why adding an impl in a library is a semver-visible act.

The way through

You own the trait if you define it. So define one:

#![allow(unused)]
fn main() {
pub trait StrExt {
    fn to_key(&self) -> Result<Key, NameError>;
}

impl StrExt for str {
    fn to_key(&self) -> Result<Key, NameError> {
        Key::parse(self)
    }
}
}

Now "users/42".to_key()? works, on a type you do not own, without breaking coherence: your crate owns StrExt, and any other crate’s competing extension trait is a different trait.

This is an extension trait, and the convention is to name it after what it extends with an Ext suffix: StrExt, IteratorExt, ResultExt.

The catch that is also the point

An extension trait’s methods exist only where the trait is in scope:

#![allow(unused)]
fn main() {
use minidb::StrExt;      // without this line, `to_key` does not exist
}

That is not a wart, it is the mechanism. Your extension methods cannot collide with anybody else’s unless a caller deliberately imports both, which is why use itertools::Itertools; is a line you write rather than something that happens to you.

Exercise

The exercise for this section is located in 07_extension_traits/00_intro

Which type to implement, and when not to

Implement it for the unsized type

#![allow(unused)]
fn main() {
impl StrExt for str { .. }       // yes
impl StrExt for &str { .. }      // no
impl StrExt for String { .. }    // no
}

Implementing for str gets you all three. A &str finds the method by auto-deref, and so does a String, because String: Deref<Target = str>. Implementing for &str instead gets you exactly one of the three and a confusing error for the other two.

The same rule applies elsewhere: implement for [T], not &[T]; for Path, not &Path.

Now do not write it

Before defining an extension trait, check whether the standard library already has the trait you are about to reinvent. For this exact case it does:

#![allow(unused)]
fn main() {
impl FromStr for Key {
    type Err = NameError;

    fn from_str(raw: &str) -> Result<Self, Self::Err> {
        Self::parse(raw)
    }
}
}

That gives callers "users/42".parse::<Key>(), and .parse() with no turbofish wherever the target type is inferred. It requires no import, because FromStr is in the prelude, and it is the spelling every Rust programmer already knows.

It also unlocks things you did not write: clap uses FromStr for argument parsing, serde can be pointed at it, and a generic function taking T: FromStr accepts your type without knowing it exists.

Prefer the standard trait. The list worth checking before inventing anything:

An extension trait is what you reach for when nothing in that list fits, or when the method genuinely has no home other than “convenience on somebody else’s type”.

So when is it right?

• Convenience layers over a foreign API. ResultExt::context() in anyhow adds an operation to every Result in your program that the standard library was never going to add.
• Methods on a trait you do not own. Itertools and FuturesExt are the canonical examples, and they are the subject of the next section.
• Keeping a public type small. An extension trait in a companion crate lets people opt into an ergonomic layer without it appearing in the core type’s documentation.

And when it is not:

• On your own type. If you own it, add an inherent method. Inherent methods need no import, show up first in the rustdoc, and take priority in method resolution.
• To reach around a missing API. If the upstream type is missing something fundamental, an extension trait in your crate is a private fix for a public problem. Consider sending a patch.

Exercise

The exercise for this section is located in 07_extension_traits/01_str_ext

Extending a trait

An extension trait does not have to extend a type. It can extend another trait, and that is how one small crate adds forty methods to every iterator in the language:

#![allow(unused)]
fn main() {
pub trait IteratorExt: Iterator {
    fn collect_sorted(self) -> Vec<Self::Item>
    where
        Self: Sized,
        Self::Item: Ord,
    {
        let mut items = self.collect::<Vec<_>>();
        items.sort();
        items
    }
}

impl<I> IteratorExt for I where I: Iterator {}
}

Four pieces, each doing a specific job.

The supertrait, : Iterator, restricts the impl to iterators and gives the default body access to Iterator’s methods. Without it, self.collect() does not exist.

The default body in the trait means the blanket impl can be empty. Anyone implementing IteratorExt by hand gets the behaviour for free.

The blanket impl covers every iterator that exists, including types written after your crate. This is the only way to add a method to types you have never heard of.

Self: Sized on the method, not on the trait. A method taking self by value needs a sized receiver, but putting Sized on the trait itself would make dyn IteratorExt impossible. Keeping the bound on the method is what lets Iterator itself be both dyn compatible and full of consuming adaptors.

Method resolution, and how it bites

Rust looks for methods in a fixed order, and the first two rules explain most surprises:

1. Inherent methods win over trait methods. If Store has an inherent export, an extension trait’s export will never be called on a Store. Adding an inherent method to a type in a later release can therefore silently steal calls from an extension trait, which is a real semver hazard.
2. Then trait methods, but only from traits in scope, walking T, &T, &mut T and then deref targets.

If two traits in scope both offer the method, neither wins:

error[E0034]: multiple applicable items in scope

The caller’s fix is fully qualified syntax:

#![allow(unused)]
fn main() {
IteratorExt::collect_sorted(iter)
}

which is correct, ugly, and not a problem the caller asked for. It is the strongest argument for keeping extension traits small and specific: every method you add to every iterator in the program is a name you have taken from everyone else.

The judgement

Extension traits are a genuinely good tool with one failure mode: they are addictive. Itertools and anyhow’s Context earn their place because they are focused, widely useful, and named so that a reader can find where the method came from.

The test before adding one: when a reader sees this method call, can they work out where it is defined? If the answer is “only by grepping the imports”, the trait is too broad or the name is too generic.

Exercise

The exercise for this section is located in 07_extension_traits/02_iter_ext

Polymorphism

Store::export renders one format, and the format is welded into the method. The next request is always CSV, or JSON, or whatever the reporting team standardised on this quarter.

So we need to be polymorphic over “some format”. Rust offers two mechanisms, and choosing between them is one of the few API decisions that is genuinely hard to reverse:

#![allow(unused)]
fn main() {
fn export_with<F: Format>(&self, format: F) -> String        // static dispatch
fn export_into(&self, format: &mut dyn Format) -> String     // dynamic dispatch
}

They look almost the same at the call site and are entirely different underneath.

Static dispatch

export_with is not one function. The compiler generates a separate copy for every F you call it with, each one knowing exactly which Format it is talking to. That is monomorphisation, and it is why generic Rust has no dispatch cost: after inlining, format.entry(..) is a direct call, often no call at all.

The costs are real but indirect: compile time, binary size, and the fact that Vec<F> can hold only one kind of format.

Dynamic dispatch

&mut dyn Format is a fat pointer: one pointer to the value, one to a vtable of function pointers. There is one copy of export_into in the binary, and the call goes through the vtable, so it cannot be inlined.

What you get for that is the thing static dispatch cannot do: a Vec<Box<dyn Format>> holding three different formats, a format chosen from a config file at run time, a plugin loaded from a shared library.

The part that surprises people

Not every trait can be a dyn. Dyn compatibility (previously called object safety) is a property of the trait, and it constrains how you write the trait even if nobody ever writes dyn Format.

The next section builds Format under that constraint, and the constraint is visible in the signature:

#![allow(unused)]
fn main() {
fn finish(&mut self) -> String;      // what we have to write
fn finish(self) -> String;           // what reads better, and is not dyn-compatible
}

Then the last section goes the other way, and takes the ability to implement a trait away from everybody else on purpose.

Exercise

The exercise for this section is located in 08_polymorphism/00_intro

Generics and dyn

Dyn compatibility

A trait can be made into a trait object only if every method can be called through a fat pointer, which means the compiler must be able to build a vtable entry for it. The rules that matter in practice:

The escape hatch for the first four is where Self: Sized on the method: such a method is excluded from the vtable and remains callable on concrete types. This is how Iterator is dyn-compatible while still having map, collect and forty other consuming, generic adaptors.

So Format is written like this:

#![allow(unused)]
fn main() {
pub trait Format {
    fn bucket(&mut self, bucket: &Bucket);
    fn entry(&mut self, key: &Key, value: &Value);
    fn finish(&mut self) -> String;
}
}

finish(&mut self) rather than finish(self), and the implementations end up doing mem::take(&mut self.output). That is a small ugliness in exchange for Box<dyn Format> being possible, and it is a decision you make once, at the trait, for all time.

Choosing

Both versions can coexist, and in minidb they do, but the default matters. A checklist that resolves most cases:

Reach for generics when the set of types is known at compile time, when the calls are hot, when the trait has generic methods or consuming methods you do not want to contort, or when you want the strongest possible inlining.

Reach for dyn when the type is chosen at run time, when you need a heterogeneous collection, when the extra copies would bloat compile times for no benefit, or when the trait is a plugin boundary.

Two rules of thumb from the standard library and its ecosystem:

• Take impl Trait in argument position by default. It is the generic version with less ceremony, and callers cannot tell the difference.
• Prefer &dyn Trait to Box<dyn Trait> where the value does not need to be owned. Box allocates; a reference does not.

The cost, measured honestly

Dynamic dispatch costs an indirect call and the loss of inlining, and that is usually irrelevant. The real difference is what the two enable, not what they cost:

• static: no Vec<F> with mixed formats, but zero overhead and monomorphised errors that mention your concrete type;
• dynamic: one function in the binary, run-time choice, and a trait you have to keep dyn-compatible forever.

The performance argument is the one people reach for first and it is usually the least important. In a function that walks a HashMap and pushes to a String, the vtable call does not appear in a profile.

Monomorphise the signature, not the body

export_with is generic, so the compiler stamps out one copy per format. What goes inside that copy is your choice, and the obvious version duplicates far too much:

#![allow(unused)]
fn main() {
pub fn export_with<F>(&self, mut format: F) -> String
where
    F: Format,
{
    let mut buckets = self.buckets.iter().collect::<Vec<_>>();
    buckets.sort_by(|(left, _), (right, _)| left.cmp(right));

    for (bucket, values) in buckets {
        format.bucket(bucket);
        // sort this bucket's entries, walk them, call format.entry
    }

    format.finish()
}
}

Every line of that is copied per format, and every copy is identical except the three calls through format. Two formats, two sorts, two loops. Add a third and you pay again.

Split it instead, and put the work behind &mut dyn:

#![allow(unused)]
fn main() {
pub fn export_with<F>(&self, mut format: F) -> String
where
    F: Format,
{
    self.render(&mut format)
}

pub fn export_into(&self, format: &mut dyn Format) -> String {
    self.render(format)
}

fn render(&self, format: &mut dyn Format) -> String {
    // the sorting and the walking, once
}
}

The generic part is now one line. render exists once in the binary however many formats there are, both public methods are thin wrappers over it, and export_into gets the whole thing for free.

When the generic parameter is there for the caller’s ergonomics and the body is large, keep the generic function thin and put the work behind &mut dyn. You keep the signature you wanted and stop paying for it per instantiation.

This is the cost worth managing. The previous section said the vtable call does not show up in a profile, and it does not. Compile time and binary size do show up, and monomorphisation is what spends them.

Exercise

The exercise for this section is located in 08_polymorphism/01_format

Sealed traits

Since chapter 6, Transaction and Writer have each carried a type parameter with nothing to say what may go in it. Transaction<'_, u32> is a nameable type. Nobody can build one, so it has been a curiosity rather than a hole, and we left it alone because the fix and the reason for the fix belong together, here.

Writing the set down is one line per trait:

#![allow(unused)]
fn main() {
pub trait AccessMode {}
pub trait Position {}
}

and putting them to work is a bound on Transaction<'store, A> and Writer<P>. That much is bookkeeping. The decision worth making is the next one, and it is the same decision for every public trait you write: is a third-party implementation of this something I want to work?

Format is an extension point. Somebody else’s crate should be able to add a format, and every method they need to do that is public.

AccessMode and Position are not. ReadOnly, ReadWrite, Root and InBucket are the only members that will ever make sense. A Transaction<MyOwnMode> would satisfy the bound and have no methods at all, because insert is defined on Transaction<'_, ReadWrite> and nowhere else.

Making a trait public and implementable is a promise. Sealing takes back half of it, and it is much easier to do now than after the trait has been published open.

The pattern

#![allow(unused)]
fn main() {
mod sealed {
    pub trait Sealed {}
}

pub trait AccessMode: sealed::Sealed {}

impl sealed::Sealed for ReadOnly {}
impl AccessMode for ReadOnly {}
}

The sealed module is private, so sealed::Sealed cannot be named outside this crate. A downstream crate can still see AccessMode, still write T: AccessMode bounds, still call every method: it just cannot write impl AccessMode for MyType, because it cannot implement the supertrait it would need.

The error a downstream user gets is honest, if not beautiful:

error[E0277]: the trait bound `MyType: Sealed` is not satisfied

Adding a # Sealed note to the trait’s documentation is worth the two lines it costs.

What sealing buys

Freedom to add methods. A trait nobody outside can implement can grow a required method in a minor release without breaking anyone. For an open trait, adding a required method is a breaking change, and adding a defaulted one still risks colliding with an implementor’s inherent method.

Freedom to assume exhaustiveness. If you know every implementor, you can match on them, add blanket impls, and rely on invariants the trait itself does not express.

A clearer contract. The trait becomes documentation of a closed set rather than an invitation.

What it does not buy, yet

None of that is cashed in here. AccessMode and Position are empty, nothing dispatches on them, and Transaction’s private fields already stop an outside implementor from building a Transaction<MyOwnMode>. Sealing changes no code that exists, and the compile_fail test in the exercise proves the mechanism works rather than that it matters.

That is the normal case, and it is still the right call, because the decision is irreversible in one direction only. Sealing an open trait breaks every downstream implementor, so it can be done freely just once, before publication. Unsealing is available forever and breaks nobody. What you are buying is the option to add a method, or to assume the set is exhaustive, in a release you have not written yet, at a price that only stays low until the crate ships.

When not to seal

Sealing is a restriction on your users, so it needs a reason. Leave a trait open when you want people to implement it: Format here, Iterator, Read, serde::Serialize.

The question to ask: is a third-party implementation of this trait something I want to work? If yes, leave it open and accept that its signature is frozen. If the answer is “that would be meaningless” or “that would break my invariants”, seal it and say so in the docs.

std seals plenty: SliceIndex, IsTerminal, and every os::unix extension trait, OsStrExt and CommandExt among them. All of them are closed sets that exist to be used rather than extended.

The nearby alternative

An enum is the other way to spell “a closed set”, and it is often better:

#![allow(unused)]
fn main() {
pub enum Format { Ini, Csv }
}

An enum is closed by construction, exhaustively matchable, and needs no ceremony. It cannot be extended by anyone, including you in a minor release, and it cannot carry per-variant behaviour without a match in every method.

The rough division: enum when the set is small and you dispatch on it; sealed trait when each member carries its own behaviour or is used as a type parameter. AccessMode and Position are type parameters, so they must be types, and sealing is the only way to close the set.

Exercise

The exercise for this section is located in 08_polymorphism/02_sealed

PhantomData, variance and brands

You have been using PhantomData since chapter 6 without asking what it does.

The usual explanation is “it silences the unused type parameter error”, which is true and is the least interesting part. The real definition:

PhantomData<T> makes the compiler treat your struct as though it contained a T, for every purpose except memory.

Four purposes, specifically.

Size: none. PhantomData<T> is zero-sized for every T, including T that are enormous. Transaction<'_, ReadOnly> and Transaction<'_, ReadWrite> have identical layouts.

Auto traits: Send, Sync and friends are decided by what a type contains, and PhantomData counts as containing:

#![allow(unused)]
fn main() {
struct Handle {
    id: u32,
    _marker: PhantomData<*const ()>,      // now !Send and !Sync
}
}

That is how a type that is nothing but an integer can be made thread-bound, which matters for handles that are only valid on the thread that created them.

Ownership and drop check: PhantomData<T> tells the compiler you own a T, so the borrow checker treats your struct as though it will drop one. This matters for types built on raw pointers, where the compiler otherwise cannot see that dropping your struct might touch borrowed data.

Variance: whether Foo<'long> may be used where Foo<'short> is expected. This one has no observable effect until it does, and then it is the entire trick behind the last exercise of the day.

Where the chapter goes

Two exercises, both of them the same move: use a marker to claim a relationship the data does not have.

1. A handle that owns its data and borrows nothing, made to behave exactly like a borrow.
2. A lifetime that no other code can name, used to make one store’s keys unusable with another. This is GhostCell, and it is the most exotic thing in the day.

If the workshop is running to time you are reading this at about half past four, and both of these are a victory lap rather than a load-bearing part of the day.

Exercise

The exercise for this section is located in 09_phantom/00_intro

Lifetimes for things that are not references

Here is a handle that remembers where a value lives:

#![allow(unused)]
fn main() {
pub struct EntryRef<'store> {
    bucket: Bucket,
    key: Key,
    _store: PhantomData<&'store Store>,
}
}

Look at what it holds: two owned names, both 'static, and no references at all. Left to itself, this type could outlive the store, be sent to another thread, and be read against a store where the entry has since been deleted. The compiler would have nothing to say, because nothing in the data says otherwise.

PhantomData<&'store Store> says otherwise. It makes the handle behave, for the borrow checker, exactly like a shared reference to the store, which is what it morally is. Every ordinary rule then applies for free:

• the handle cannot outlive the store;
• the store cannot be mutated while the handle exists;
• several handles can coexist, because shared borrows do.

None of that is code you wrote. It is one field of a type that occupies no space.

The second bullet is doing more work than it looks. EntryRef cannot dangle, so what the borrow prevents is subtler: an insert at the same place would leave the handle valid and pointing at a different value, and nothing would signal the swap. That is the ABA problem, the same reason you cannot insert into a HashMap while iterating it.

What the borrow does not buy is identity. Store::read still returns an Option because nothing ties a handle to this store: two stores can be borrowed for the same region, and a lifetime cannot tell two values apart. Closing that gap is the next section.

Where else this shows up

Once you know the shape you find it in every wrapper over a foreign resource: an index into an arena, a row id from a database handle, a slab key, a GPU buffer handle. Anywhere the value is a plain integer that is only meaningful relative to something else.

Which PhantomData to write

The choice affects variance and auto traits, so it is worth being deliberate rather than copying:

The last one looks bizarre and is the workhorse of the next section.

For our handle, PhantomData<&'store Store> is right: shared, tied to the store, and it keeps the handle Send if the store is.

Exercise

The exercise for this section is located in 09_phantom/01_entry_ref

Branded lifetimes

A Key parsed anywhere works with any Store. In a system with one store that is fine. In a system with several, using store A with a key you built while thinking about store B compiles, returns None or the wrong row, and is a real class of bug.

The types cannot tell the stores apart, because there is only one Store type. What we need is a way to make each value have its own type, and Rust has exactly one thing that is fresh at every call site: a lifetime.

The trick

#![allow(unused)]
fn main() {
pub struct Scoped<'brand> {
    store: Store,
    _brand: PhantomData<fn(&'brand ()) -> &'brand ()>,
}

pub fn scope<F, R>(changes: F) -> R
where
    F: for<'brand> FnOnce(Scoped<'brand>) -> R,
{
    changes(Scoped { store: Store::new(), _brand: PhantomData })
}
}

Two pieces, neither guessable, both essential.

for<'brand> is a higher-ranked bound: the closure must work for every lifetime, so it cannot be written to expect a particular one. Each call to scope therefore hands the closure a fresh, anonymous lifetime that no other call, and no code outside, can name or unify with.

PhantomData<fn(&'brand ()) -> &'brand ()> makes 'brand invariant. Without it the lifetime would be covariant, so a longer brand could be shortened to match a shorter one, two brands would happily unify, and the whole thing would silently do nothing.

That is why the marker is a function type. Function types are contravariant in their arguments and covariant in their return, and the only lifetime that satisfies both at once is the exact one. fn(T) -> T is the standard spelling of “invariant in T”, and unlike &mut T it keeps the type Send and Sync.

Variance, briefly

Variance is the rule for when one type may substitute for another:

Covariance is what lets you pass a &'static str to a function wanting &'a str, and you have relied on it all day without noticing. Invariance is what you reach for when a lifetime is being used as an identity rather than as a duration, which is exactly what a brand is.

What it buys

The compile error is the product:

#![allow(unused)]
fn main() {
let stolen = scope(|store| store.key("42").unwrap());

scope(|store| store.get(&users, &stolen));      // does not compile
}

Beyond stopping mix-ups, this is the foundation of a family of zero-cost APIs. If a library can prove an index was checked against a particular collection, it can hand out an accessor that skips the bounds check without unsafe at the call site. That is what GhostCell and the generativity crate are for, and it is how indexing-style crates offer checked-once, used-many access.

Should you use this?

Usually not, and it is worth being blunt about it at the end of a long day.

The costs are heavy: every API has to be inside a closure, the lifetime is contagious across every type that touches a branded value, error messages become genuinely hard to read, and the technique is unfamiliar enough that a reviewer will need the explanation you just read.

Reach for it when mixing up instances is both easy and silently wrong, and when the API is narrow enough that the closure ceremony is a one-time cost. Arena indices are the honest use case. Two stores in an application are usually better served by naming the variables carefully.

The reason it is the last thing in this workshop is not that it is the most useful. It is that it is the furthest point on the line the whole day has been walking: state in the types, permission in the types, protocol in the types, and finally identity in the types.

Exercise

The exercise for this section is located in 09_phantom/02_branded