Introducing 'Hermit': Actually Portable Wasm

Today we’re introducing Hermit, a toolchain that compiles self-contained binaries from WebAssembly System Interface WASI modules. Hermit is great for creating and sharing CLI tools without having to trudge through complicated build workflows for specific platforms and architectures. Hermit bundles a WebAssembly runtime, WASI configuration, and a WebAssembly module into one executable (called a hermit). The same hermit executable runs on multiple platforms, bringing the portability of Wasm to a new level!

Key features:

1. Package and run a Wasm module as a multi-platform executable

single executable format for all programs running anywhere
bundles wasm and runtime configuration (defined in a Hermitfile)

2. Control host system access levels and capabilities of applications

grant specific disk/network/env access to wasm code using the Hermitfile

3. Extremely portable artifact

the same hermit-compiled executable runs across MacOS, Linux, and Windows
simplify build and release workflows

Why?

WebAssembly is “write once, run anywhere”, but that “anywhere” comes with one big caveat: there has to be a WebAssembly runtime wherever the module goes. Major browsers have had WebAssembly runtimes for several years, but those aren’t helpful for many use cases such as running a Wasm CLI app.

In the case of a Wasm CLI app, why can’t I just run something like: $ ./uuid.wasm?

Of course, if you have any of your favorite wasm runtimes installed, you could use it to run the module.

  wazero run uuid.wasm

Running that module should print something like “ceefbc9e1b2440d493c4d79f5c957596” to stdout - meaning that when executed with wazero as shown above, this Wasm has access to WASI-provided system resources. WASI can be more precisely configured to selectively grant access to system resources like disk and network:

  wazero run --mount /usr/local/wasm:/usr/local/wasm

With the deny-by-default security posture of Wasm runtimes, all system resources must be explicitly provided to the instance of the Wasm code when it is initialized. These requirements can get a little chaotic if you have certain allowed network hostnames, a variety of host file paths, or specific environment variables you want the guest code to be able to access. Take this snippet for example:

  
// locate the wasm bytecode
wasm := loadWasm("code.wasm")

// create a new runtime with configuration
cfg := wazero.NewRuntimeConfig().WithCustomSections(true)
rt := wazero.NewRuntimeWithConfig(ctx, cfg)
wasi_snapshot_preview1.MustInstantiate(ctx, rt)

// customize the resource access for this module:
resourceConfig := wazero.NewModuleConfig().
		// map `stdin` to this input data
		WithStdin(input).
		// map `stdout` to this output buffer
		WithStdout(output). 
		// map paths
		WithFSConfig(
			wazero.NewFSConfig().WithDirMount("/usr/local/wasm", "/usr/local/wasm"),
		).
		// add this env var to wasm program's ENV
		WithEnv("API_TOKEN", "xyz") 

// get an instance of the wasm module 
mod, err := rt.InstantiateWithConfig(ctx, wasm, config)

Imagine you have dozens of filepaths, environment variables, and networks to specify — it can become a bit cumbersome. In this context, the Hermitfile becomes a useful way to simplify and encode this configuration into a clear format. This snippet is the equivalent Hermitfile which applies the same configuration:

  # Hermitfile

FROM code.wasm
MAP ["/usr/local/wasm"]
ENV API_TOKEN=xyz

You can use a Hermit CLI to build a static executable with the Wasm module and this configuration baked in. The output binary can run on macOS, Linux, and Windows systems!

  
sh ./hermit.com -f Hermitfile -o code.com
# chmod +x code.com

sh ./code.com 
# this program will only be able to read/write /usr/local/wasm
# and connect to api.github.com, redbean.dev, and dylib.so
# and use the API_TOKEN=xyz env var

Don’t read too much into the .com extension, it could go away someday soon. You can already remove it on Unix-like systems, but it is required on Windows. This file is a multi-platform x86_64 executable.

How does it work?

The Cosmopolitan Libc library is used to make Actually Portable Executables, which run on most x86_64 systems (and emulated elsewhere), but is only usable for programming languages that use libc to interact with the system instead of making syscalls themselves. Building a WebAssembly runtime to the Cosmopolitan Libc enables the creation of self-contained executables for a multitude of programming languages.

We chose the WebAssembly Micro Runtime (WAMR) for the initial implementation. Why? WAMR looked relatively easy to adapt to the Cosmopolitan Libc as it is written in C and intended to be portable to a variety of targets. WAMR also has a fully featured WASI implementation, which is needed to connect Wasm modules with the outside world.

The Hermit configuration specifies what resources to share from the host system to the web assembly. By default only stdin, stdout, and stderr is shared. This configuration and Wasm module can easily be inspected and changed. Since hermit executables are also zip files, tooling such as Info-ZIP’s zip and unzip utilities can be used to modify and read hermits.

  
# `unzip` it to see `hermit.json`, which is the configuration used each time the module is instantiated

unzip -p code.com hermit.json

Try it out!

The easiest way to create a hermit is using the Hermit CLI, which can be downloaded here. To specify the configuration necessary to build and run a hermit, you create a Hermitfile . A Hermitfile is a Dockerfile-like format to configure your hermit. The goal of the Hermitfile is to make it easier to package Wasm and provide the Wasm code with additional capabilities: e.g. which network hosts, or filepaths it can access. The FROM directive is necessary to specify the input Wasm module to hermitize. MAP can be used to map in directories from the host file system, ENV specifies environment variables. More details on the Hermitfile format can be found in the README. Once you’ve created your Hermitfile and built your WASI command pattern Wasm module – let’s call it cli.wasm – you can use the hermit cli to turn it into a hermit:

  
# ensure the Hermit CLI is executable
chmod +x hermit.com (on Unix-like operating systems)

# build a new hermit (cli.com) based on the configuration in your Hermitfile
./hermit.com -f Hermitfile -o cli.com (you may need to `sh ./hermit.com`)

# make the resulting hermit executable and run it!
chmod +x cli.com && ./cli.com (you may need to `sh ./cli.com`)

Building the hermit may take a while depending on the size of the Wasm module. Once it’s done, you should have a working hermit. NOTE: Take caution running hermits you haven’t built yourself, they are still native executables. If you have issues making a working Hermitfile, it may be helpful to look at the Hermitfile for the examples modules: cat, cowsay, count_vowels, ubuntu (it’s painfully slow, but yes, that is an ubuntu container translated to Wasm, bundled into a Hermit executable!)

Unfinished business…

In case you were wondering, the Hermit CLI is implemented as a hermit! What better way to verify the functionality of a hermit than to be self-hosting? The Wasm module in the Hermit CLI is written in Rust to get strong type checking and easily link in libraries. The Hermit CLI was a great application to port to Wasm and Hermit as it uncovered the opportunity to solve issues with building and using hermits:

Hermitfile directive ENV_PWD_IS_HOST_CWD was created to instruct the runtime to pass the host current directory into the Wasm via the $PWD environment variable as WASI provides no way of initializing the current directory for a module. We’re continuing to think about other ways to set this and other directives.
Hermitfile directive ENV_EXE_NAME_IS_HOST_EXE_NAME was created to instruct the runtime to pass the path to the executable name into the Wasm via the $EXE_NAME environment variable so the Hermit CLI can copy it in order to make a new hermit.

The Hermit implementation is in its infancy. Contributions are welcome to improve Hermit. We think Hermit is a compelling way to package Wasm into distributable programs and are eager to get feedback. If you think of a feature that would be useful, please let us know!

Known limitations:

Currently only supports one runtime (WAMR). With the Hermitfile as an abstraction layer, we intend to be able to re-use the same configuration and bundle a different runtime where useful.
WAMR is used in interpreted mode so modules with intensive computing, such as zipping, to create hermits is slow.
- we also disable SIMD since the WAMR interpreter doesn’t support it, so the slow-down can be even more substantial.
NET is not implemented to map networking.
LINK is not implemented to link multiple wasm modules.
If Wasm programs want to use the current directory they need to either MAP the current directory directly or use ENV_PWD_IS_HOST_CWD in their Hermitfile and set the current directory to $PWD in their Wasm program.
Only WASI Command-style “main” entrypoint programs are supported as FROM values. Hermit will expect to be able to call a _start function and do the proper WASI initialization. In the future, we intend to add an ENTRYPOINT directive that will allow any export function to be called as the entrypoint to a Hermit.

We encourage you to investigate using it in place of Bash, Python, or Node scripts for internal tooling you’d like to write in Rust or Go. We’re excited to see what you build! Don’t hesitate to reach out, and we’d be grateful for your contribution if you’re interested in improving Hermit.

Steve Manuel, CEO & Co-founder
Gavin Hayes, Senior Software Engineer

PS - Hermit crabs are pretty cool: