DX Spotlight: proto by moonrepo

“One tool to rule them all”

As with any creative profession, software development requires a diverse set of tools to get the job done, but using numerous disjointed version managers for various languages and runtimes, each with unique workflows, CLI commands, and config files reduces developer productivity and increases frustration. Multiply that as project and team sizes grow larger and more complex.

This is the exact reason why moonrepo built proto, a unified toolchain and CLI that manages versions for multiple languages, dependency managers, and CLIs across various platforms. proto comes with built-in support for languages such as Python, Rust, and Go and runtimes such as Bun, Deno, and Node (npm, pnpm, yarn), and its feature set includes automatic version detection, granular project configuration options, and high-performance.

This ensemble of capabilities is certainly impressive, but the value of a toolchain is largely driven by the breadth of tools that can be incorporated into that chain, delivering more benefits and options for end users through their combined effect. This, however, requires significant investment in time, effort, and ruthless roadmap prioritization. But what if the community or customers were empowered to build extensions into the core offering? Now that’s a force multiplier any product manager can get behind…

And this is why I’m most excited by proto’s pluggable architecture.

Plugins For the Win

proto empowers its users to augment the toolchain with custom plugins, enabling new and non-standard tools to join the party. This is a powerful feature, but offering a plugin system is just one aspect; another challenge is delivering one that operates smoothly and doesn’t limit users in how those plugins are created, distributed, and integrated. Often plugin authors are forced to use the same language as the product they’re extending or fight through the “valley of suck” to learn some kind of bespoke domain-specific language (DSL). Additionally, it’s essential to streamline the distribution of plugins, making the process smooth and straightforward for both creators and end users, with effortless integration into the core product, while maintaining a fundamental level of security for the host system.

So how does proto address these concerns?

proto supports plugin creation through TOML files, which is a simple and solid approach for situations where a static configuration file is sufficient for integrating things like pre-built CLI binaries that can be downloaded, and integrated into the toolchain.

However, recognizing that users may need a more powerful and expressive option for plugins that demand a more complex set of business logic and behavior, proto turned to WebAssembly (Wasm) to bring the capabilities of general-purpose programming languages to a plugin creator’s fingertips. proto itself is written in Rust, but it supports plugins in any language that can compile to Wasm, which includes JavaScript/Typescript, Go, Haskell, C/++, Zig and of course…Rust. Note, there’s an element of future-proofing here as the plugin architecture will almost automatically inherit any languages that can target Wasm in the future.

Aside from plugin creation, third-party developers also need an effective way to distribute their plugins to end users in order to ensure a thriving ecosystem. Here again, proto is able to harness the power of WebAssembly, which provides a compact, portable binary compatible with any platform. This opens the door to any number of distribution methods including registries, cloud storage, or even bundling the plugin within the core application itself. proto recommends a simple and effective workflow that entails building the plugin as a .wasm file and publishing it as an asset as part of a GitHub release.

But what about security? Wasm modules are limited in their ability to access anything (e.g. filesystems, network connectivity, etc.) on the Host system unless given explicit permission, and thus, proto plugins are similarly restricted. This is a feature and not a bug, as it endows the proto plugin system with a capability-based security mechanism out of the box, enabling it to open up host capabilities (e.g. executing commands, logging, managing environment variables, etc.) to plugins selectively through an apparatus called Host Functions.

Gaining the Advantage with Extism

WebAssembly has proven to be a transformative force for proto’s plugin system, yet its complexity demands considerable effort to piece together all required components. This complexity is precisely why proto has embraced Extism, a framework crafted to tap into the power of WebAssembly and streamline its integration process.

Extism is an amazing product that has accelerated our development and iteration speeds. Without Extism, a lot of time and resources would be spent understanding the intricacies of the WASM ecosystem, all of the different tools, runtimes, and more. Instead, we can simply import the Extism PDK, define and export a Rust function, compile to WASM, and call that function from our host runtime! Thanks to Dylibso, this process was extremely easy, and instead of focusing on the implementation details of WASM, we can focus on the actual product.

— Miles Johnson,

Check out this post to learn more about how Extism enhances the developer experience of using WebAssembly

Onward

proto is making significant strides in redefining toolchain management for developers, delivering a flexible version management system alongside a powerful plugin framework. By leveraging the strengths of WebAssembly, it offers a secure and seamless way to integrate an extensive range of tools, enhancing the developer experience. The growing catalog of third-party plugins is a testament to its impact, and it will be fascinating to observe the continued expansion and success of this ecosystem.

Dominique Saulet, Director, TPM