DX Spotlight: Shopify's Function Runner

Shopify is a household name in the world of e-commerce, powering storefronts for millions of sellers around the world. One of my favorite data vizualizations is their realtime Black Friday / Cyber Monday sales tracker. On that weekend alone, Shopify infrastructure and tools supported merchant sales topping $7.5 Billion!

I’ve long admired Shopify for its dedication to developer experience and its drive to enable platform innovation even from outside the company. From the outset, Shopify has been a developer-focused technology company. Liquid templates enable deep customization to a store’s look and feel, a broad set of APIs make integrations a breeze, and their App platform lets third-parties bring new tools onto the platform and power a lucrative marketplace.

But, over the past few years, Shopify has been adding another groundbreaking capability to their platform: Functions. As e-commerce changes and merchants create new business models and invent new products to market & sell, more customization was necessary. To keep up with near infinite customization possibilities, Shopify needed a way to give developers the most flexible solution for extensibility: run developers’ custom code.

Wait a second… run arbitrary, untrusted, third-party code?! 🚩🚩🚩

Until recently, this would have been a major undertaking - the isolation and security requirements meant building significantly complex systems and implementing techniques & mitigations at the bleeding edge of information security research. However, Shopify saw an early glimpse of the future in a technology that provides just the right kind of security and isolation: WebAssembly.

Here’s a great overview from Shopify about the platform and its capabilities:

By using WebAssembly (Wasm), Shopify can execute customer code directly in the flow of commerce. Giving developers maximal extensibility has opened the doors to some incredible new tools for merchants to fine-tune their stores to increase sales & improve customer experiences. One such application comes from Shopify Functions developer, Sammy Isseyegh, who has been building Discount Kit on the Functions platform:

Developing with Shopify Functions really is a game changer for the Shopify ecosystem. With our application Discount Kit, we run our Shopify Functions (aka Wasm files) on Shopify infrastructure (to apply discounts). Discount Functions run every time a shopping cart changes (a lot!) and need to run FAST, so Shopify have imposed quite stringent requirements on file size, Wasm instruction limits, input data size. We’re really excited that the tooling around Wasm has become good enough to call it “maturing”. It’s almost so good that you don’t even need to know you’re using Web Assembly: npm run shopify app generate -> npm run dev and off you go, you’re running Wasm in production.

— Sammy Isseyegh,

What’s so intriguing about Shopify Functions is that the logic lives in the hot-path, between a customer and their wallet - so things need to move fast. You can’t put a slow operation in the checkout flow, even slight interruptions are known to reduce sales conversions and increase abandoned carts. So Functions must execute complex logic at breakneck speed. To help developers shine a light on the performance characteristics of their Functions, Shopify provides the function-runner, a CLI tool to simulate customer actions (checkouts, cart updates, etc.) and run Functions locally.

Let’s hear what Sammy has to say:

Using the function-runner you can throw some fake shopping carts at your newly built function, and it will give you some interesting data back: how many Wasm instructions were run, the file size etc, without having to interact with any of the underlying tools. There’s even a handy flag --profile which will spit out a .perf file/heatmap to see where your instructions are going (use a debug build to see useful function names instead of numbers!). There’s only so much you can test and validate in a development environment and that’s why observability in production is so important. It even allows you to be proactive in your customer support (the people who matter at the end of the day!). This is where projects like Dylibso’s Observe SDK is so exciting: it brings that same visibility to production with so little effort (thanks to automatic instrumentation).

By giving developers the insights they need to squeeze out all the performance they can while they build Functions, Shopify helps merchants sell successfully, and swiftly transact with customers.

It’s inspiring to see Shopify put so much effort into developer experience by offering their users the tools they need to build the best products they can. As one of the WebAssembly ecosystem early adopters, Shopify is setting a high bar for what it means to provide a comprehensive Wasm offering, and I’m excited to see what other innovations they have up their sleeves!

— Steve Manuel, CEO + Co-founder

P.S. As Sammy mentioned, Dylibso provides a production-grade observability & monitoring solution for WebAssembly users! It is open source, and available for host applications in Rust, Go, and JavaScript on GitHub. We’ve also added function & memory allocation tracing in our own fork of Shopify’s function-runner, which you can try out here: Automatic Instrumenting Function Runner.