Benjamin Egelund-Müller

Stateful functions as a service

Thu, 20 May 2021 14:31:02 +0000

One of the trends in cloud computing that I’m most excited about is stateful functions as a service (FaaS). The technology is still in its very early stages, but I think it’s the next leap for serverless.

Serverless functions today

The serverless functions we know today are services like AWS Lambda and Google Cloud Functions. They essentially consist of one massive load balancing layer that receives all requests (for HTTP functions) and events (for event processing functions), and routes them to a machine that is running your function, dynamically spinning machines up and down depending on load.

The result is a lovely developer experience where you just bundle up your code and hand it over to the cloud provider. You don’t have to worry about where your code runs or how it scales. And when there aren’t any requests or events to handle, it doesn’t cost anything!

These functions are stateless, meaning they don’t have any persistent disks and don’t have any shared memory. The way they can keep state is by connecting to an external data system, such as Postgres, Redis or FaunaDB. That’s not a problem for many ETL and CRUD applications – most services don’t rely on local disk or memory between requests anyway.

Use cases for stateful serverless functions

Nonetheless, there’s a long tail of interesting applications that rely on local state, and I think they’ll become more popular in the future. They’re things like:

Streaming aggregations that roll up many events over a period of time. For example, counting unique visitors per page in real-time.
Streaming joins that pair events from different sources. For example, joining observations with matching timestamps from two different IoT sensors.
Real-time collaboration, where many users edit the same object and receive updates. For example, multi-person document editing or multi-player games.
Low-latency metadata lookups at scale. For example, routing tables or looking up user permissions.

Today, you need machines with dedicated memory and disk space to handle these use cases. And you might also need a distributed configuration store like Consul to manage sharding, or a stream processing system like Flink to coordinate and shuffle data between machines.

How will they work?

A lot of the use cases for stateful functions depend on what kinds of guarantees the platform will be able to provide. The dream is for a function invocation to treat global state as an in-memory object that it manipulates without interference from concurrent functions. In practice, that’s probably not feasible and you might need some map/reduce-style logic for merging distributed state changes.

There are some systems today that can claim they’re a “stateful FaaS platform” (see the resources below), but it’s far from a solved problem. There are many things to consider, such as consistency, isolation, exactly-once processing, latency, hot keys, etc. Under the hood, the platform will have to coordinate where to run functions depending on which physical machines hold what state, and also how to merge changes consistently. Data will have to be shuffled and replicated between machines depending on load. These problems become even harder if you want to run stateful functions in different geographic regions to get lower end-user latency.

A world computer

If your serverless functions can manipulate global state without worrying about scalability, consistency or latency, you can effectively treat the stateful FaaS platform as one giant machine that never goes down. You could get rid of your external database altogether – just write data to the global state, and read or update it in later function invocations. I like the idea of a “world computer”, a term the Ethereum project uses (Ethereum’s smart contracts are essentially stateful functions, although not scalable).

In theory, it’s the end-state for cloud, where you write your code like it runs on one giant server that scales infinitely and is responsive across the globe. In practice, there’s probably going to be caveats, but it’s going to be exciting to see where it leads.

Resources

Here are some useful resources for learning and thinking about stateful serverless functions:

Three inspiring developer experiences

Fri, 23 Apr 2021 10:00:00 +0000

I love trying out new developer tools. Since I started building Beneath, I’ve probably tested dozens of developer tools looking for ~~great ideas to steal~~ inspiration.

One thing I’m always on the lookout for are features that make me go “wow”. Developer tools tend to cover a lot of complexity, so that kind of experience isn’t all that easy to create. In this post, I’ve put together three “wow” experiences from different developer tools that I think are a great source of inspiration.

Example 1: Project setup in Vercel

The first example that comes to mind is Vercel’s project setup. Vercel is a platform that helps frontend developers deploy websites.

The “wow” experience is unmistakable the first time you create a new project in Vercel. You just select a template, connect to a Git provider, and boom! It creates a repo, builds, and deploys the site globally right away. It feels pretty magical to have a website with CI/CD up and running before even pulling the source code.

Your browser does not support HTML video.

Vercel bundles several best practices for modern web projects, such as CI/CD, branch deploys, serverless functions, and edge-caching. Today, even for a personal web project, those features are awesome to have, but normally each of them add more complexity. I think it’s impressive the way Vercel has managed to combine all these features in such a surprisingly simple way.

Example 2: Python package management in Replit

The second example I want to highlight is Replit’s package management for Python. Replit is an online IDE that lets you write and run code in the browser. It has many neat features, including the ability to run web services and collaboratively edit code. I’ve used it several times for user workshops for testing Beneath’s Python developer experience.

The “wow” experience I want to highlight is the way Replit automatically installs Python packages. If you’re in a Python environment in Replit, and you try to run a Python file that imports an external module, Replit will detect if it’s not already installed and add it to your environment using Poetry, a brilliant Python package manager.

Your browser does not support HTML video.

In contrast with Vercel’s project setup, this is certainly a small feature, but I’ve had some traumatizing experiences with Python package management, and I almost universally forget to run pip install ... when running Python code, so when I first encountered this feature, I couldn’t help but smile!

It was also my first exposure to Poetry, a tool I’ve since used for all my Python projects. While credit really goes to Poetry for a lot of the hard work of this feature, such as getting rid of requirements.txt, I think it’s definitely clever how Replit spotted the opportunity to leverage Poetry to transparently provide such a delightful feature.

Example 3: Ad-hoc queries in BigQuery

The last example I’ll share in this post is running ad-hoc queries with Google BigQuery. BigQuery is a serverless data warehouse that’s part of the Google Cloud Platform. As with most data warehouses, its core feature is running SQL queries that aggregate or transform large datasets.

Even after years of using it, BigQuery continues to elicit a “wow” from me when I need to quickly run an ad-hoc SQL query on a large dataset. Unlike most data warehouses, BigQuery is completely serverless and so massively parallelized that it runs most queries in seconds from a cold start regardless of the data size. I just open the console, type a query, click run, and get a crazy fast response.

Your browser does not support HTML video.

In this video, I ran a query against one of the built-in public datasets. It aggregated a 509gb dataset with more than one billion rows in 2.8 seconds from a cold start with no prior configuration. I didn’t have to deploy a cluster, or even consider the memory or disk requirements of the workers. BigQuery console isn’t a particularly great user experience, but it’s hard not to be awed at the scale of compute power BigQuery is able to unleash in an instant.

Wrapping up

The three developer experiences highlighted in this post are pretty different. In the case of Vercel, they have managed to integrate several complex features in such a thoughtful way that the end experience becomes simpler. In the case of Replit, they have created a delightful affordance by cleverly baking in a powerful package manager. And in the case of BigQuery, the serverless query engine allows them to run surprisingly fast queries from a cold start. Despite these differences, I think they all share a delightful simplicity.

I feel like I’ve only scratched the surface of “wow”-worthy developer experiences. In this post, I’ve focused on modern developer services, but it’s crazy to think about the magic embodied in many of the tools we take for granted, like compilers and text editors. I’ll be writing more on this topic in the coming weeks.

I’d also love to hear about your favorite developer experiences. Share them with me on Twitter. If I get enough good input, I’ll compile a longer list!

Consuming → Producing

Thu, 08 Apr 2021 14:07:52 +0000

Welcome to my new blog. I’ll be using it to write about and reflect upon the things that occupy me, such as data science, data engineering, developer tools, my experiences as a founder, book reviews, and random ideas.

My motivation for starting this blog is to produce and share more content. I feel that it’s too easy to passively consume content on the internet without engaging, and sometimes even without reflecting. I hope that having a place of my own to write can make me a better internet citizen.

I hope you will find something interesting and useful here.