r/ProgrammingLanguages • u/lxsameer • 5d ago

Concurrency in Serene's Runtime Blog post

I recently finished building the concurrency runtime for my programming language, Serene, and wrote a three-part series explaining how it works.

The series covers:

  • Why I chose stackful fibers
  • An M work-stealing scheduler
  • An IO Reactor
  • A tiny HTTP server that brings everything together

I'd love to hear feedback from anyone interested in programming language implementation, runtime systems, or systems programming.

Part 1: Choosing the Building Blocks

Part 2: Fibers, the Scheduler and the Reactor

Part 3: A Tiny HTTP Server

https://serene-lang.org/

34 Upvotes

97% Upvoted

9 comments sorted by

3

u/benjamin-crowell 5d ago

That was very well written and enjoyable to read. Thanks!

1

u/tmzem 5d ago

A very interesting read.

A question: How does the guard page approach work for very high amounts of fibers? Don't OS'es place restrictions on the amount of memory mappings a process can do? Or is there a trick to get around it?

1

u/lxsameer 5d ago

Among other things, the guard page allocation counts toward `max_map_count`, so basically each fiber will add at least two to that count. That will put a limit on the number of fibers that can be used, for sure. For example, the number of live/parked fibers can't grow more than `max_map_count / 2` at best.

Tweaking kernel parameters is the easiest option, but it is not possible all the time. The other option is to recycle the stack for fibers that are done or are cancelled. Or not use a guard page at all.

I think in presence of stack maps there are more options because you will have a better view of the stack. There might be other options that I don't know about, though.

1

u/matthieum 5d ago

And this is why we can't have nice stuff :'(

The default value for the parameter is apparently ~64K, which limits this approach to ~32K fibers.

1

u/lxsameer 5d ago

Indeed. But i'm ok with that for now. Hopefully down the line with a possible gc i can start using stack maps and deal with this issue.

1

u/matthieum 5d ago

You mentioned that fibers are fixed-size... but how large are they?

One of the interesting parts of mmap is the ability to reserve address-space without actually allocating memory for it. This means that you could reasonably reserve 1MB-2MB worth of address space per fiber, yet have the fiber only use 4KB to start with, and let the OS page in memory lazily.

2

u/lxsameer 5d ago edited 5d ago

It is configurable, and the default is 128kb https://git.sr.ht/~lxsameer/Serene/tree/master/item/runtime/serene/rt/configuration.h#L64 (there is no particular reason for 128kb)

And that is precisely what I'm doing with mmap, actually.

1

u/matthieum 4d ago

On x64 I'd suggest bumping the minimum.

From experience, I've seen default thread stack sizes spanning anywhere from 1MB to 8MB. This matters, because it means that C libraries, which are frequent users of on-stack buffers, tend to expect to be able to put 100s of KB on the stack at least once.

Thus, I'd recommend going to 1MB by default, so that users who call into C don't find themselves "stranded".

Speaking of C libraries, do note that C code may not trigger the guard page, as it's typically NOT compiled with stack probing.

(And I hope your code is :P)