Browsertech Digest: How Rayon is making CAD collaborative

Hey folks! For this issue I had the pleasure of chatting with Bastien Dolla, co-founder of the collaborative, browser-based CAD tool Rayon.

But first: Taylor and I will be at Strange Loop and the local-first software unconference this month. If you’ll be at either, hit reply and let me know.

Interview with Bastien Dolla (Rayon)

Some tl;dr tidbits:

• Rayon started by storing documents in a Postgres database, but quickly found it to limit their ability to scale and switched to storing on S3.
• Projects in Rayon can contain hundreds of thousands of objects. At that scale, Rust+Wasm is important not just for speed, but also to ensure a tight memory layout.
• There are performance advantages to writing your own WebGL renderer for a specific domain, rather than using an off-the-shelf one. In Rayon’s case, examples include computing hatched areas and dashed lines directly in a GPU shader.

To be fully transparent about my own bias, I should note that Rayon is a customer of Jamsocket (a product I work on), which comes up briefly in the conversation. With that out of the way, here’s the interview.

The Rayon story

Paul: To get started I'd love to hear the backstory for Rayon.

Bastien: Rayon is really the product of both Stanislas and I coming from two different, but similar backgrounds. We were both working in architecture or real estate, and tech.

Stan and I met three years ago. We were both fascinated by AI, and how AI could be applied to design architecture. At the time, Stan was a student at GSD at Harvard, and I was in France doing a startup. He had written some papers that were super interesting, and I met him through that online.

We connected and we decided to do a side project together, with an exhibit in Paris about AI and architecture.

Unfortunately, we opened the exhibit with like 300 people, five days before we were confined to our apartments due to COVID.

But in a good way, that gave us time to think about the tools used in architecture. At the beginning, we were convinced that there was something to do with AI. And the more we talked with people about it, the more we realized that there was a more important need, which was the editor itself.

People were looking for better tools. If you wanted to add AI, you first needed to address the main problem in the room, which was that architects don't have the modern tools they deserve. You needed to build the stack up.

And we thought, who's building this? It seems that everybody wants this, who's building it?

And no one was doing it. So we said, why not us? Let's go, let's build this. And fast-forward two years later we, we left our respective companies to go full time on the idea.

Paul: It sounds like from the beginning, it was going to be browser based. There was never a question of, “should we do this for desktop or browser?”

Bastien: Yeah. The thing that was super common in our industry was people would get a proprietary CAD file, and they could not open it because they don't have the proper software installed.

And if you want to install it, then you have to download a 200 megabyte application. You have to fill in your phone number, your email, create an account, etc. The whole ordeal takes half an hour just to be able to open the file.

You're not going to do that if you're in a meeting, and you need to open a file right away. Since you can't do that, the only reliable way to share your information is through exporting your work as PDFs. But it’s just a view, a static view.

So we knew right from the get go that if you want to be really open to collaboration, people need to share their work via a simple link. And the only common denominator that all people have is the browser.

It's especially important in our industry because most projects have a lot of stakeholders that are in different companies. It's a bit different from software design, where you might have different stakeholders all in the same company. In architecture, it's very common that you have an owner, an architect, the contractors, the engineers.

They're all in different companies, so they don't always have the same tools. So, it's even more important that they have this common denominator where they can open the files, regardless of what their admin IT has authorized to install on their computer. This is why building a browser-first tool was really important.

Paul: How have you found that to be received in the architecture/construction industry? Are people pretty open to switching from desktop to browser to software, or is there friction?

Bastien: At the beginning, we were really cautious about this, especially the online parts.

We thought people would tell us, “oh, you have to be online to access it. This is going to be a problem.”

It turns out, it's really not. People can't really work without internet access anyway and internet connections are more or less everywhere now.

We still sometimes get the objection, “oh, but what if I'm offline?” Especially because people go on work sites where there's not always internet connection.

But solving this has more to do with having a robust offline mode. And you can have that with a browser application.

Tech stack

Paul: Tell me about the stack what you're using.

Bastien: We’re building the client with two technologies. On the one hand, we have a UI that is classic JavaScript/HTML/CSS app.

And on the other hand, we have kind of a game engine, a core engine that is built in rust and compiled to Wasm. So very similar to what Modyfi is doing, or what Figma is doing. We were very inspired by Figma.

And then on the backend, we have two backends. One backend that is very classical for all the metadata layer on models, users, organization, comments, threads, and whatever.

And then we have the multiplayer real-time backend, the one using Drifting in Space and Jamsocket, that is also built in Rust.

Paul: And so on the front end in the WebAssembly, that's also Rust, is it?

Bastien: Yeah, exactly.

I picked Rust at the beginning because it was the language I kind of knew. I didn't know it very well, to be honest, but I didn't know C++ at all. I really didn't want to learn C++, and started using Rust and loved it.

One of the interesting things about having two main languages in the frontend is that we get to compare the way we work in TypeScript and Rust.

One of the things that is hard in what we do is dealing with some very complex geometry logic. The bug surface is huge, because the state of the app is almost infinite. So having the safety that a compiler brings to Rust is hugely important for us. We would have way more bugs if we had done this in TypeScript, I'm sure.

Also, in terms of performance, being able to have a control over the memory layout was something important for us because CAD models can be huge. Especially they can have a very large number of entities.

This means that you to deal with many, many small objects. JavaScript is not great at handling that powerfully. With Rust you get abstraction for free when you work on that, which is very useful for us.

Paul: When you talk about large documents, what sort of size are we talking about?

Bastien: An average size CAD model would have maybe something like 50,000 entities. An entity being, let's say, a line or multiple lines. But it can go up to 500,000 for large models.

So you need to be able to handle that number of entities on the canvas. And of course, since you're an editing tool, you need to be able to do that at 60 FPS, which is challenging!

Wasm/Typescript boundary

Paul: How do you draw the line between what’s handled on the TypeScript side and what’s handled on the WebAssembly side?

Bastien: We try to build the Rust code to be as pure as we can. So the Rust libraries have almost no calls to the browser APIs, save for the WebGL renderer.

Paul: Does the JavaScript side own the source of truth of the document, or does the Rust side?

Bastien: The data lies on the Wasm part. JavaScript is only the go-between. It just acts as a layer of communication.

This is super important, because the data can be very large. You cannot have it in both places, you need to decide where it lies. And so it lies in the buffer of the Wasm linear memory.

Enforcing invariants in the document data

Paul: Something we've talked about a bit before is the idea that, when you have a single backend, you're able to enforce arbitrary invariants. I'm curious what some examples of those invariants are in the context of CAD?

Bastien: An example would be, we have the ability to create layers exactly like in Illustrator. So you draw stuff on layers, but layers can also be nested into other layers to create a hierarchy.

Once you enable that, you risk the creation of cycles, where a layer might be the child of another layer that is also its parent, which is bad.

This is really easy to prevent in a single user application. But if you're in a multiplayer application, you might create that situation by mistake because people are editing two different layers at the same time and they can unwillingly create a cycle.

The only way to guarantee that this is not going to happen has to be at the data layer in the backend, to make sure we never persist a model with that cycle.

The more complexity you add, the more invariants you add. So you need to be able to check all this in the back end.

Paul: How do you balance correctness with responsiveness? Do you apply changes optimistically before the server has responded?

Bastien: The way we work is very similar to Git.

We create commits, which are diffs of the state of the model. Those commits are sent to the database, which checks if they're viable, which means they don't break any of the invariants. If so, they can be applied to the model. And if they are applied, then they're sent back to all the other clients connected to the same model, which will apply those commits to their own history.

That's basically how it works. If two conflicting commits arrive to the backend, we apply them in the order they arrive in the backend, and then they're sent back to all the clients.

Of course, it has implications for the undo/redo stack. A good example being: I've created a rectangle which is red. I'm changing the color to blue, but then somebody deletes it. So what should happen when I undo the color change?

Should I recreate the rectangle that has been deleted by another one? Or should I just not do anything?

And what happens if the person that has deleted undoes the deletion? Should I get the rectangle back? Yes. But should I be able to now to undo my color change? Probably.

So you need to handle all those cases, which is not that easy. But I think this is what's really interesting about multiplayer apps.

Paul: If a client makes a change but another concurrent change has invalidated it, how do you handle that situation?

Bastien: If a commit is invalid due to another client's change, it will be rejected by the central authority of the backend, and the client that emitted it will need to reverse it.

This is where having a unique session-lived backend is so important, because you get that linear application of the modification. You get that sequence to make sure you still have causality and everything is in the correct order.

That's how it works basically, very simple.

Paul: How is the data ultimately persisted?

Bastien: At the beginning, we were using a relational DB to store the model's data.

But rapidly we realized that it was not viable, because there are so many entities. To store a hundred models, you're already in the millions [of rows] and very, very rapidly you're in the billions.

It's very hard to scale a relational database in that way. But also it's not needed, because the data is very local to the model. It's way more efficient to have one database per model in storage. So basically, storing into a file.

Which was kind of a downer, because so many desktop applications have tried to be “cloud first” by just setting the file on a remote server. They just run the same application that you would have a new client and just store files. It felt like, “ah, this is not very cloudy”.

But it was definitely a good choice. We store models as files on blob storage, which is way more scalable because you don't create this huge single database. Also, it's convenient because we actually need the whole data of the model in memory on the backend to be able to check those invariants.

We have the client running the backend, and that way you can check everything exactly the same way you would in the client. Once you put all this together, it makes a lot of sense to just have files that you load once, when the model is opened, and that you periodically save back.

Paul: When you do a migration, do you do it all at once, or as-needed when files are opened?

Bastien: We decided to do data migration lazily on the clients. When a client receives a model that has an old format, the first thing it does is create a commit that migrates the model to the current format.

I suspect that this is true to any design tool: if you had to look at the mass of models that you have in storage, the percentage that are actually active is going to be a very low number. People create a lot of models all the time, copies or stuff that they won't need anymore.

So it's more efficient to do it this way, and also it's less stressful because you don't need to do a big migration.

Paul: In terms of the client and server, it sounds the validation logic exists on both, is that accurate? Do they share all of the data model code?

Bastien: That's one of the great things with having the code based in Rust. We can use the same crates on the backend and on the frontend.

That way we reuse a lot of the serialization, deserialization, all the structures. It's completely shared.

That's something I really like about this architecture.

WebGL Rendering

Paul: I'd love to dive into the the graphics side too. You mentioned that the WebAssembly doesn't interface a lot with the browser, but that the WebGL canvas is an exception?

Bastien: Yeah, exactly. The main exception is the canvas.

One of the big things we do is vector graphics. It's quite a specific part of computer graphics. And to build something that was performant, we had to build our own renderer.

So the Rust part actually has its own renderer.

Paul: How did you decide on WebGL over the higher-level canvas or even SVG?

Bastien: We did some performance benchmarks to try the different solutions out. We also looked at some feedback from other tools. We knew, for example, that Figma had built their own renderer, which is very impressive by the way.

We really wanted to have a solution that could go a long way, so we thought it was better to just do our own WebGL renderer. The nice thing about building a low-level renderer is you can really adapt it to your domain. For example, if you need dashes or hatching, you can build a custom shader and do it on the GPU instead of, you know, drawing a million lines.

Paul: Did you write your own text render, or how do you do the text?

Bastien: We did. Text rendering is a huge and difficult problem. But the constraints and requirements we have for CAD tool are way lower than for design tool that is very focused on text.

Text in CAD is really about communicating. So we did a very simple, as simple as we could, text renderer on the GPU.

Paul: Are you taking an SDF (signed distance field) approach?

Bastien: Yeah, we do. We're using MSDF (multi-channel signed distance field), which is nice because it's very fast.

But it has its own limitations. We're actually starting the work on a new renderer which is going to be quite different. I think we will render text as vectors.

Paul: Interesting, I'm curious what the limitation you hit was.

Bastien: The problem with the MSDF approach is that it's not that easy to render the glyph class on the client. So usually what you do is you limit yourself to a certain set of glyphs and pre-rendered those to a texture.

That has a limitation, because people need certain glyphs that you don't support, especially for non-European languages, where the number of glyphs can become very large.

Undervalued libraries

Paul: Are there particular crates or JavaScript libraries that are in your stack that you think are undervalued?

Bastien: On the TypeScript side, we use a lot XState, which is a great finite state machine library. Finite state machines are very useful to represent the logic of a CAD command, because often the commands are very complex in the way that you need to handle different events, mouse events, keyboard events in different states. Which is exactly what a FSM models. So we use them a lot.

Paul: Is there anything that you think people should know?

Bastien: One thing we think is super important and really interesting when building a centralized cloud based application the ability to give a full version control experience to CAD users. The idea that a user should have a full history of all the modifications done to a model, and the ability to create branches and merge them back, something that is very familiar to software engineers.

That is completely non-existent in the CAD world, where it's as important because you're building something in the real world. If somebody makes a mistake at some point, it's really important that there is a trace of who has done what, when.

We believe that would be incredibly useful to people in the industry, and be a real game changer.

That's something we've been building towards step-by-step. It's not done yet. But we have the building blocks.

Paul: I love that. I think that's to me, more interesting than a lot of the real-time collaboration, because when I think about how I collaborate with people, day-to-day, so much of it is this asynchronous, GitHub-style workflow where I do some work and then package it up nicely as a pull request and then say “you can comment on this”, you can kind of act on a set of changes as a first class item. And it sounds like that's exactly the type of workflow you're trying to build for here.

Bastien: Yeah, exactly. But you also don't want to have the complexity of, you know, rebasing your branch on the master, squashing your commits and so forth. People in the industry are not going to put up with that.

So if you build your multiplayer on top of a coherent commit history, then you can have something that starts to look very similar to Git version control. I think that will be very important in the future for Rayon.

Thanks for reading! Until next time,

-- Paul