React Server Actions - Versioning, Filenames, and Other Considerations

I’ve been building a new thing called Ampwall since the beginning of 2023 and it’s going quite well. I announced it via Twitter post on September 29 (same day as Woe’s fifth album release!) and the response was intense. I’m doing it full-time and I am optimistic.

Ampwall is built using Next.js. Specifically, I’m using Next.js 13’s divisive App Router. I like it quite a bit. I server-rendering, I think we ship too much JavaScript to clients and make our lives as engineers harder when we insist on writing APIs for things that should just use server templates. But I also love React and TypeScript, so I don’t want to give up on these if I don’t have to. Next.js 13’s embrace of React Server Components and Server Actions ticks all the boxes: server rendering with TypeScript and React! Beautiful.

Vercel announced Server Actions were stable last week as part of their Next.js 14 release event. This has spawned a lot of conversation, critique, and drama, most of which I find rather dull or knee-jerky or immature. But one topic piqued my interest: versioning. How do you version server actions? What do we need to consider when deploying new versions?

The problem

Version skew is well-defined and understood by software engineers. It can be a challenging problem for products with long-running client sessions. One benefit of the explicit API client-server relationship is the explicit definition and publishing of public API interfaces. Experienced engineers intuitively understand this. It’s typical for changes to be flagged during code reviews: “This will break old clients”, we should mark this argument optional and watch logs until 99% of users are on the new version”, or “We should put this endpoint in a v2 namespace”.

Server Actions essentially create RPC endpoints in Next.js servers. This is magical and it works wonderfully. When you define a function with the magic 'use server' directive or put it in a 'use server' file, it will be executed by the server.

'use server';

// This can be called from the client but it will execute on the server
export async function foo() {
  return 'bar';
}

As I understand it, this works by creating an ID representing this function and outputting code that, from a client, makes a POST and references the ID as the value of a header called Next-Action.

So then: how do we version Server Actions? More urgently, if I deploy five versions of my app across five days and a client fails to reload their window past day 1, what will happen when they interact with foo()?

What’s in a name?

The answer to all of this comes back to how the Next-Action ID is generated. From what I can tell, the ID comes from this function. It creates a SHA1 hash using a combination of the file name and function name. This matches what I’ve seen: a function called foo in a file called bar will always have the same ID, regardless of its implementation. So where versioning is concerned – if we want to introduce a breaking change to a Server Action, maybe adding a required argument to unlock some new behavior without breaking folks who haven’t reloaded yet – we can create fooV2. We could do something like this:

'use server';

export async function foo(optionalArg?: string) {
  if (optionalArg) {
    // do the new thing
  } else {
    // do the old thing
  }
}

export async function fooV2(requiredArg: string) {
  return foo(requiredArg);
}

But this is a mighty footgun: renaming a function changes your server’s public interface. “Well, duh, of course, just like renaming an API endpoint is a breaking changes.” Yes, but React Server Actions are a new paradigm with a fuzzy line deliberately drawn between client and server, invisible to an engineer working in a Client Component and easily confused with a plain ole backend async function if you’re working in the server.

React Server Actions bring us one simple refactor away from introducing version skew in a way that might be extremely surprising. Reorganize your files? Fix a typo in a function? Rename something to be more explicit or fit its purpose better? Breaking API changes.

Don’t give me bad news

Ok, so you fixed a typo in a function while doing something else. It’s so trivial that nobody thought about it during a code review, you obviously spelled “cart” as “crat” and that needed to be fixed. Version skew has been introduced but at least you’ll know from logs, right?

Nope. When a client POSTs to a Server Action function that does not exist, the server swallows it silently and returns 200. You will not know something is wrong unless the client that called it is looking for a response and fires a loggable error.

New day, new best practices

What do we do with this knowledge? I’m doubling down on some approaches and considering others.

First, the things I’ve been doing since day 1: no anonymous functions with use server, no Server Actions defined in files that aren’t explicitly dedicated to them. I put all of my Server Actions in a folder called controllers because I treat them like the C in MVC. This limits the likelihood of having to move a Server Action to another file and changing its hash.

Next, I’m considering something else: the server action itself is just a one-line wrapper around an implementation.

'use server';

export async function fooV1(input1: string, input2: number) {
  return fooImpl(input1, input2);
}

This offers a few benefits: it makes Server Actions look weird in a way that will hopefully stand out to someone reviewing, it decreases the likelihood that some well-intentioned engineer will be mucking about in a file where they can accidentally cause problems, and it limits the responsibility of the Server Action to the routing layer of the server.

How could this be improved?

It seems clearly to me that we need a way to explicitly set or seed a Server Action.

'use server';

// Great opportunity for a decorator
@actionKey('foo')
export async function foo() {
  // impl
}

// Or just add it as metadata here?
foo.actionKey = 'foo';

// Maybe a string literal?

export async function foo() {
  actionKey`foo`;
}

This breaks the dependency between the declaration and the public interface, or at least gives us a way to control the output.

Is this really worth it?

When I posted about this on Reddit, my recommendation that we need a way to control this was met with a great quip: “Like, say, a URL?”. That’s a good point. If we’re explicitly keying our endpoints, we’re sort of just creating an alternate path to a public API, aren’t we?

I’d say we still benefit from avoiding the ritual boilerplate of API declarations. Server Actions, for me, have been part of a mighty improvement to my Developer Experience, and I don’t think that explicitly keying them would negate their benefit. If anything, explicit keys would help engineers understand that their Server Actions are already API endpoints. It would make them more predictable, less magical, and help folks doing code reviews or tracking regressions. I think we’d be able to find a way to add an eslint rule to require explicit keys.

I’m going to keep using Server Actions but I’m watching this closely. They’re still young and I’m optimistic that the experience of using them and managing projects will only improve over time.

Next.js App Router Client Cache Busting

UPDATE: 24 hours after posting this, Vercel responded to the outstanding issue. A few days later, they started a discussion about it. They’ve committed to improving the caching experience.

ORIGINAL POST

Next.js 13’s App Router has caused no shortage of controversy. Lots of people are upset about Server Components. Some blogs seem to think the sky is falling. Me? I love it. Love. Big heart eyes, harps strumming, floating to a cloud.

As much as I love React, as confident as I feel with it and TypeScript, I think that the benefits of the SPA are lost on most products. I think that the massive amount of code we push to clients is embarrassing. The rituals of API requests, the complexities of client state management… I’m over it. At the risk of sounding like an old man: I miss Rails. I miss having a server that can talk to my database and spit out HTML that loads quickly on clients. One deployment, one environment, way less boilerplate. Obviously, as a professional React developer, there are tons and tons of amazing things that simply cannot be done with server-rendered pages; like I said, I love React. But I think we’ve reached peak SPA.

React Server Components give me what I want. The server talks to my database, talks to APIs, prepares data and renders as much HTML as possible. Then it lets me elegently drop into the client as needed. React Server Actions give me simple RPC calls. The move to RSC is accelerating modern approaches to CSS-in-JS (I’m currently working on a personal project using Panda and it is loevly!) and even though things are changing fast, it’s giving me what I’m looking for.

But the App Router has its share of problems. Among them is its highly opinionated caching rules, particularly its client-side caching rules. There is an ongoing discussion about this in their GitHub issues. It is the single most commented open issue in the project right now, 286 comments at the time of my writing. You can find it at https://github.com/vercel/next.js/issues/42991. Vercel so far have not responded to it.

I slapped together a quick workaround for the problem. It relies on revalidatePath, a function that Vercel limits usage of in its free tier, so it will not be for everyone. But if you’re hosting somewhere without such a restriction – maybe you’re hosting on a Node.js server so you’re not as concerned about counting function invocations – here’s my approach.

High Speed FTDI + Android Comms (OR) Why Am I Always Reading 0 Bytes?

I’ve been working on a project with some very unfamiliar tech. The project involves communication between a new Android app (Kotlin) and an FTDI 232R connected to an Arduino. I encountered a problem that baffled everyone on the team for weeks and was about to be labeled a “general incompatibility between FTDI devices and Android apps” until I stumbled upon the solution. Before I describe the solution, I’m going to document some basic details. While working on this, I was unable to find any examples of people having this problem, so there was an intense feeling of isolation as I struggled on and off for weeks to resolve it. My hope is that this might help someone identify the same problem in their system.

tl;dr

If you’re experiencing mystery “0 bytes available” errors, you might need to change your latency timer setting. The problem is described here. I also strongly recommend you read the longer document from which that excerpt is drawn, AN23B-04 Data Throughput, Latency and Handshaking. We immediately resolved our issue with a call to setLatencyTimer((byte) 1); and very small reads (64 bytes at a time, no more) but ultimately settled on an event character and larger reads. Full details below.

Detailed Notes

Our Arduino’s firmware is capable of sending a few different messages across the wire. Each message is small, anywhere from 16 bytes up to around 256. Most of these are on-demand: send a command from the application, the Arduino decodes it, then it sends one message in response that is either an ACK or the data that you’ve requested. There is one exception: one particular message from the app will trigger the start of an infinite stream of 44-byte messages at a frequency rate specified in the request. In this case, the Arduino is reading sensors, performing some basic analysis, and spitting it out across the wire for the app to do with as it pleases. The app reads this constant stream of bytes, does its own analysis, puts it on the screen, etc,…

Our minimum acceptable streaming rate is 300hz but we hope for closer to 500hz or greater, so our baud is currently 460800.

We encountered an issue whereby the app was constantly being told 0 bytes were available for read. The problem was extremely inconsistent and weird. The following were true:

  • We could ALWAYS open the port from our app.
  • We could ALWAYS transmit successfully from the app to the Arduino. We knew this because logs on the firmware indicated that the right bytes arrived in the right order.
  • We would SOMETIMES receive the correct response. It was all or nothing: sometimes we would query for available bytes and be told 0, other times we would see the expected number.
  • We could RARELY start the data stream. Once we sent the message to start streaming, the app would always believe there were 0 bytes available for read. Once that state was encountered, no other messages would be sent across the wire until we rebooted the firmware. It seemed to be more likely to fail as our streaming rate exceeded 100hz. Our target was 300hz or greater, so this was a serious problem!

Adding to the mystery, this seemed specific to the FTDI chip. Our first draft of this used the Arduino’s programming port for serial data transfer at 115200 baud. We were losing a lot of packets from the lack of flow control but it never failed to respond to messages.

More troubling was the fact that a C++ test application seemed to communicate correctly. This pointed towards a code problem with the Android app.

We tried three different libraries in attempts to resolve this. Those were:

  • usb-serial-for-android - An open-source library that is pretty well maintained and offers a lot of features. Unfortunately, it doesn’t support automatic flow control, so we worried we wouldn’t be able to use it long term.
  • UsbSerial - Another open-source library. This one is not nearly as well maintained and it has quite a few open issues that describe some pretty heinous bugs. I opened an issue after I found that calling the wrong method during initialization would result in all your sent messages being replaced by two 0 bytes for every one byte in your message! Brutal. It supports flow control but it has so many problems that I unfortunately couldn’t recommend it, even if it supported what we needed.
  • FTDI’s official d2xx - The official closed-source library for FTDI devices. It hasn’t been updated in two years but by virtue of being official, we expected it to be more reliable or at least more full-featured. The closed-source part is a bummer and I think it would be a much better library if not for that, but that’s another story. This was the library we wound up using and we will continue to do so.

All three of these libraries exhibited the same behavior! This started looking like a major issue with FTDI devices. I ordered a few Prolific PL2303-based serial cables to test as an alternative but kept researching in the meantime.

I began looking at FTDI’s official test apps and their example Android app code. The example code is… not… great… but in taking notes, I came across a mysterious call to setLatencyTimer(). This led me to this, which appeared to describe our problem exactly. It specifically remarks, “While the host controller is waiting for one of the above conditions to occur, NO data is received by our driver and hence the user’s application. The data, if there is any, is only finally transferred after one of the above conditions has occurred.” I did some more reading and found the longer AN23B-04 Data Throughput, Latency and Handshaking which explained this and many other concepts. This document was particularly enlightening. I feel like the embedded software development world is full of extremely dense, unapproachable technical specs that assume a ton of highly specific knowledge; by comparison, this document was a breath of fresh air and explained things from a high enough level that I came away feeling more capable of anticipating behavior as I continued troubleshooting.

It appeared that we were never hitting any of the three rules fast enough to trigger a read. It still doesn’t totally make sense to me; I feel like we should have eventually hit 4Kb to trigger the send, but maybe I never let it sit long enough to get there? Or maybe there was another timeout value that was clearing the buffer before then. What I do know is that if I set the latency timer down to 1ms and ensured we never requested more than 64 bytes at a time, our data read problems went away. We could stream at 500hz and messages would usually start showing up as soon as we hit the button. This change was as simple as setLatencyTimer((byte) 1); and making sure that we never requested more than 64 bytes during a call to read. The immediate problem was solved and it was clear that we did not have some incompatibility between FTDI and our Android app.

I say that it would “usually” start showing up because it still exhibited strange behavior. Very often, I would start the stream through the app’s interface and nothing would happen. Then I’d send another message (“get hardware version”) and not only would it get my hardware version, it would also recognize that data was streaming in. Other times, I would request our largest payload, a system configuration, and it would return 31 bytes of the 200+ we expect. Just like with the stream, I’d send any other message (“get firmware version”) and it the remaining 200+ bytes would show up.

I wound up making a few other changes to resolve this problem and improve the behavior overall.

First, using more information gleaned from the Data Throughput, Latency and Handshaking document, I thought that we be better off using the FTDI’s support for Event Characters than the latency timer. Our encoding rules use a 0 byte as a delimiter, so it was an obvious choice. This allowed me to increase our maximum read size up to 256 bytes, which helped in the event that our read loop was delayed and we had to quickly get through a backlog of data. (I could probably go higher but I’m being pretty careful right now, I want to keep things moving.) Finally, I modified the read loop to also be responsible for writes, added a FIFO queue for outgoing messages, and (crucially) a 50ms timeout of the loop after every single message sent. The 50ms timeout was the most significant piece – it was the final change that ensured that we stopped seeing partial messages or messages that only arrived after a subsequent send. I don’t have a good answer for why that was necessary but given the complexities of the d2xx library, reading from USB in general, the FTDI and its buffers, and the Arduino, it’s not too surprising that things can get out of sync if you’re moving fast.

With the implementation of the event character, the buffered writes added to the loop, and the timeout after writing, we appear to be running smoothly. So smoothly, in fact, that I was able to remove the setLatencyTimer call entirely and just leave it at its default. As configured, data is sent as soon as a 0 is hit or 256 bytes are available, whichever comes first. (Typing this out, I realize that I should probably just set it to the exact size of our largest message, there’s no way it could ever be smaller and having an incomplete message does us no good!)

To summarize, we went through two rounds of improvements that changed our situation from bleak to beautiful.

Round 1:

  • Set the FTDI’s latency timer to 1ms
  • Limit our max read size to something small to prevent a “jerky” feel

Round 2:

  • Revert latency timer to default
  • Enable an event character keyed to our delimiter, a 0 byte – this is the key
  • Set a max read that’s a bit bigger than our typical messages to help us catch up if we ever have a huge backlog and want to get the queue down (again, I don’t know what this situation is)

As it happens, the d2xx library is the only one of the three that supports configuration of latency timer, event character, and flow control. One of the two open source libraries supports the latency timer, the other claims to support flow control, and neither supports the event character. Only the closed-source official FTDI library d2xx supports all three, so we’ll be sticking with that.

It appears that our use case of extremely high streaming rate combined with tiny messages at a very high baud is somewhat unique. If we had been sending larger messages at a slower rate, I don’t think we would have encountered this. Our 44 byte messages at 300hz were the problem.

I spent many lonely weeks fighting with this. Failure to resolve it would have been a major problem for the project. In the end, the solutions I found were new to the whole team, which included many people with much more experience than me when it came to FTDI chips, which should go to show you how esoteric some of these configuration parameters very well may be. This is my first project writing Kotlin, working on Android, or using FTDI devices at all, so I while I’m disappointed that it was such an unpleasant struggle to get it done, I am pleased to have it behind me. I sincerely hope this helps somebody avoid going through the same experience.

A React Dev's Preliminary Thoughts on Jetpack Compose

I recently, unexpectedly, found myself learning the basics of Android development so I could manage a large project. When it took longer than expected to find the right people to do the development, I wound up also applying some of this knowledge by porting over a very large amount of code from Python and TypeScript to a prototype app. The goal was to get a head start on implementing business logic, then it turned into business logic plus communication with hardware so we could verify some external systems, and finally it turned into that + some toy interfaces so we could easily demonstrate to incoming team members how the business logic could be applied to the view.

This was and still often is a painful experience, as working in an unfamiliar language, framework, and platform so often is. Luckily, while the framework (Android) and the platform (also Android) are often very foreign, Kotlin itself has been a dream. In many ways, it feels like the language I’ve always wanted: the friendliness of Ruby with a powerful type system that has Java’s specificity but feels closer to TypeScript in its expressiveness, with the performance of the JVM. Coroutines are amazing. Now that SharedFlow has been released, it also provides concurrency and message passing that’s as easy as Go’s goroutines and channels. Wonderful!

My classes and their tests moved from Python and TypeScript to Kotlin with absolutely no difficulties. There’s nothing like porting over thousands of lines of code and unit tests to make you feel more comfortable with a language. I did my best to modify it to look more like idiomatic Kotlin but I’m looking forward to having someone review it and taking me to school.

But this isn’t about Kotlin, it’s about Jetpack Compose. I first read about it some months ago and noticed then that… wow, it sure sounds like React for Android. Plenty of comments online said the same thing, or said that it was just Flutter for Android, or SwiftUI for Android, but plenty of people said those were just React for X. As a React developer, this made me happy, since I like the idea of working in other languages and platforms. At the time, I didn’t know if or when I’d apply it. And then…

…I wound up here. It became clear that the best way to prove that my business logic really worked, test our new Arduino-based robot control system, and start testing our new mechanical components would be to provide a simple interface that replicated the states of our legacy product. While I felt very comfortable with Kotlin and reasonably comfortable with Android navigation, fragments, and activities, I did not feel at all comfortable with the classic UI framework. It felt very jQuery to me, like going back in time to the bad old days of web development. Jetpack Compose, on the other hand, had just reached alpha (alpha 7 by the time I got to it) and reviews were positive. Since this was just a prototype interface, it seemed like the best option for slapping something together. The person or people who inherit the app could then decide whether to continue betting on Jetpack Compose or fall back to the battle tested, if cumbersome, classic.

I’ve spent a few hours kicking the tires of Jetpack Compose. Since I only get to be a total amateur for a little while, I thought it would be good to jot down some of my preliminary thoughts while they’re still fresh.

Tl;dr: I like it!

To get it right out of the way, I really like Jetpack Compose! I’d happily continue building with it. I’ve seen comments on Android forums from experienced developers who say that it is the future of Android development and I hope that is true. It feels just like React in so many ways. Almost all of my experience transferred immediately: useState becomes mutableState, props are still props, it has its own version of useEffect. The way you organize components (composables) is the same, so you can think about the view and its state almost exactly the same way you would think about it in React! This is awesome.

I was able to bang out my first prototype interface, a crude recreation of one of our most complex, dynamic interfaces, in a couple of hours. Today, I very quickly prototyped another interface – a simple control panel that spits out some dynamic values coming from sensors and provides controls to load additional information or adjust the rate of the streaming data – in about 45 minutes. All of the methods that interact with the business logic plug right in. I’m not even using their effects yet, just passing props down, and I feel productive.

Some stray thoughts on the positives:

  • The fact that it’s Kotlin all the way down is extremely refreshing. Working in React forces one to context switch constantly: first it’s TypeScript, then it’s TSX, then it’s CSS, oh well here it’s a Styled Component so it’s TS-in-CSS. Being able to just live and breath one language is wonderful. I’ve read about plenty of other languages or frameworks that allow this – Elm comes to mind – and my thought had always been “But I’m good at all these things, I’m productive, it’s what everyone does. Why change it?” Avoiding context switching is real!

  • I think I like the fact that the builtin layout composables are so rigidly defined. That is to say, I appreciate that instead of a div with display: flex; and flex-direction: column; it’s just Column. It’s almost like this is what happens when you design a layout system in 2020 instead of gradually bolting on behavior over many decades…

  • I haven’t used it yet but the fact that it has an official Animation library is exciting. This tutorial paints a great picture.

  • I really really really like being able to create MutableState inside of a ViewModel. Hell, I really like ViewModel in general. One of my biggest gripes about React right now is the way it can hide complexity in hooks, letting some component deep down in the depths of a view subscribe to some external event and hook into all kinds of crazy side-effects. Hooks also force you to think exclusively in terms of side-effects, like you’re building a huge Rube Goldberg machine. By comparison, MutableState in a ViewModel feels like the best of worlds. Data flow is still unidirectional – I’m still feeding props down through composables – but the option of having some of this logic originate in an isolated world dedicated to business logic makes things a lot more straightforward.

  • It’s nice that the body of a composable is just Kotlin. Not some Kotlin-markup hybrid where you can use expressions but not statements ala TSX/JSX – full-on Kotlin. Even though I don’t feel limited by TSX these days, I think this will cut down the learning curve for new devs significantly compared to what one finds with React. Being able to use a when expression or assign variables right in the body of the code is freeing. It’s also a double-edged sword that I’ll comment on later.

It’s not all roses, though. Some of my immediate concerns:

  • The in-IDE previews are slooooow. I find that I don’t preview things as much as I’d like to because every save causes a rebuild that takes forever. I’m sure this must be part of its alpha-ness, but it really stands out in contrast with how well the rest of it seems to work.

  • As much as I appreciate the explicit, rigidly defined composables (Column, Row, etc,…) and their unique styling parameters and modifiers, I worry that it’s going to take forever to really get fluent in making things look good. Say what you will about CSS but at least you can be reasonably sure that most of the properties you know for one element will work on another. Maybe that’s too generous, but at least you’ll know where to start. This isn’t as true with Compose. While I picked up positioning of composables and their contents pretty quickly, making things look the way they do in the wireframes is still a challenge because I’m constantly digging into them to see what they can do.

  • The aforementioned issue wouldn’t be as much of a problem if not for the lack of best practices and examples out there. Almost worse, a lot of what you can find right now refers to earlier versions and removed APIs. Luckily, I’m not trying to build something for production and the rest of it is familiar enough that I’m confident I can invest some time in just about every issue and still be ahead of schedule. But it’s worth considering if you’re not confident with the mental model required to build this kind of app.

  • It’s going to need to improve its application state management story. While Context works fine for most of us these days and Compose has the corresponding Ambient, there are still cases where the web needs Redux and it seems likely that the same will be true for Android. There is a ReduxKotlin project in the works, so maybe it will take off. Or maybe there’s a better option, something more appropriate for this ecosystem.

  • It’s also going to need a better solution for forms. My kingdom for React Hook Form for Jetpack Compose!

A last one that I want to expand upon beyond a quick bullet point:

I worry that the ability to mix composable function calls and Kotlin code could lead to readability and predictability challanges.

The TSX syntax places limitations on what I can do once I start describing the view. Flow control is limited, side-effects are verboten. It’s predictable, it’s clear. It can be noisey if someone abuses ternaries or boolean statements to provide flow control, but we can train ourselves to expect it.

Jetpack Compose enforces no such order. I can do anything, anywhere, at any time. This can be nice if used sparingly but it doesn’t take much to imagine it going off the rails. Picture a wall of view code where, for some reason, a variable is reassigned 200 lines deep. Suddenly, the predictable view is a little less predictable. Refactoring is just a little trickier. Composables become just a bit more complex and harder to predict. It will be important for the community to establish best practices around this!

So that’s where I stand right now. It’ll be interesting to review this in a few months and see how much of this was accurate. In the meantime, I am optimistic.

Three.js + TypeScript + MTL + OBJ + MTL + WTF

At work, we’re working on implementing a very cool 3D visualization for use throughout our app. I’m not prepared to demo it but I am very prepared to talk about its architecture, cause lemme tell you, it was quite a challenge. It took a significant amount of help from a friend who is a 3D artist, many three.js code snippets, careful reading of documentation and source code, some Stack Overflow, and some good luck before I was able to consider this a success. I hope that the details I provide will help someone else save some time.

First, some background. I maintain a React app built using TypeScript. I use three.js in a few places in our app, but to date, this has always been somewhat simple: we take positions in space, we draw some spheres, we add a mesh, and… that’s it, really. In one view, we stream position data in from our hardware and I move a sphere accordingly; in another, we just present captured data and it’s relatively static. This was challenging when I was completely new to three.js but now that I’ve spent some time with it, I now feel like it’s pretty straightforward.

Enter the retail 3D model. We obtained resources in Maya format and needed to get it into the browser. Additionally, it needed to render quickly, since we target a somewhat under-powered device and often have multiple instances of the visualization in a given window, and it needed to be somewhat small, since some of our users are in isolated areas where they must rely on satelite internet. With every step, I encountered new challenges. It was like a boss rush in a video game. Below, I will walk through some of the key portions of this process and provide you with a few different ways of completing this task.

Before continuing, I’ll say that this is not going to be an exhaustive dive into how everything works. I’ll provide some detail but it will help for you to have a solid understanding of creating a basic three.js scene. You know about objects and the scene, the camera, lights, the renderer, meshes, geometries, objects, etc,… You also understand the basics of OOP, since three.js relies heavily on this, and you understand why we require and import content. You’re using webpack, casuse you’re gonna need it for some of this. You’re probably using TypeScript but you can easily ignore some of that if you need to and it’ll still work, it’ll just require you to keep more in your head.

Finally, I want to say that I’m not an expert on this and it is likely that better ways of solving this problem exist. Please don’t be mad at me if something here sucks. All code samples were adapted from my production code, so if something seems wrong or broken, it is probably due to my hasty copy/paste/refactor to omit pieces that won’t matter to you.

Ready? Let’s do it.

First attempt: Rigged model to OBJ and MTL

Maya, Blender, and I’m going to any other 3D modeling software worth using will support exporting to the OBJ and MTL formats, so we start with that. These are big text files and you can open them in your preferred text editor if you’d like. In my case, it was necessary to modify the MTL so it could find my materials’ images, which existed as .jpg files. Those modifications go beyond the scope of this post but please email if you need help, I can give you some tips.

Once exported, Three.js provides two classes, OBJLoader and MTLLoader, that make this somewhat simple, but there’s still some work to be done.

When dealing with all this stuff, I found that I was constantly trying to satisfy the needs of three different parties: the three.js library, the code as it relates to Webpack, and TypeScript. This was a recurring challenge that got easier as I went.

The completed code for my first attempt looked something like this:

// satisfy TypeScript
import {
  MTLLoader,
  OBJLoader,
  Scene
} from 'three';

const OBJ_NAME = 'viz.obj';
const MTL_NAME = 'viz.mtl';

// make webpack aware of the content so I can reference it in the code
[OBJ_NAME, MTL_NAME].forEach(filename => {
  require(`../../../public/3d_resources/${filename}`);
});

// satisfy the three.js libraries
require('imports-loader?THREE=three!three/examples/js/loaders/MTLLoader.js');
require('imports-loader?THREE=three!three/examples/js/loaders/OBJLoader.js');

// in your `constructor` or in some instance method called

class VizWrapper {
  scene: Scene;

  constructor() {
    // renderer, camera, scene are all created -- code omitted
    // finally...

    new MTLLoader().load(`/static/media/${MTL_NAME}`, creator => {
      creator.baseUrl = ('/static/media/');
      creator.preload();

      const objLoader = new OBJLoader();
      objLoader.setPath('/static/media/');
      objLoader.setMaterials(creator);
      objLoader.load(OBJ_NAME, obj => {
        this.scene.add(obj);

        // then start the render loop using your custom `animate` method
        this.animate();
      });
    });
  }
}


A few things of note here:

  • We import at the beginning to keep TypeScript happy. Strangely, this will bring the types into scope but it doesn’t actually load the libraries, I guess because something something three.js?
  • We already modified our webpack config so file-loader knows what to do with obj and mtl files. This ensures that explicit require statements will put them in /static/media, which we need for our code.
  • Three.js libraries have the obnoxious quality of expecting a global THREE variable. We can use imports-loader as you see above to feed it THREE, which satisfies their dependencies and brings the contents of those libraries into scope so we can actually use the classes in our code.
  • I don’t hang onto my MTLLoader or OBJLoader, I let them get garbage collected.
  • The exported OBJ file will have names that correspond to materials in the MTL, and the MTL will have details about how to find files on disk.
  • My initial export from Maya gave me MTL files that didn’t work correctly in Three.js. I had to modify them, email me if you need help.

This will work but it’s SLOW. We’re doing this whole process every time the visualization loads. Maybe that’s ok for you, maybe you have a single view and it’s ok for it to take a moment, or maybe your files are small, but for me, it was a problem.

An additional issue I had was that my rendered model was hideous. There were no curved edges, which I learned is called a “faceted model.” Think Virtua Fighter if you’re in your 30s and grew up with that generation of arcade games. We’ll discuss this later, but tuck it away for now.

PROS of this method:

  • It’s simple to implement.
  • You can find plenty of examples of this out in the world.

CONS of this method:

  • It’s slow and it is a blocking operation. Your whole browser will hang.
  • OBJ files are BIG, possibly too big for you.
  • Imported files are ugly.

Those cons were totally unacceptable for me, so I kept going.

Second attempt: Rigged model to OBJ and MTL with OBJLoader2

In addition to OBJLoader, three.js provides a different loader, OBJLoader2. Its developer and maintainer works from a separate repo here, where he explains many aspects of his loader. In addition to supporting more of the OBJ spec, it also uses Web Workers to improve performance, the first of the three issues identified below. Unfortunately, I found fewer examples of how to make this work. My first draft of this looked something like this:

import {
  Group,
  MaterialCreator,
  OBJLoader2
} from 'three';

const OBJ_NAME = 'viz.obj';
const MTL_NAME = 'viz.mtl';

[OBJ_NAME, MTL_NAME].forEach(filename => {
  require(`../../../public/3d_resources/${filename}`);
});

// Note the additional dependency required by OBJLoader2
require('imports-loader?THREE=three!three/examples/js/loaders/LoaderSupport.js');
require('imports-loader?THREE=three!three/examples/js/loaders/MTLLoader.js');
require('imports-loader?THREE=three!three/examples/js/loaders/OBJLoader2.js');

class VizWrapper {
  constructor() {
    // once again, initialize scene, renderer, camera, etc
    // finally...

    const objLoader2 = new OBJLoader2();
    objLoader2.setPath('/static/media/');

    const onLoadObj = (event: any) => {
      const group = event.detail.loaderRootNode as Group;
      this.scene.add(group);

      // start the animation with your custom method
      this.animate();
    };

    const onLoadMtl = (loadedMaterials: MaterialCreator) => {
      objLoader2.setModelName('my-viz');
      objLoader2.setMaterials(loadedMaterials);
      objLoader2.load(OBJ_NAME, onLoadObj, null, null, null, false);
    };

    objLoader2.loadMtl(`/static/media/${MTL_NAME}`, null, onLoadMtl);
  }
}

Some notes:

  • OBJLoader2 is required instead of OBJLoader.
  • We need an additional dependency, LoaderSupport, before we require OBJLoader2
  • OBJLoader2 relies on a series of callback functions that need to be defined ahead of time. I find this harder to reason about, I don’t know why.
  • OBJLoader2 uses MTLLoader under the hood. Looking at this code, I’m wondering if I need to require MTLLoader.js at all? Experiment with that.
  • The onLoadObj callback gets an instance of Event, an interface not defined anywhere. Looking at it in the console, I noticed that event.detail.loaderRootNode is a Group, the object we’re used to dealing with. we can just target that and proceed normally.
  • The onLoadMtl callback function is calling methods on objLoader2, which we need to define that at the start. It will link up objects to their materials for you.
  • We kick this process off by calling objLoader2.loadMtl. The overall process is essentially the same: load the materials, then load the geometries, then add it to the scene, but we wrote the code backwards. Cool… :-

PROS:

  • This is MUCH faster when it comes to actually getting everything loaded. Loading is no longer a blocking operation.
  • It uses the same resources we already created for the slower OBJLoader procedure, so it’s basically a free performance update.

CONS:

  • There’s fewer documentation and resources available for this. If something goes wrong, it’s harder to find help.
  • I personally found this harder to reason about and write. It felt backwards.
  • There aren’t types available for some of these objects, so I’m defeating type safety with those explicit as SomeCrap expressions. Unfortunate.

Outstanding:

  • Still haven’t solved our file size issue.
  • Still have flat, ugly surfaces.

Third attempt: Smoothed OBJ and MTL with OBJLoader2

Our files were importing faster but it was ugly as hell. My 3D consultant friend remarked that the problem was possibly the file type – we should consider FBX instead of OBJ – or it wasn’t being smoothed correctly at some point. She theorized that we could either export differently or pursue smoothing in three.js. A lot of googling and reading later, I discovered that this is a pretty well-recognized problem with an easy fix: compute vertex normals in your geometries.

This is one of the posts detailing the process. It describes a problem, which is that all of the Loader classes use BufferGeometry, which does not allow for the changes we need to make. The solution is to convert your buffer geometries back to less efficient Geometry objects, clean them up, and then turn them back into BufferGeometry. This was correct, but it was missing one critical step that I’ll highlight below.

Here’s what it looks like:

// using the same objLoader2 example, only changing the contents of `onLoadObj`

const onLoadObj = (event: any) => {
  const newObj = new Object3D();
  const group = event.detail.loaderRootNode as Group;
  group.traverse(mesh => {
    if (mesh instanceof Mesh) {
      const newGeometry = new Geometry().fromBufferGeometry(mesh.geometry as BufferGeometry);
      newGeometry.mergeVertices();
      newGeometry.computeVertexNormals();

      // THIS IS THE MISSING PIECE!
      (mesh.material as Material).flatShading = false;
      newObj.add(new Mesh(new BufferGeometry().fromGeometry(newGeometry)), (mesh.material as Material).clone())
    }
  });
  this.scene.add(newObj);
  // start the animation with your custom method
  this.animate();
};

Instead of just taking the group and throwing it in the scene, we break it apart, mergeVertices, computeVertexNormals, disable flatShading on the material, and build a new Mesh. We add this mesh to a new Object3D and then add that to the scene. We end up with a smooth, curved 3D rendering! Victory!

PROS:

  • It’s curved and beautiful! Success!

CONS:

  • Our files are still huge.
  • This is even slower cause now we’re doing all this extra work on every load.

Fourth attempt: Rigged model to FBX

We’ve got a nice looking model but it’s slow and huge. My friend had mentioned the FBX file as an alternative to OBJ and I was curious about whether that could solve some of these problems. Three.js has an FBXLoader, so we decided to give that a shot to see what would happen.

We went back to the beginning and tried exporting our rigged model from Maya as FBX. Unfortunately, this threw some errors and the geometries were totally screwed up when importing into Three. After a little research and experimentation, I found an alternative: we could import our working OBJ into Blender and export back out to FBX. The export worked without an error and we could start experimenting in Three.js.

At this point, I discovered a great benefit of using FBX: its modern version is a binary format. Our 16.2 MB file was now a 3.7 MB file, completely reasonable for our use as long as it didn’t involve a huge drop in quality. Three.js didn’t support the binary format until recently, but we found evidence that it should work now.

After a significant amount of trial and error, I got this working. I’m going to demonstrate it first without materials and then I’ll explain how to do it with them.

import {
  FBXLoader,
  Scene
} from 'three';

(window as any).Zlib = require('zlibjs/bin/zlib.min.js').Zlib;
// tslint:disable-next-line:no-implicit-dependencies
require('imports-loader?THREE=three!three/examples/js/loaders/LoaderSupport.js');
// tslint:disable-next-line:no-implicit-dependencies
require('imports-loader?THREE=three!three/examples/js/loaders/FBXLoader.js');

const FBX_NAME = 'viz.fbx';
[FBX_NAME].forEach(filename => {
  require(`../../../public/3d_resources/${filename}`);
});

class VizWrapper {
  scene!: Scene;

  constructor() {
    const fbxLoader = new FBXLoader();
    fbxLoader.load(`/static/media/${FBX_NAME}`, group => {
      this.scene.add(group);
    });
  }
}

Notes:

  • We have an entirely new dependency: zlib, aka zlibjs on npm. Here. Not to be confused with zlib on npm, which will not work. This is required by the FBXLoader. We need to require it and attach it to the window, which is awful. There’s probably a way to use imports-loader to fix this.
  • We’re still using LoaderSupport, FBXLoader will die without it. We don’t need to import this or Zlib since we don’t interact with them in our code.

There’s a huge problem with this: FBX files usually have materials embedded, but I must have missed a step in my OBJ + MTL import into Blender, so the export lacked them. FBX is much more powerful file format, supporting animations, cameras, and lighting – things we don’t want in this case. There’s no clear way to separate them out and neither the Three.js documentation nor the code nor Blender provide any clear path to resolving it and explicitly providing external materials. This is unusable as is.

I played around with a few approaches to fixing it. I got my hands on a rigged Blender model, from which I could export an FBX with materials embedded, but this hurt file size and included those aforementioned objects that I want to avoid. I just want geometries and external materials! I went back to the drawing board.

PROS:

  • We’ve got a tiny geometries file.

CONS:

  • It doesn’t put anything on the screen because we don’t know how to connect it to our materials.

Fifth attempt: Rigged model to FBX with manual MTL hookup

Looking at the FBX source and type definitions, I noticed that the argument provided to the callback (second argument) of fbxLoader.load was an instance of our familiar Group. As we’ve seen, a three.js Group is a container of Mesh objects, and each Mesh object will have a Material. Curious about what my broken imported FBX meshes would recognize as their materials if I didn’t hook them up, I looped through and sent each to the console.

fbxLoader.load(`/static/media/${FBX_NAME}`, group => {
  group.traverse(mesh => {
    if (mesh instanceof Mesh) {
      console.log(mesh.material);
    }
  })
});

This revealed that each Material had a name key and these names were familiar: they were the same names referenced in our .OBJ and .MTL files. It seemed reasonable that there would be some way to manually do what objLoader and objLoader2 did automatically.

Being ASCII files, the .OBJ and .MTL files are pretty easy to make sense of if you look for commonalities, especially where materials are concerned. The .MTL file identifies each material with a newmtl directive, a string name; on the other side, the .OBJ has a very clear usemtl directive that relies on this same name. When using objLoader and objLoader2, we had a simple way of wiring these up: loader.setMaterials(creator). We do not have such luxury with our FBX Loader, where it expects the materials to be embedded in the file, but it does set that same name on the FBX’s imported-but-broken Material.

Examining the code, I noticed that the object provided to the mtlLoader.load was an instance of MaterialCreator. The three.js documentation for this object was severly lacking but the TypeScript definitions were not, and there we found one promising instance method:

create( materialName: string ) : Material;

It does exactly what you’d expect: give it a name that it knows and it will spit out a material. Here’s the complete code:

import {
  MTLLoader,
  Mesh,
  Material,
  Geometry,
  BufferGeometry,
  FBXLoader,
  Scene,
  Object3D
} from 'three';

(window as any).Zlib = require('zlibjs/bin/zlib.min.js').Zlib;
// tslint:disable-next-line:no-implicit-dependencies
require('imports-loader?THREE=three!three/examples/js/loaders/LoaderSupport.js');
// tslint:disable-next-line:no-implicit-dependencies
require('imports-loader?THREE=three!three/examples/js/loaders/FBXLoader.js');

const FBX_NAME = 'viz.fbx';
const MTL_NAME = 'viz.mtl';
[FBX_NAME, MTL_NAME].forEach(filename => {
  require(`../../../public/3d_resources/${filename}`);
});

class VizWrapper {
  scene!: Scene;

  constructor() {
    new MTLLoader().load(`/static/media/${MTL_NAME}`, creator => {
      creator.baseUrl = ('/static/media/');
      creato.preload();
      const fbxLoader = new FBXLoader();
      fbxLoader.load(`/static/media/${FBX_NAME}`, group => {
        const newObj = new Object3D();
        group.traverse(mesh => {
          if (mesh instanceof Mesh) {
            const newGeometry = new Geometry().fromBufferGeometry(mesh.geometry as BufferGeometry);
            newGeometry.mergeVertices();
            newGeometry.computeVertexNormals();
            newGeometry.name = mesh.name;
            (mesh.material as Material).flatShading = false;

            const finalBufferGeom = new BufferGeometry().fromGeometry(newGeometry);
            // We fix materials here
            const finalMaterial = creator.create((mesh.material as Material).name))
            const finalMesh = new Mesh(finalBufferGeom, finalMaterial);
            newObj.add(finalMesh);
          }
        });
        this.scene.add(newObj);
      });
    });
  }
}

That does it! We’ve created new meshes using the geometries from our tiny FBX file with the materials from our MTL file. I bet we could find a way to skip the MTL file and instead just load the JPGs directly from disk, but that’s a problem for another time. We’re almost there!

PROS:

  • Small files!
  • Readable code
  • Looks great

CONS:

  • Performance still isn’t great. Our load is a blocking operation again and we’re doing the same work on each init of our visualization.

Sixth attempt: FBX + MTL with reusable components

We’re almost there! It looks good and it’s small enough to send over the wire but we’re still blocking the DOM to import files and create Geometries over and over again.

There’s an easy way to fix this: geometries and materials can be saved and reused, but meshes cannot. Instead of doing all this work in our constructor, let’s do the heavy stuff once when the file is required and then do the bare minimum work when the visualization is loaded.

We’ll define a new TypeScript interface, VizPiece. We’ll also create somewhere to put our viz pieces so they won’t get garbage collected. This eats up a little memory but it’s better for performance. Then we’ll define some functions to load everything and kick it off right away. After that, we can just loop through our vizPieces array in the constructor of our wrapper classes, building Mesh instances along the way. This is pretty cheap and we’ll notice a significant performance improvement as a result.

It could look like this:

// viz_preload.ts
// requires shared files, interfaces, and calls upon your loader

import {
  BufferGeometry,
  Material
} from 'three';

const files = ['texture_jpg1', 'texture_jpg2'];
files.forEach(name => {
  try {
    require(`../../../public/public_resources/${name}.jpg`);
  } catch (e) {
    console.log(`could not require ${name}Dif.jpg`);
  }
});

export interface Viziece {
  geometry: BufferGeometry;
  material: Material;
  name: string;
}

// populated by the `load()` function below
const vizPieces: VizPiece[] = [];

import load from './fbx_loader';
load(vizPieces);

// for use in three.js code
export default vizPieces;
// fbx_loader.ts
// your loader is defined here, the load function is the only thing exported and it takes the array to populate

import { VizPiece } from './viz_preload';
import {
  BufferGeometry,
  FBXLoader,
  Geometry,
  LoadingManager,
  Material,
  MaterialCreator,
  Mesh,
  MTLLoader
} from 'three';

(window as any).Zlib = require('zlibjs/bin/zlib.min.js').Zlib;
// tslint:disable-next-line:no-implicit-dependencies
require('imports-loader?THREE=three!three/examples/js/loaders/LoaderSupport.js');
// tslint:disable-next-line:no-implicit-dependencies
require('imports-loader?THREE=three!three/examples/js/loaders/FBXLoader.js');
// tslint:disable-next-line:no-implicit-dependencies
require('imports-loader?THREE=three!three/examples/js/loaders/MTLLoader.js');

const FBX_NAME = 'viz.fbx';
const MTL_NAME = 'viz.mtl';
[FBX_NAME, MTL_NAME].forEach(filename => {
  require(`../../../3d_resources/${filename}`);
});

const loadingManager = new LoadingManager();

const loadGeometry = (loadedMaterials: MaterialCreator, vizPieces: VizPiece[]) => {
  const fbxLoader = new FBXLoader(loadingManager);
  fbxLoader.load(`/static/media/${FBX_NAME}`, group => {
    group.traverse(mesh => {
      if (mesh instanceof Mesh) {
        const newGeometry = new Geometry().fromBufferGeometry(mesh.geometry as BufferGeometry);
        newGeometry.mergeVertices();
        newGeometry.computeVertexNormals();
        newGeometry.name = mesh.name;
        (mesh.material as Material).flatShading = false;

        vizPieces.push({
          geometry: new BufferGeometry().fromGeometry(newGeometry),
          material: loadedMaterials.create((mesh.material as Material).name),
          name: mesh.name
        });
      }
    });
  });
};

const load = (vizPieces: VizPiece[]) => {
  new MTLLoader(loadingManager).load(`/static/media/${MTL_NAME}`, creator => {
    creator.baseUrl = ('/static/media/');
    creator.preload();
    loadGeometry(creator, vizPieces);
  });
};

export default load;
//VizWrapper.ts

import vizPieces from './viz_preload';

import {
  Mesh,
  Object3D,
  Scene,
} from 'three';

export class VizWrapper {
  scene: Scene;

  constructor() {
    // create your scene, renderer, etc...

    if (vizPieces.length > 0) {
      const newObj = new Object3D();
      vizPieces.forEach(vizPiece => {
        const newMesh = new Mesh(vizPiece.geometry, vizPiece.material);
        newMesh.name = vizPiece.name;
        newObj.add(newMesh);
      });
      this.scene.add(newObj);
      // your custom load method
      this.animate();
    } else {
      // handle this somehow
      // throw an error, load it manually, ignore it?
    }
  }
}

So to recap, our new process is like this:

  • We create an array to hold the component pieces of our visualization.
  • We populate that when the app loads, incuring a small performance penalty once at the beginning.
  • We loop through that array, combining those pieces to create usable objects, when the visualization needs to be displayed.

PROS:

  • On-demand renders of our visualization are fast and it looks good.
  • File size is small.

CONS:

  • We incur an up-front performance penalty for everyone, regardless of whether they are going to see the visualization or not.

That one might be a deal-breaker for you but maybe not. If it is, maybe you use the Web Worker-powered ObjLoader2. A pleasant option with this implementation is the flexibility to define multiple loaders, each exposing a load function that takes the same parameters, allowing us to swap everything out by changing the file from which we import. In my code, while I test, I defined one for each of the processes outlined above. It might also be reasonable to pursue an FbxLoader that uses Web Workers, but that’s something for another day.

So there you have it! This represents many days worth of troubleshooting, so I really hope it can be of use to someone else. If it is helpful, please shoot me an email and let me know, I’d love to hear what you think.

subscribe via RSS