Respin: upspin revival
tl;dr: I revived the upspin project source code. See it at https://github.com/filmil/upspin, and https://github.com/filmil/upspin-gdrive. Read on to learn what that actually means.
History
Upspin was a project intended to provide a global namespace for all digital artifacts. It ended up mostly being used as a distributed file store, although the idea was more general than that. It was quite useful even as storage. And as far as distributed filesystems go, it was by far the simplest portable way to share storage between different machines.
Unfortunately, for all its utility, it saw relatively little uptake, which led the authors to announce infrastructure turndown in February 2025. The main piece of infrastructure that got turned down was the Upspin keyserver, which used to be hosted at key.upspin.io, and which used to store public keys of all the upspin users. This data store allowed Upspin encryption to store data blobs encrypted at rest.
There was a tacit assumption in all of the Upspin system that this keyserver be always accessible. The turndown of the keyserver, therefore, meant that the upspin systems everywhere stopped working.
This nicely illustrated the distributed computing adage attributed to Leslie Lamport:
A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable.
The other thing that stopped working was a way to get up to date binaries.
It’s gone, what to do?
Since I already had a somewhat involved data setup that used upspin backed by Google Drive, I was also interested in finding ways to continue using it. It turns out, Upspin was the most convenient way to share data among my computers, and I was interested in continuing to use it. There were no satisfactory replacements: the official Google Drive client doesn’t work on Linux, unofficial ones are slow and unreliable, Dropbox et al are also spotty, and “eternal” stores like Perkeep seem to have a huge initial setup cost.
In theory, it would be straightforward to deploy a simple federated keyserver, if you modify the source code to look up a key for user foo@example.com at a dedicated keyserver key.example.com. In practice, the assumption was baked somewhat deeply into the upspin libraries, meaning that somewhat deep changes were necessary, further meaning that somewhat deep and detailed knowledge of the code base was required to make it happen.
This is all well and good, but takes time. Recently with the advent of AI assistants, orienting in the code base became much easier, which suggested to me that maybe the time was ripe to try and revive the project, such that existing data storage could be reused.
Here’s what I did
This is exactly what I did: I revived the project to the point where an existing upspin user can deploy the new binaries and should have access to their existing data store. It is possible to adapt this such that you can get full functionality, so long as you are able to run your own keyserver. But this isn’t something I needed, since the use case where I actively share data with others never really materialized. However, it might be something you’d be inclined to do, I assume that if you are the kind of person who bothers with Upspin, you might be also inclined to take up the work yourself.
What I did instead was as described below. I am almost sure that the original upspin authors would find some choice words1 to say about my changes to the original project code and infra. I certainly didn’t ask for anyone’s opinion before ding this. However, it was something that I could easily do, and could do well, and the outcome is something I can maintain.
-
Moved build infra to Github. Original authors preferred a workflow that relied more on the traditional mailing lists and patches. I’m soft and weak, and I like Github.
-
Moved the build system to bazel. This makes maintenance easier for me, as maintaining the code means more than just building go binaries. The main reason is that, since I’m just one person (in contrast to 53 contributors to the original project), I have limited ability to build out the workflows needed to maintain a somewhat complex project like this. Bazel allows me to reuse bazel build rules to do things like build binaries, build containers, and build deployment recipes for cloud VMs. All of this is something one can do while maintaining go build purity. But, I don’t work on this for pay, and I have a day job so I’m motivated to cut the toil short as much as I can. To all who are ticked off by this: I’m sorry. I wrote “why I bother with bazel” to explain a bit.
-
Added scaffolding for the Gemini AI assistant. Some upspin authors have been famously vocal against AI. But, I’m one person, my time is limited, and AI coding assistant allows me to go further, and faster. With this in place, I can offload some tasks to Gemini CLI2, or Jules on the Web, and I used that plentifully.
-
Implemented (with Gemini’s help) a “local” keyserver, which is a read-only keyserver that you can host on your local infrastructure. I had some nascent attempts to do this before, but it turns out I didn’t catch all the spots where
key.upspin.iowas hardcoded, making for a very confusing deployment experience. -
Modified the upspin libraries to use the local key server, such that
user@domain.comgets looked up atkey.domain.com, instead of the now-defunctkey.upspin.io. -
Modified the Google Drive Storeserver OAuth process, which used the now-defunct OOB OAuth flow so that you can initialize the server again.
-
Stood up the Github workflows to build binary releases and containers on the regular, and automatically run required tests on any changes to the code.
Here’s what I didn’t do
Please bear in mind that the code revival is very simplistic.
It provides stable fundamentals for the long-term availability of the binaries and infra. It does not provide user friendliness. If you have use cases and good will to change that, send patches.
-
I didn’t create a generalized keyserver. The local keyserver is a read-only keyserver, which can only serve keys you already have on hand. This means you can’t do
upspin setupserverand have this work automatically. I didn’t do that because I already have my keys and have no reason to. -
I also did not bother to ensure that the keyserver access is verified. The traffic is (should be?) served via HTTPS with current certs, and all other cryptographic properties have been simply reused from what was already there.
-
Nothing in this process makes the upspin data storage guarantees any better than what they have been before. Upspin is and was alpha quality, and you should not store on it anything you can’t afford to lose. In the process of making the adaptations, I destroyed all my upspin instance data twice. I still don’t know exactly why.
-
The Google Drive Store server uses a restricted client ID for OAuth, which will not allow anyone else to deploy it without recompiling. It isn’t hard to adapt so that it works in the general case, but I had no reason to do that for a user base of one.
Conclusion
The main takeaway is that upspin can now work again for you, but—for now—3only if you are an existing former user and you have all your keys on hand.
After a prolonged session of adapting the existing code in a fork, I now have something that appears to work on my machine. All unit tests pass, excluding end-to-end tests which relied on a non-public infrastructure, and including tests added to confirm the keyserver redirection.
The main issue, in case you see this as a chance to get something useful for yourself, is that I haven’t actually made this usable by others. If you want to make it happen, you should fix the remaining issues. Send patches.
In the end, please remember that my contribution here is extremely minimal: I automated something that basically anyone could have, and I heavily used AI-assisted coding to orient myself in a codebase which continuously escaped my efforts to understand it. I probably ruined some of the esthetics of the original project in the process. For that, I apologize.
-
Some of them went on an adventure to design and build out an entire new programming language and ecosystem, after dealing with the complexities of Google’s build system. ↩︎
-
Why Gemini and not say Claude? Well, Gemini somehow seems to fare much better with Bazel than Claude does. I learned to wrangle Gemini well, and it mostly does what I tell it to. Unlike Claude, which I do not understand (yet?). ↩︎
-
(See how I used the em-dash there? :) This text was written in full by a human, though.) ↩︎