The Open Source Ratchet

I have often observed that the race between corporate code and code written by hobbyists and volunteers is like the race between the Tortoise and the Hare. The Hare is fast, but inconsistent in its speed. The tortoise is slow, but never stops.

Similarly, companies have the time and resources to pour into making code really good, really fast. However, having such a "high metabolism", they must turn a profit with that code. Businesses fail, and priorities change within a business at lightning speed. An example of this is Timescale dropping Promscale. They did this presumably because they found it to be an unprofitable venture. There is nothing wrong with this; but it is a perfect example of this Hare-like behavior: they spent lots of time and resources quickly and thoroughly building a great piece of software, only to let it rot because of a change in priorities. (I must admit some personal angst here, but I also wish them well. I understand this is how it works.)

Conversely, hobbyists and volunteers are slow, since they are limited to their free time. Ironically, though, this allows them to have all the time in the world. After all, their code need not turn a profit. As long as the code sparks joy for someone, it will remain worked on. Old, old projects like irssi and Pidgin continue to be worked on to great effect, because someone likes them. The result of this tortoise-and-hare dichotomy is that a common story plays out whenever innovations are made in the technology space.

There are three phases in this race.

First, a new idea finds its way into the minds of programmers, often introduced or popularized by a corporation. Spreadsheets by Microsoft Excel, Online collaborative document editing by Google Docs. Centralized, logged, searchable chatting was introduced by Slack. And on and on. The corporations race to develop and perfect the idea they have created (or, in some cases, copied from others), and even perfect it. The successful ones establish themselves as dominant in the space. This is phase I of the innovation cycle.

In phase II, stabilization of the idea occurs. The hare, having raced ahead, now lags. Corporations fail and stop working on their code as the more dominant players vie for more and more market. As they approach a monopoly, they slow down and start to become complacent in their productization and innovation efforts. There is no need for the effort; they are ahead. Not only that, but the needs of the market have stabilized the technology in question. The market as a whole gains a complete picture of what they want.

My favorite phase, phase III, occurs when the idea becomes commoditized. After phase II, the technology has stabilized and therefore has ceased to be a moving target. The hobbyists and volunteers, though limited in their time, are not limited in their ardor. This enables them to implement the technology themselves. The tortoise overtakes the hare. This eats into the market share of the coporation, but they are not usually to phased by this; Often, they will market themselves as "the choice of professionals" and those professionals will continue to buy the corporate product. Meanwhile, the rest of us can continue to enjoy the tech without having to deal with reliance on the corporation.

This has happened over and over and over. I present here a small list of technologies as examples of it:

I've heard this effect sometimes called the "Open Source Ratchet". The idea is that open source can progress, but never regress. This is because hobbyists can always take it up and keep it going.

In my own personal life, I think of the Pidgin GroupMe plugin. Indeed, lots and lots of Pidgin plugins fit this example. For a while, all the corps adopted open standards like XMPP. Then they advanced far ahead, adopting Slack-like innovations and creating their own proprietary protocols. Pidgin could not keep up.

But those protocols have since stabilized, allowing time for hobbyists to catch up to them. Pidgin now has plugins for MS Teams, Discord, Slack, and of course, GroupMe, the plugin which I now maintain.

A year ago I got a linux laptop at work. There wasn't really a good groupme chat client for Linux, and I didn't want to use the browser. I found this code by Alyssa Rosenzweig of Asahi Linux fame. It was a GroupMe plugin for Pidgin. It was broken and only sorta worked, but the bones were good. It was enough that I was intrigued. After some initial work, I got it to work "well enough" for my needs. Then someone asked if there was a Windows compiled version of it. It took some work and doing, but I got that working, too, and was also able to smooth some sharp edges of the plugin that was bothering me over a year of use. Now, it's very useable. I can even use it on Windows! It took time, but I have time. And now, it's done.

Open source, by definition, is available to anyone in the world to be worked on. This is what makes it so hard to kill. It can go forward, but never backward. If we participate in its movement, we can be proud of our accomplishments, and pleased that the code we write for ourselves may last much longer than the code we write for our bosses. And the world will slowly become a better place, too.