The Red Hat Drama Is Highlighting Open Source Software's Growing Pains

One of the most complicated debates in recent computing history is picking up steam in the open-source world. And it points to a complex state of affairs around licensing, business, and protecting an important ecosystem for open-source software.

At the center of that debate? The highest-profile commercial player in the open-source space, Red Hat. The IBM-owned company, which contributes heavily to the open-source community, has gotten cold feet around downstream or derivative uses of its flagship Linux distribution, Red Hat Enterprise Linux (RHEL), which requires a commercial support license to use in many cases.

In the past, Red Hat had offered a free-to-use downstream distribution, CentOS, but in 2020, it chose to replace it with an upstream distribution, CentOS Stream—and importantly, ended formal support for the CentOS distribution years early. The benefit from an open-source standpoint was that CentOS worked the same as RHEL, down to the bugs—but when that was lost, new third-party distributions such as Rocky Linux and Alma Linux took CentOS’ place.

Now, Red Hat is taking steps to rein those third parties in. On one level, what Red Hat did was to remove code from RHEL from one specific publicly accessible server, pushing users to register to access the code or access the upstream CentOS Stream code instead. But on another, it was received by some as a betrayal of open-source principles, particularly the General Public License (GPL).

The news, first announced in a blog post last month with the unassuming title of “Furthering the evolution of CentOS Stream,” was not received well by the broader Linux community—a group of small businesses, hobbyists, and developers who have bought into the free-as-in-speech ideals of the open-source ecosystem. Developers in particular are not the target audience for RHEL, but often need to develop for it. Mike McGrath, the vice president of Core Platforms for the company, said in an interview that the company has taken steps to make RHEL easier for noncommercial users to access.

“Where it used to be very difficult for anybody to get free RHEL, it’s actually pretty easy now to come and get a developer SKU and all this stuff,” McGrath says in an interview. “And we’ve been working very hard behind the scenes to make sure that RHEL is as accessible to as many people as possible.”

He admitted, though, that the move was in direct response to “downstream rebuilders” who were selling support contracts, which he said was going beyond the letter of the General Public License, which requires open-source code to be distributed to end users. (Notably, NASA acquired a support contract for Rocky Linux last month from CIQ, a company separate from the Rocky Linux project that specializes in support contracts.)

“They’re not just exercising their GPL rights, they’re also competing with us, and we detected one of them just doing some shenanigans,” he says.

For its part, Rocky Linux has defended its approach, saying that Red Hat’s Terms of Service and End User License agreements have elements that run counter to the spirit of the GPL. In a blog post last week, it said that it had determined methods to continue offering a downstream distribution that go around Red Hat’s policy limitations—most notably, by utilizing widely available hosted cloud images to automate the detection of changes between RHEL versions.

“Our legal advisors have reassured us that we have the right to obtain the source to any binaries we receive, ensuring that we can continue advancing Rocky Linux in line with our original intentions,” the company wrote.

Red Hat’s stances aren’t sitting well with some of the community’s most high-profile developers, such as Jeff Geerling, a YouTube host known for his Raspberry Pi builds and for being an expert at Ansible, an automation platform primarily maintained by Red Hat.

Geerling has had a formal contractor relationship in the past with Red Hat, whose software he has used off and on since 1999, but he has found himself on the other side of the chasm from the company in this debate, publicly pledging to no longer support RHEL users that rely on his open-source tools. In an interview, he described the situation as one that appears to have gotten out of hand.

“It’s kind of like they threw a nuclear bomb into the Linux community,” Geerling said. “And I don’t think that they knew it was nuclear. I think they thought it was like a hand grenade.”

The decision to change CentOS in 2020, he says, was already a significant blow, as that had brought him back to the Linux ecosystem, and the recent news has even raised concerns about his work with Ansible—though he still plans to work with that tool at this time.

“In my experience with Red Hat, CentOS was core to that whole thing, and they took it away—and they took it away again, after they promised they wouldn’t take away the git sources,” Geerling said.

In response to stances like Geerling’s, McGrath wrote a follow-up post that further made clear the company’s stance regarding downstream distributions, with one line in particular catching the attention of users: “Simply rebuilding code, without adding value or changing it in any way, represents a real threat to open source companies everywhere. This is a real threat to open source, and one that has the potential to revert open source back into a hobbyist- and hackers-only activity.”

When asked to articulate his point, he explained that the one-to-one nature of CentOS that Rocky Linux and AlmaLinux are replicating could have a long-term damaging effect to companies that rely on these tools.

“They’re not sending code upstream like their very mission is simply to be identical to us,” he says. “So even if we have a bug like there, they want to also have the bug and I think that that is that’s something that I think has always bothered us at Red Hat. And we never really figured out a way to articulate it.”

Not everyone has been won over by this argument—for example, Apache Software Foundation developer Graham Leggett, noted in a tweet that Red Hat often uses his work downstream with no concerns about compensation. Jonathan Wright, the infrastructure team lead for AlmaLinux, wrote that he was able to deepen his work in the open source community, including on Red Hat’s Fedora, through AlmaLinux.

And Geerling has suggested that this may point to some weaknesses with the GPL that are either getting overlooked or need to be changed.

Finally, some critics have pointed out that Rocky Linux and AlmaLinux downstream builds exist on platforms that Red Hat is not servicing with RHEL, such as the Raspberry Pi—potentially exposing future users to the benefits of RHEL.

McGrath, while stating he understood these stances, defended the company’s point of view on RHEL and downstream users, while saying what the company was doing did not suggest a broader licensing rethink was necessary.

“If you take those rebuilds to their natural conclusions—and that applies to any open source company—they are a siphoning action on Red Hat, and therefore, also on the community,” he says.

In many ways, the Red Hat conflict points to a broader ethical debate that is beginning to play out in the community. 

Linux, explicitly built as a community project, has evolved into something that is often both maintained and widely used by large companies. Canonical, for example, maintains Ubuntu, which is used as the basis of literally dozens of mainstream Linux distros, such as the popular desktop distro Pop!_OS, which is itself maintained by the PC manufacturer System76. Red Hat also maintains Fedora, a mainstream Linux distro that runs upstream of RHEL.

Over time, concerns are rising that these commercial interests may not run astray of community needs. In one case, a prominent Linux educator, Jay LaCroix of LearnLinux.tv, announced he was moving away from Ubuntu for his server installs, in favor of the community-developed Debian. While he says Canonical has not proven as bad as Red Hat, he did point to similar concerns that could threaten the Linux community long-term.

“When it comes to me, I can’t use something in good conscience that’s made by a company that actively works against the community I serve,” he said.

Geerling pointed to a broader philosophical discussion about what open source is meant to support, and how it gets used in commercial contexts. In the case of Red Hat, in his view, he says this leads to a pendulum effect between the needs of its engineering and community teams, and the more commercial business needs.

“When I was contracting there, it seemed like the pendulum was swinging that way and at this point, it feels like that pendulum is purely in the business side,” he said. “And engineering and community needs are basically taking a backseat because of the current economy.”

For his part, McGrath noted that, while it understands broader community concerns and the need to ensure developers are supported, the company often must navigate multiple differing commercial use cases beyond community needs—from research institutions to Fortune 500 businesses. Even in the case of research institutions, RHEL can find itself used by scientists, but also managing the technology of a Big Ten football team.

“Something that we’ll continue to work on going forward is making sure that we’re being fair with pricing if we have pricing at all—there’s plenty of free stuff still,” he says. “But we also don’t feel obligated to try to cater to every possible Linux user. There are tons of choices out there. People are welcome to choose what’s best for them.”

There is risk of a slippery slope effect, that Red Hat’s strategy, if successful, might find support from other open-source companies who decide to restrict downstream uses. Geerling admits that this risk of corporate open source becoming at odds with end users was part of the reason he has become so passionate in this debate—even if it costs him some burned bridges.

“I don't want this to be the inflection point,” Geerling says.