Publishing more often by iterating more frequently (a toolchain discussion)

[semioticrobotic]

I'd like to initiate a discussion about ways we might release new versions of the guide more frequently by iterating more rapidly.

I was lucky enough to share some of these thoughts with @quaid, @bproffitt, and @shaunix at the All Things Open conference recently; now I want marshal my thoughts on the matter by writing something here.

At the moment, our previous release is more than three years old. And while we've pushed useful and important changes to the repository since then, we haven't brought all these together in a new version. I still use (reference, cite, add to, etc.) The Open Source Way handbook frequently, and I'd like to see the resource continue to grow and thrive with new versions and activity that inspires others to jump in and contribute, too.

My assessment of the situation (accuracy yours to determine!) is that our publishing process is a bit overwrought. We've got a nice, streamlined editorial process, but getting materials from "polished, ready to go!" to "live, now part of the guide!" seems to be a sticking point. I wonder if our AsciiDoc-based toolchain might be too complicated, getting in our way more than propelling us forward.

I say this because I've recently been experimenting (productively) with new Git-based, Markdown-to-website generators that have made publishing prose from GitHub repositories like ours incredibly straightforward. The state of the art in this space has advanced quite a bit since we initiated our project, and some of these tools are much more useful for projects like ours than perhaps they were when we got started. GitBook (a SaaS solution available at no cost of open source projects) and Docusaurus (a self-hosted option) come to mind, as they're the two with which I've become most familiar.

Tools like these really streamline the work of publishing Markdown-based documents as websites, and I think adopting something like this could accelerate our own practice here. They take the "legwork" out of publishing updates to handbook-style resources, reflecting changes to a codebase almost immediately after those changes land in the project's main branch. I recognize that adopting a tool like this would require a bit of refactoring in our codebase (AsciiDoc and Markdown are similar but not identical!), but I want to suggest that this hill may be worth the climb—especially if it means releasing more than once every half-decade.

What do others think?

[semioticrobotic]

It strikes me, too, that in the event that adopting a new publishing toolchain seems undesirable/intimidating for the community, we might at least consider automating our current publication process (which is already script-based) with GitHub Actions to remove the need for extensive human intervention every time a new release is warranted.

[bproffitt]

I think this is a fantastic idea, and I am glad you have stepped up to initiate this. I will commit myself and other Red Hat resources to helping with a switch to the MD format, however we can be of assistance.

[semioticrobotic]

Thanks for this affirmation, @bproffitt. I'm away for another few days but perhaps when I'm back in the saddle we could schedule an informal call to discuss in more detail.

Update: Meeting discussion floated.

[semioticrobotic]

Thanks for a productive meeting today, @bproffitt, @shaunix, and @quaid. It was great to see you all again.

At the meeting, we decided to move ahead with a plan to refactor the Open Source Way guidebook and port it to Markdown for use with GitBook. To that end, we outlined the following next steps:

• Clean up guidebook repository by moving instructional files to the-project (see 7bdd423 and theopensourceway/the-project@268972c)
• Create new working branch of guidebook project (see working branch 2025-refactor)
• Create GitBook account and add maintainers
• Connect GitBook organization to project repository and working branch
• Begin refactoring the guidebook by converting files from AsciiDoc to Markdown
• Seek GitBook sponsorship as a community-driven open source project

I volunteered to get us started so I will just start at the top of the list and get rolling.

[semioticrobotic]

Well, it took a little time, but I finally have a working prototype ready for review:

https://the-open-source-way.gitbook.io/guidebook

@theopensourceway/maintainers, please take a look and see what you think!

At the moment, the GitBook site is tracking a working branch of the GitHub project, 2025-refactor. If and when we decide to move forward following review, we can merge that branch, set the GitBook to track main, and update URLs as necessary.

I have plenty of thoughts/notes on the project in light of recently touching each chapter, but I'll hold those for a moment until we've all had a chance to review and comment on the work.

Happy Thanksgiving to those celebrating! 🦃

[semioticrobotic]

Oh, and one other thing.

The main branch features several stubs of chapters-in-progress. I've moved these to a temporary location (the-project/editorial/chapter-drafts) for safe keeping while we hack around with the refactor.

Ultimately, what I would like to do is open a new working branch for each of these chapters, so authors can work simultaneously on bringing them to life and we can merge into main (and publish) when each is finished. But I would prefer to do this after we've first merged any changes to the main branch, as it may eventually have a new folder structure.

I understand that this represents something of a departure from our previous way of working, but I will eventually be proposing changes to that process, too, now that we're entertaining use of a different publication platform.

[bproffitt]

Okay, I have been away on PTO (some of us know how to do that, BB ;) ), and will review this this week.

[semioticrobotic]

Plenty of Time for Open Source. I know how to do that. Thanks for reviewing!

[bproffitt]

Got into this and I have to say this is pretty great. The format of the final product is solid, and I love the navigation and display elements. I found the source code in the 025-refactor branch and the md files look pretty clean and consistent. A couple of questions:

• Were there any conversion-related issues/cleanup concerns?
• Do we care that the "Powered by Gitbook" stamp is all over the site? Part of me is like "cool, support our fellow open source projects" and part of me is also "it feels like a NASCAR sticker."

[semioticrobotic]

So glad you're liking it, @bproffitt! That gives me confidence. Some answers for you:

Were there any conversion-related issues/cleanup concerns?

Conversion wasn't perfect, but involved pretty much the level of effort I expected it to. In the end, I had to touch each file manually to fix fiddly things, and you'll likely see some artifacts from that (I'm sure I didn't catch everything in a single pass). The issues that stand out to me are:

• GitBook likes _ characters for designating italics in Markdown. When publishing through the GUI, it will replace all * characters used for this purpose with _ (example commit). Not really serious; just something we should note.
• The conversion undid our one-line-per-sentence formatting convention for source files. Each paragraph is now a line. I started correcting that at first, but then gave up—it would just have taken too long, and that's the kind of thing that will sort itself out in further editorial passes and revisions anyway. But I appreciate that convention we established and eventually want to restore it. I just hope GitBook does not consistently undo that work every time we push a new version from its GUI.
• Switching to GitBook may force us to confront our inconsistent use of hyperlinks versus endnotes. Some chapters use nothing but the former; some use nothing but the latter. Most use both. GitBook does not handle Markdown-formatted endnotes as well as I'd like it to. Instead, I had to use the GitBook GUI to manually re-add all endnotes back into each chapter after "typesetting" the plain text versions. Endnotes then get rendered as hover-over popups—pretty neat. But if we're going to continue using them (and if we're going to continue publishing alternate versions of the book, e.g. PDF and ePub, we should), then we need to work on refining our practice here.
• GitBook requires a different format for file frontmatter/metadata. It reads certain fields separated by --- and --- at the start of each file (example commit). The description line gets reproduced as a chapter subtitle. You can also track author and updated. I had to learn these things by experimenting, because I don't see them documented anywhere! So I will keep looking for a complete list of available parameters we can define in the file frontmatter.

Those are the things that come to mind immediately. I'll follow up if anything else strikes me (likely to happen as soon as I step away).

Do we care that the "Powered by Gitbook" stamp is all over the site? Part of me is like "cool, support our fellow open source projects" and part of me is also "it feels like a NASCAR sticker."

I agree that it's perhaps a little more intrusive than it needs to be. However, if we intend to continue with GitBook as our publishing platform, I don't think we have much choice in the matter. Since we're now (as in: as of today) on the GitBook community plan, the banner is standard issue. We'll need to consider it our reciprocation for the freebies we receive if we choose to move forward here.

[semioticrobotic]

I've opened PR #241 so people can grasp a little more concrete what I'm advocating here.

[semioticrobotic]

Happy 2025, everyone! What would help us move this discussion forward in the new year? I am currently outlining priority projects for the coming months and would very much like to continue contributing here.

[bproffitt]

I'm going to need a moment to catch my collective breath and figure out what 1Q 2025 looks like. I should be more settled in a week's time. BKP

…

On Tue, Jan 7, 2025 at 3:45 PM Bryan Behrenshausen ***@***.***> wrote: Happy 2025, everyone! What would help us move this discussion forward in the new year? I am currently outlining priority projects for the coming months and would very much like to continue contributing here. — Reply to this email directly, view it on GitHub <#237 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPIAZMD4INMC6G6MPB7NND2JQ4EZAVCNFSM6AAAAABRA5P7UCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNZWGU2TQNQ> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[semioticrobotic]

Totally understandable! Thanks for checking in. 😄

[semioticrobotic]

It's been a few months, so (and, I realize, at the risk of becoming sort of a pest) I figured I would revisit this discussion for a gut-check from other maintainers. @bproffitt @shaunix @quaid @samus-aran: I'm wondering what ya'll think about where we stand. I'm still very keen to refresh and push this project ahead in 2025.

[semioticrobotic]

Hearing no objections, then, I'll assume a lazy consensus and prepare to make these changes in the next few days, unless someone objects. I will:

• Merge Refactor the guidebook as Markdown-first #241
• Designate https://theopensourceway.gitbook.io/guidebook the new canonical source of the guidebook
• Launch a new, simpler project website using GitHub Pages
• Get to work on open issues!

The only thing I will not be able to do is redirect our project URL, theopensourceway.org, to the new landing page.

Always grateful for feedback or guidance.

[bproffitt]

These all sound great. I apologize for the lack of feedback. As I mentioned to The Boss the other day, I (wrongly) assumed you were going to move forward. I regret the misunderstanding.

[semioticrobotic]

No sweat, @bproffitt. I'm just glad for the affirmation that we're on the right track here. I will work on the merge and follow-up items in just a bit. Meanwhile, can I lean on you to help with the domain's DNS update, assuming we wish to point to the updated and refreshed website with theopensourceway.org? I think your infra team is managing that one.

[bproffitt]

Yeah, just ping me directly and let me know what you need and when.

[semioticrobotic]

Email off to you, @bproffitt! Thanks for your willingness to investigate.

[semioticrobotic]

As of earlier today, we are live! Many thanks to all for your willingness to try this new direction. I think we can close this discussion.