Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add blog posts about Binder federation member and docker stacks #356

Merged
merged 11 commits into from
Jan 29, 2025
17 changes: 17 additions & 0 deletions content/blog/2023/docker-stacks-support/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
title: "2i2c supports Jupyter Docker Stacks ARM builds"
date: "2023-12-01"
authors: ["Chris Holdgraf", "Yuvi Panda"]
tags: [open source]
categories: [impact]
featured: false
draft: false
---

The [Jupyter Docker Stacks](https://jupyter-docker-stacks.readthedocs.io/) project provides a collection of ready-to-use Docker images for Jupyter environments. These images are used by many in the Jupyter community, including 2i2c which uses them as base images for our JupyterHub deployments.

The project recently began publishing [ARM-compatible images](https://github.com/jupyter/docker-stacks/issues/1019) alongside the standard x86 images, making it easier for users with ARM-based systems (like M1 Macs) to use these environments. However, building and hosting these ARM images comes with additional cloud computing costs that were being personally covered by [@mathbunnyru](https://github.com/mathbunnyru), one of the project's maintainers.

A part of 2i2c's mission is supporting upstream communities that we rely on, especially where the upstream project has limited resources. For this reason, we've decided to support Jupyter Docker Stack's ARM building costs, with a total budget of `$2000` (approximately `$150` per month). As a regular user and beneficiary of the Jupyter Docker Stacks, we believe it's important to contribute to the maintenance and sustainability of this crucial piece of infrastructure that benefits the entire Jupyter community.

We hope this support helps the Docker Stacks project remain healthy, and continue providing high-quality, multi-architecture images that work across different computing platforms. We'll revisit this decision as the landscape of technology providers changes and other options arise.
35 changes: 35 additions & 0 deletions content/blog/2025/binder-singlenode/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
title: "Supporting mybinder.org with a single node federation member"
date: "2025-01-18"
authors: ["Yuvi Panda", "Chris Holdgraf"]
tags: [open source]
categories: [impact]
featured: false
draft: false
---

{{% callout note %}}
If you're interested in supporting mybinder.org with cloud resources, financial resources, or human resources, please see the [Support Binder](https://mybinder.readthedocs.io/en/latest/about/support.html) page for how you can help.
{{% /callout %}}

[mybinder.org](https://mybinder.org) is a massive public service for creating and sharing reproducible computational environments. It is managed by the JupyterHub team and [members of the mybinder.org federation](https://mybinder.readthedocs.io/en/latest/about/federation.html). One of the challenges running [mybinder.org](https://mybinder.org) is identifying suppliers of cloud credits or financial resources to support the service. Two years ago, [Google stopped supporting mybinder.org federation with cloud credits](https://medium.com/jupyter-blog/mybinder-org-reducing-capacity-c93ccfc6413f), and last month [the federation lost more capacity](https://discourse.jupyter.org/t/mybinder-org-reduced-capacity-stability/31750).

While we're thinking through long-term ways to support Binder, 2i2c also wanted to provide short-term support for the project, and experiment with cheaper models for deploying Binder's reproducible environments. Making it simpler / easier to add capacity to the mybinder.org federation would both improve some of Binder's short-term capacity shortage, and may also provide an easier pathway for others to support the project in the future.

So we're launching an experiment to deploy a Binder federation member via the simplest and cheapest cloud solution we could identify. The Kubernetes (and in particular, [k3s](https://k3s.io/)) ecosystem has matured, and there are some interesting cloud offerings that make a _single-node kubernetes cluster_ logistically accessible for workflows like mybinder.org. We're going to deploy a single-node Kubernetes cluster running mybinder.org sessions, hosted on [Hetzner](https://www.hetzner.com/cloud/).

Initially this is just an experiment to see how it goes, whether it's sustainable from a cloud costs and labor standpoint, and what we can learn and share for others. Here's a rough plan:

- 2i2c sponsors a max of €300 a month (with some currency conversion noise) to fund the cloud costs of a new federation member.
choldgraf marked this conversation as resolved.
Show resolved Hide resolved
- We'll provide in-kind labor to run this node, and treat it as an organizational investment in learning new Kubernetes and cloud infrastructure workflows.
- In six months, se'll evaluate how much effort it was to run this node for mybinder.org, whether it meaningfully helped with mybinder's capacity, and whether it was sustainable for us from a time and labor perspective.
choldgraf marked this conversation as resolved.
Show resolved Hide resolved

Our rationale is that, with a cheaper cloud provider like Hetzner, we can get a pretty large node running persistently for around €300 a month. The equivalent cost on AWS would be $1,500, and on Digital Ocean it'd be about $1,000, so this is already roughly a factor of three cheaper than the previous cheapest option. In addition, we've been developing some experience running single-node Kubernetes workflows via [k3s](https://k3s.io/) and a collaboration with [Carl Boettiger](https://carlboettiger.info/), so we think that we've got some expertise to deploy and manage this efficiently. Finally, because it's a single node cluster, there is no auto-scaling (one reason it is so cheap), which reduces a lot of the complexity we'll have to manage. These are acceptable tradeoffs for a service like mybinder.org, which runs entirely ephemeral sessions with very limited resources and no promises about uptime, persistence, etc.

We're excited to experiment with new ways to support mybinder.org, and to identify sustainable models for deploying Binder and Jupyter infrastructure for communities in a way that could benefit the wider ecosystem as well. We'll report back on how this experiment goes!


{{% callout note %}}
If you're interested in supporting mybinder.org with cloud resources, financial resources, or human resources, please see the [Support Binder](https://mybinder.readthedocs.io/en/latest/about/support.html) page for how you can help.
{{% /callout %}}

2 changes: 1 addition & 1 deletion content/blog/2025/jupyterhub5-upgrade/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: 2i2c hubs now run JupyterHub 5.0
date: "2025-01-17"
authors: ["Georgiana Dolocan"]
tags: []
categories: []
categories: [enhancements]
featured: false
draft: false
---
Expand Down
Loading