-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add blog posts about Binder federation member and docker stacks
- Loading branch information
Showing
3 changed files
with
53 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
--- | ||
title: "2i2c supports Jupyter Docker Stacks ARM builds" | ||
date: "2023-12-01" | ||
authors: ["Chris Holdgraf", "Yuvi Panda"] | ||
tags: [open source] | ||
categories: [impact] | ||
featured: false | ||
draft: false | ||
--- | ||
|
||
The [Jupyter Docker Stacks](https://jupyter-docker-stacks.readthedocs.io/) project provides a collection of ready-to-use Docker images for Jupyter environments. These images are used by many in the Jupyter community, including 2i2c which uses them as base images for our JupyterHub deployments. | ||
|
||
The project recently began publishing [ARM-compatible images](https://github.com/jupyter/docker-stacks/issues/1019) alongside the standard x86 images, making it easier for users with ARM-based systems (like M1 Macs) to use these environments. However, building and hosting these ARM images comes with additional cloud computing costs that were being personally covered by [@mathbunnyru](https://github.com/mathbunnyru), one of the project's maintainers. | ||
|
||
A part of 2i2c's mission is supporting upstream communities that we rely on, especially where the upstream project has limited resources. For this reason, we've decided to support Jupyter Docker Stack's ARM building costs, with a total budget of `$2000` (approximately `$150` per month). As a regular user and beneficiary of the Jupyter Docker Stacks, we believe it's important to contribute to the maintenance and sustainability of this crucial piece of infrastructure that benefits the entire Jupyter community. | ||
|
||
We hope this support helps the Docker Stacks project remain healthy, and continue providing high-quality, multi-architecture images that work across different computing platforms. We'll revisit this decision as the landscape of technology providers changes and other options arise. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
--- | ||
title: "Supporting mybinder.org with a single node federation member" | ||
date: "2025-01-18" | ||
authors: ["Yuvi Panda", "Chris Holdgraf"] | ||
tags: [open source] | ||
categories: [impact] | ||
featured: false | ||
draft: false | ||
--- | ||
|
||
{{% callout note %}} | ||
If you're interested in supporting mybinder.org with cloud resources, financial resources, or human resources, please see the [Support Binder](https://mybinder.readthedocs.io/en/latest/about/support.html) page for how you can help. | ||
{{% /callout %}} | ||
|
||
[mybinder.org](https://mybinder.org) is a massive public service for creating and sharing reproducible computational environments. It is managed by the JupyterHub team and [members of the mybinder.org federation](https://mybinder.readthedocs.io/en/latest/about/federation.html). One of the challenges running [mybinder.org](https://mybinder.org) is identifying suppliers of cloud credits or financial resources to support the service. Two years ago, [Google stopped supporting mybinder.org federation with cloud credits](https://medium.com/jupyter-blog/mybinder-org-reducing-capacity-c93ccfc6413f), and last month [the federation lost more capacity](https://discourse.jupyter.org/t/mybinder-org-reduced-capacity-stability/31750). | ||
|
||
While we're thinking through long-term ways to support Binder, 2i2c also wanted to provide short-term support for the project, and experiment with cheaper models for deploying Binder's reproducible environments. Making it simpler / easier to add capacity to the mybinder.org federation would both improve some of Binder's short-term capacity shortage, and may also provide an easier pathway for others to support the project in the future. | ||
|
||
So we're launching an experiment to deploy a Binder federation member via the simplest and cheapest cloud solution we could identify. The Kubernetes (and in particular, [k3s](https://k3s.io/)) ecosystem has matured, and there are some interesting cloud offerings that make a _single-node kubernetes cluster_ logistically accessible for workflows like mybinder.org. We're going to deploy a single-node Kubernetes cluster running mybinder.org sessions, hosted on [Hetzner](https://www.hetzner.com/cloud/). | ||
|
||
Initially this is just an experiment to see how it goes, whether it's sustainable from a cloud costs and labor standpoint, and what we can learn and share for others. Here's a rough plan: | ||
|
||
- 2i2c sponsors a max of €300 a month (with some currency conversion noise) to fund the cloud costs of a new federation member. | ||
- We'll provide in-kind labor to run this node, and treat it as an organizational investment in learning new Kubernetes and cloud infrastructure workflows. | ||
- In six months, se'll evaluate how much effort it was to run this node for mybinder.org, whether it meaningfully helped with mybinder's capacity, and whether it was sustainable for us from a time and labor perspective. | ||
|
||
Our rationale is that, with a cheaper cloud provider like Hetzner, we can get a pretty large node running persistently for around €300 a month. The equivalent cost on AWS would be $1,500, and on Digital Ocean it'd be about $1,000, so this is already roughly a factor of three cheaper than the previous cheapest option. In addition, we've been developing some experience running single-node Kubernetes workflows via [k3s](https://k3s.io/) and a collaboration with [Carl Boettiger](https://carlboettiger.info/), so we think that we've got some expertise to deploy and manage this efficiently. Finally, because it's a single node cluster, there is no auto-scaling (one reason it is so cheap), which reduces a lot of the complexity we'll have to manage. These are acceptable tradeoffs for a service like mybinder.org, which runs entirely ephemeral sessions with very limited resources and no promises about uptime, persistence, etc. | ||
|
||
We're excited to experiment with new ways to support mybinder.org, and to identify sustainable models for deploying Binder and Jupyter infrastructure for communities in a way that could benefit the wider ecosystem as well. We'll report back on how this experiment goes! | ||
|
||
|
||
{{% callout note %}} | ||
If you're interested in supporting mybinder.org with cloud resources, financial resources, or human resources, please see the [Support Binder](https://mybinder.readthedocs.io/en/latest/about/support.html) page for how you can help. | ||
{{% /callout %}} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters