Skip to content

Commit c0c36c4

Browse files
authored
Merge pull request #42 from buildkite/update-rails-docker-parallel-example
Update Rails Docker Parallel Example (consistency sweep)
2 parents b8f7e87 + ef202bc commit c0c36c4

File tree

4 files changed

+73
-49
lines changed

4 files changed

+73
-49
lines changed

.buildkite/template.yml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,12 @@
1-
# Used by the 'Add to Buildkite' button in the readme
21
name: "Rails Parallel Docker Example"
2+
description: "An example pipeline that runs parallel tests for a Rails app using Docker and Knapsack."
3+
emoji: ":rails:"
4+
color: "#FDF5F5"
5+
languages:
6+
- "Ruby"
7+
- "Rails"
8+
- "Docker"
9+
- "Shell"
310
steps:
4-
- command: "buildkite-agent pipeline upload"
5-
label: ":pipeline:"
11+
- label: ":pipeline:"
12+
command: "buildkite-agent pipeline upload"

License.md renamed to LICENSE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
The MIT License (MIT)
22

3-
Copyright (c) 2016 Buildkite Pty Ltd
3+
Copyright (c) 2025 Buildkite Pty Ltd
44

55
Permission is hereby granted, free of charge, to any person obtaining a copy
66
of this software and associated documentation files (the "Software"), to deal

README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Buildkite Rails Docker Parallel Pipeline Example
2+
3+
[![Build status](https://badge.buildkite.com/aab023f2f33ab06766ed6236bc40caf0df1d9448e4f590d0ee.svg?branch=main)](https://buildkite.com/buildkite/rails-docker-parallel-example/builds/latest?branch=main)
4+
[![Add to Buildkite](https://img.shields.io/badge/Add%20to%20Buildkite-14CC80)](https://buildkite.com/new)
5+
6+
This repository is an example [Buildkite](https://buildkite.com/) pipeline that demonstrates how to run a large number of parallel test jobs for a [Rails](https://rubyonrails.org/) application using [Docker](https://www.docker.com/) and [Knapsack](https://github.com/ArturT/knapsack).
7+
8+
👉 **See this example in action:** [buildkite/rails-docker-parallel-example](https://buildkite.com/buildkite/rails-docker-parallel-example/builds/latest?branch=main)
9+
10+
[![Add to Buildkite](https://buildkite.com/button.svg)](https://buildkite.com/new)
11+
12+
<a href="https://buildkite.com/buildkite/rails-docker-parallel-example/builds/latest?branch=main">
13+
<img width="1491" alt="Screenshot of Buildkite Rails Docker Parallel example pipeline" src=".buildkite/screenshot.png" />
14+
</a>
15+
16+
<!-- docs:start -->
17+
18+
## How it works
19+
In the [pipeline configuration file](.buildkite/pipeline.yml) the `parallelism` property for the test step is set to 20. When a build, the step will appear 20 times in the pipeline, each with different environment variables exposed so you can divvy up your test suite accordingly. You can set the parallelism as high as you need to bring down your build times.
20+
21+
This example:
22+
- Uses Docker Compose to orchestrate the environment (Redis, Postgres, Memcached)
23+
- Runs tests in parallel across 20 containers using Buildkite’s `parallelism` feature
24+
- Leverages [Knapsack](https://github.com/ArturT/knapsack) for even test distribution
25+
- Is suitable for use with autoscaling environments like the [Buildkite AWS Stack](https://github.com/buildkite/elastic-ci-stack-for-aws)
26+
27+
Key files:
28+
- [`docker-compose.yml`](docker-compose.yml): defines app services
29+
- [`Dockerfile`](Dockerfile): sets up the Ruby environment and database waiters
30+
- [`pipeline.yml`](.buildkite/pipeline.yml): configures a parallel test step
31+
- [`scripts/ci/parallel_specs.sh`](scripts/ci/parallel_specs.sh): runs specs in parallel
32+
33+
### Parallel test execution
34+
35+
In [`pipeline.yml`](.buildkite/pipeline.yml), the test step is configured with:
36+
37+
```yaml
38+
parallelism: 20
39+
```
40+
41+
This causes the step to run 20 times in parallel. Each instance is exposed to environment variables like `BUILDKITE_PARALLEL_JOB` and `BUILDKITE_PARALLEL_JOB_COUNT`, which can be used to split the test suite dynamically.
42+
43+
## Alternate setups
44+
45+
> 💡 Not using Docker? Try the [Rails Parallel Example](https://github.com/buildkite/rails-parallel-example)
46+
47+
> 💡 Want dynamic splitting/allocation? See the [Knapsack Pro version](https://github.com/KnapsackPro/buildkite-rails-docker-parallel-example-with-knapsack_pro)
48+
49+
50+
## Further Reading
51+
52+
See the [parallelizing builds guide](https://buildkite.com/docs/guides/parallelizing-builds) for more information to create parallelized and distributed builds with Buildkite.
53+
54+
See the [Containerized Builds with Docker guide](https://buildkite.com/docs/guides/docker-containerized-builds) for information on how to use Docker Compose with Buildkite pipelines.
55+
56+
Consult the [Knapsack documentation](https://github.com/ArturT/knapsack) for configuring your database and dependent services to support running parallel steps on the one machine.
57+
58+
<!-- docs:end -->
59+
60+
## License
61+
62+
See [LICENSE.md](LICENSE.md) (MIT)

Readme.md

Lines changed: 0 additions & 45 deletions
This file was deleted.

0 commit comments

Comments
 (0)