Skip to content

Commit 5478c5b

Browse files
superhxKaimingWan
superhx
and
KaimingWan
authored
feat(others): migrate other files (#930)
* feat(others): migrate other files Signed-off-by: Robin Han <[email protected]> * feat(test): add s3 unit test tag Signed-off-by: Robin Han <[email protected]> --------- Signed-off-by: Robin Han <[email protected]> Co-authored-by: KamiWan <[email protected]>
1 parent 3fecccf commit 5478c5b

25 files changed

+952
-276
lines changed

‎.github/ISSUE_TEMPLATE/bug-report.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
name: "\U0001F41B Bug report"
3+
about: Something is not working
4+
title: "[BUG] "
5+
labels: bug
6+
assignees: ''
7+
8+
---
9+
10+
### Version & Environment
11+
12+
<!--
13+
14+
Please give your AutoMQ version and environment info:
15+
- AutoMQ version (e.g. `v0.1.0`):
16+
- Operating System (e.g. from `/etc/os-release`):
17+
18+
-->
19+
20+
21+
### What went wrong?
22+
23+
<!--
24+
Describe the unexpected behavior
25+
-->
26+
27+
### What should have happened instead?
28+
29+
<!--
30+
Describe what you wanted to happen.
31+
-->
32+
33+
### How to reproduce the issue?
34+
35+
1.
36+
2.
37+
3.
38+
39+
### Additional information
40+
41+
Please attach any relevant logs, backtraces, or metric charts.

‎.github/ISSUE_TEMPLATE/feature-request.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
name: "\U0001F680 Feature request"
3+
about: Suggest an idea for AutoMQ
4+
title: "[Enhancement]"
5+
labels: enhancement
6+
assignees: ''
7+
8+
---
9+
10+
### Who is this for and what problem do they have today?
11+
12+
13+
### Why is solving this problem impactful?
14+
15+
16+
### Additional notes

‎.github/ISSUE_TEMPLATE/good-first-issue.md

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
name: "⭐ Good first issue"
3+
about: Design good first issue for new contributors
4+
title: "[Good first issue] "
5+
labels: good first issue
6+
assignees: ''
7+
8+
---
9+
10+
### Background
11+
12+
<!--
13+
14+
Please give your issue background.
15+
e.g. Now AutoMQ Kafka's logs are only stored in the local disk. We want to store them in the cloud object storage as well to offer the ability to query logs from object storage. Store logs on the object storage is cheaper and more reliable.
16+
17+
-->
18+
19+
### What's our expectation for the issue
20+
21+
<!--
22+
23+
e.g. Local file logs still exist. When log is flushed to local file system, the log data will upload to object storage as well. The log path will be like `s3://bucket-name/automq/cluster-id/broker-id/logs/xx`.
24+
25+
-->
26+
27+
### How to started
28+
<!--
29+
30+
Guide the developer how to complete the issue,including:
31+
32+
e.g.
33+
- Precondition:
34+
- You need to know the principal of how AutoMQ print logs to local file system.
35+
- What main classes are involved when you are coding:
36+
- `ExampleClassA`
37+
- `ExampleClassB`
38+
- Other tips:
39+
- You can refer to the `ExampleClassA` and `ExampleClassB` of AutoMQ Kafka to get some inspiration.
40+
-->
41+
42+
### Reference
43+
- [Kafka Official Document](https://kafka.apache.org/documentation/)

‎.github/workflows/build.yml

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one or more
2+
# contributor license agreements. See the NOTICE file distributed with
3+
# this work for additional information regarding copyright ownership.
4+
# The ASF licenses this file to You under the Apache License, Version 2.0
5+
# (the "License"); you may not use this file except in compliance with
6+
# the License. You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
name: Build
17+
on:
18+
pull_request:
19+
types: [ opened, reopened, synchronize ]
20+
push:
21+
branches: [ "master", "develop" ]
22+
23+
jobs:
24+
paths-filter:
25+
runs-on: ubuntu-latest
26+
outputs:
27+
build-kos: ${{ steps.filter.outputs.build-kos }}
28+
steps:
29+
- uses: actions/checkout@v4
30+
- uses: dorny/paths-filter@v2
31+
id: filter
32+
with:
33+
filters: |
34+
build-kos:
35+
- '.github/workflows/**'
36+
- 'core/**'
37+
- 'metadata/**'
38+
- 'shell/**'
39+
build-kos:
40+
needs: [ paths-filter ]
41+
if: ${{ needs.paths-filter.outputs.build-kos == 'true' || github.event_name == 'push' }}
42+
uses: ./.github/workflows/build_kos.yml
43+
build-result:
44+
runs-on: ubuntu-latest
45+
needs: [ build-kos ]
46+
if: ${{ always() }}
47+
steps:
48+
- uses: actions/checkout@v4
49+
- name: Collect build result
50+
run: |
51+
if echo build-kos-${{ needs.build-kos.result }} | grep -E 'cancelled|failure' -o > null
52+
then
53+
echo "There are failed/cancelled builds"
54+
exit 1
55+
else
56+
echo "All builds are successful/skipped"
57+
exit 0
58+
fi

‎.github/workflows/build_kos.yml

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one or more
2+
# contributor license agreements. See the NOTICE file distributed with
3+
# this work for additional information regarding copyright ownership.
4+
# The ASF licenses this file to You under the Apache License, Version 2.0
5+
# (the "License"); you may not use this file except in compliance with
6+
# the License. You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
# This workflow uses actions that are not certified by GitHub.
17+
# They are provided by a third-party and are governed by
18+
# separate terms of service, privacy policy, and support
19+
# documentation.
20+
# This workflow will build a Java project with Gradle and cache/restore any dependencies to improve the workflow execution time
21+
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-java-with-gradle
22+
23+
name: Kafka on S3 - CI
24+
25+
on:
26+
workflow_call:
27+
28+
permissions:
29+
contents: read
30+
31+
jobs:
32+
checkstyle:
33+
name: "Checkstyle"
34+
runs-on: ${{ matrix.os }}
35+
strategy:
36+
matrix:
37+
os: [ ubuntu-22.04 ]
38+
jdk: [ 17 ]
39+
steps:
40+
- name: Checkout
41+
uses: actions/checkout@v4
42+
- name: Set up JDK ${{ matrix.jdk }}
43+
uses: actions/setup-java@v3
44+
with:
45+
java-version: ${{ matrix.jdk }}
46+
distribution: "zulu"
47+
- name: Setup Gradle
48+
uses: gradle/[email protected]
49+
- name: Checkstyle
50+
run: ./gradlew --build-cache rat checkstyleMain checkstyleTest
51+
spotbugs:
52+
name: "Spotbugs"
53+
runs-on: ${{ matrix.os }}
54+
strategy:
55+
matrix:
56+
os: [ ubuntu-22.04 ]
57+
jdk: [ 17 ]
58+
steps:
59+
- name: Checkout
60+
uses: actions/checkout@v4
61+
- name: Set up JDK ${{ matrix.jdk }}
62+
uses: actions/setup-java@v3
63+
with:
64+
java-version: ${{ matrix.jdk }}
65+
distribution: "zulu"
66+
- name: Setup Gradle
67+
uses: gradle/[email protected]
68+
- name: Spotbugs
69+
run: ./gradlew --build-cache spotbugsMain spotbugsTest
70+
test:
71+
name: "Unit Test"
72+
runs-on: ${{ matrix.os }}
73+
strategy:
74+
matrix:
75+
os: [ ubuntu-22.04 ]
76+
jdk: [ 17 ]
77+
steps:
78+
- name: Checkout
79+
uses: actions/checkout@v4
80+
- name: Set up JDK ${{ matrix.jdk }}
81+
uses: actions/setup-java@v3
82+
with:
83+
java-version: ${{ matrix.jdk }}
84+
distribution: "zulu"
85+
- name: Setup Gradle
86+
uses: gradle/[email protected]
87+
- name: Unit Test
88+
run: ./gradlew --build-cache metadata:S3UnitTest core:S3UnitTest

‎.github/workflows/docker-release.yml

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
name: Docker Release
2+
3+
on:
4+
push:
5+
tags:
6+
- '[0-9]+\.[0-9]+\.[0-9]+'
7+
- '[0-9]+\.[0-9]+\.[0-9]+\-[A-Za-z0-9]+'
8+
9+
jobs:
10+
docker-release:
11+
name: Docker Image Release
12+
strategy:
13+
matrix:
14+
platform: [ "ubuntu-22.04" ]
15+
jdk: ["17"]
16+
runs-on: ${{ matrix.platform }}
17+
permissions:
18+
contents: write
19+
steps:
20+
- name: Checkout Code
21+
uses: actions/checkout@v4
22+
- name: Set up JDK ${{ matrix.jdk }}
23+
uses: actions/setup-java@v3
24+
with:
25+
java-version: ${{ matrix.jdk }}
26+
distribution: "zulu"
27+
- name: Setup Gradle
28+
uses: gradle/[email protected]
29+
- name: Get project version
30+
id: get_project_version
31+
run: |
32+
project_version=$(./gradlew properties | grep "version:" | awk '{print $2}')
33+
echo "PROJECT_VERSION=${project_version}" >> $GITHUB_OUTPUT
34+
35+
- name: Build TarGz
36+
run: |
37+
./gradlew -Pprefix=automq-${{ github.ref_name }}_ --build-cache --refresh-dependencies clean releaseTarGz
38+
39+
# docker image release
40+
- name: Cp TarGz to Docker Path
41+
run: |
42+
cp ./core/build/distributions/automq-${{ github.ref_name }}_kafka-${{ steps.get_project_version.outputs.PROJECT_VERSION }}.tgz ./docker/
43+
- name: Set up QEMU
44+
uses: docker/setup-qemu-action@v3
45+
- name: Set up Docker Buildx
46+
uses: docker/setup-buildx-action@v3
47+
- name: Login to Docker Hub
48+
uses: docker/login-action@v3
49+
with:
50+
username: ${{ secrets.DOCKERHUB_USERNAME }}
51+
password: ${{ secrets.DOCKERHUB_READ_WRITE_TOKEN }}
52+
- name: Build and push
53+
uses: docker/build-push-action@v5
54+
with:
55+
context: ./docker
56+
push: true
57+
tags: ${{ secrets.DOCKERHUB_USERNAME }}/kafka:${{ github.ref_name }}
58+
platforms: linux/amd64,linux/arm64

‎.github/workflows/e2e-run.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Run E2E tests
2+
on:
3+
workflow_call:
4+
inputs:
5+
runner:
6+
required: true
7+
type: string
8+
suite-id:
9+
required: true
10+
type: string
11+
test-yaml:
12+
required: false
13+
type: string
14+
test-path:
15+
required: false
16+
type: string
17+
storage-path:
18+
required: true
19+
type: string
20+
21+
jobs:
22+
run_e2e:
23+
name: "Run E2E tests"
24+
runs-on: ${{ inputs.runner }}
25+
env:
26+
TC_GENERAL_MIRROR_URL: "mirrors.ustc.edu.cn"
27+
steps:
28+
- name: Checkout
29+
uses: actions/checkout@v4
30+
- name: Clean last running results
31+
run: |
32+
rm -rf results
33+
rm -rf "${{ inputs.storage-path }}/${{ inputs.suite-id }}"
34+
- name: Setup Gradle
35+
uses: gradle/[email protected]
36+
- name: Run E2E tests with yaml
37+
if: ${{ inputs.test-path == '' }}
38+
run: ./tests/docker/run_tests.sh
39+
env:
40+
ESK_TEST_YML: ${{ inputs.test-yaml }}
41+
shell: bash
42+
- name: Run E2E tests with path
43+
if: ${{ inputs.test-path != '' }}
44+
run: ./tests/docker/run_tests.sh
45+
env:
46+
TC_PATHS: ${{ inputs.test-path }}
47+
shell: bash
48+
- name: Move results
49+
run: |
50+
results_path="$(pwd)/results/$(readlink results/latest | cut -d'/' -f5)"
51+
mv "${results_path}" "${{ inputs.storage-path }}/${{ inputs.suite-id }}"
52+
if: ${{ always() }}
53+
shell: bash
54+
- name: Bring down docker containers
55+
run: ./tests/docker/ducker-ak down
56+
shell: bash
57+
if: ${{ always() }}

0 commit comments

Comments
 (0)