-
Notifications
You must be signed in to change notification settings - Fork 20
[Docs] Add CLion GDB Remote Debugging Support via Docker #43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
YoungHypo
wants to merge
8
commits into
bytedance:main
Choose a base branch
from
YoungHypo:debug_guide
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
b6dc8e7
add new dockerfile for debug mode and debug_guide.md
YoungHypo cb86c0b
add imgs for debug_guide.md
YoungHypo db1e9e6
Update debug_guide.md
YoungHypo 40e355d
add img for local terminal check
YoungHypo 3770f47
Update debug_guide.md for adjusting the img size
YoungHypo d47b5a6
Merge branch 'main' into debug_guide
YoungHypo 91e1eec
update debug_guide.md for its intro
YoungHypo 20265a9
fix review from copliot
YoungHypo File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,130 @@ | ||
| # build GCC | ||
| FROM debian:bullseye AS gcc-builder | ||
| ENV GCC_VERSION=9.3.0 | ||
| RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ | ||
| build-essential wget \ | ||
| libgmp-dev libmpfr-dev libmpc-dev \ | ||
| && apt-get clean && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| WORKDIR /build | ||
| RUN wget https://mirrors.tuna.tsinghua.edu.cn/gnu/gcc/gcc-${GCC_VERSION}/gcc-${GCC_VERSION}.tar.gz \ | ||
| && tar -xf gcc-${GCC_VERSION}.tar.gz \ | ||
| && cd gcc-${GCC_VERSION} \ | ||
| && mkdir build && cd build \ | ||
| && ../configure \ | ||
| --prefix=/usr/local/gcc-${GCC_VERSION} \ | ||
| --disable-multilib \ | ||
| --enable-languages=c,c++ \ | ||
| --disable-bootstrap \ | ||
| --disable-nls \ | ||
| --disable-libsanitizer \ | ||
| --disable-libvtv \ | ||
| --disable-libssp \ | ||
| --disable-libquadmath \ | ||
| --disable-libgomp \ | ||
| --disable-libada \ | ||
| --disable-libstdcxx-pch \ | ||
| && make -j $(nproc) \ | ||
| && make install-strip DESTDIR=/gcc-install \ | ||
| && cd /build && rm -rf * | ||
|
|
||
| # build Bison | ||
| FROM debian:bullseye AS bison-builder | ||
| ENV BISON_VERSION=3.4.2 | ||
| RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ | ||
| build-essential wget \ | ||
| flex \ | ||
| m4 \ | ||
| && apt-get clean && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| WORKDIR /build | ||
| RUN wget --no-check-certificate https://ftp.gnu.org/gnu/bison/bison-${BISON_VERSION}.tar.gz \ | ||
| && tar -xf bison-${BISON_VERSION}.tar.gz \ | ||
| && cd bison-${BISON_VERSION} \ | ||
| && ./configure --prefix=/usr/local \ | ||
| && make -j $(nproc) \ | ||
| && make install DESTDIR=/bison-install \ | ||
| && cd /build && rm -rf * | ||
|
|
||
| # final stage | ||
| FROM debian:bullseye | ||
| COPY --from=gcc-builder /gcc-install/usr/local/gcc-9.3.0 /usr/local/gcc-9.3.0 | ||
| COPY --from=bison-builder /bison-install/usr/local/bin/bison /usr/local/bin/bison | ||
| COPY --from=bison-builder /bison-install/usr/local/lib/liby.* /usr/local/lib/ | ||
| COPY --from=bison-builder /bison-install/usr/local/share/bison /usr/local/share/bison | ||
|
|
||
| # online required part | ||
| RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \ | ||
| cmake \ | ||
| curl \ | ||
| python3.9 \ | ||
| python3.9-dev \ | ||
| python3.9-venv \ | ||
| libgmp10 \ | ||
| libmpfr6 \ | ||
| libmpc3 \ | ||
| m4 \ | ||
| pkg-config \ | ||
| git \ | ||
| git-lfs \ | ||
| libssl-dev \ | ||
| libreadline-dev \ | ||
| zlib1g-dev \ | ||
| libcurl4-openssl-dev \ | ||
| libldap2-dev \ | ||
| libsasl2-dev \ | ||
| libsasl2-modules-gssapi-mit \ | ||
| libkrb5-dev \ | ||
| libnuma-dev \ | ||
| libmecab-dev \ | ||
| libaio-dev \ | ||
| libncurses-dev \ | ||
| libtirpc-dev \ | ||
| gdb \ | ||
| gdbserver \ | ||
| openssh-server \ | ||
| net-tools \ | ||
| rsync \ | ||
| && apt-get clean && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # configure Python and other tools | ||
| RUN ln -sf /usr/local/gcc-9.3.0/bin/gcc /usr/local/bin/gcc \ | ||
| && ln -sf /usr/bin/python3.9 /usr/bin/python3 \ | ||
| && ln -sf /usr/bin/python3.9 /usr/bin/python \ | ||
| && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py \ | ||
| && python3.9 get-pip.py \ | ||
| && rm get-pip.py | ||
|
|
||
| # validation | ||
| RUN gcc --version | grep "9.3.0" \ | ||
| && bison --version \ | ||
| && curl --version | grep "curl" \ | ||
| && python3.9 --version | grep "3.9" \ | ||
| && cmake --version | grep "cmake version" \ | ||
| && gdb --version | grep "GNU gdb" \ | ||
| && rsync --version | grep "rsync" \ | ||
| && echo "All version checks passed!" | ||
|
|
||
| ENV PATH=/usr/local/gcc-9.3.0/bin:/opt/tiger/typhoon-blade:/opt/common_tools:$PATH | ||
| ENV LD_LIBRARY_PATH=/usr/local/gcc-9.3.0/lib64 | ||
|
|
||
| # SECURITY WARNING: | ||
| # The root password is set via the ROOT_PASSWORD build argument (default: 'root'). | ||
| # DO NOT use the default password in production or internet-accessible environments. | ||
| # Always override ROOT_PASSWORD at build time for any non-debug use: | ||
| # docker build --build-arg ROOT_PASSWORD=your_strong_password -f Dockerfile.debug_env . | ||
| ARG ROOT_PASSWORD=root | ||
| RUN mkdir -p /var/run/sshd && \ | ||
| echo "root:${ROOT_PASSWORD}" | chpasswd && \ | ||
| sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config && \ | ||
| sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config && \ | ||
| echo 'ClientAliveInterval 60' >> /etc/ssh/sshd_config | ||
|
|
||
| # Application ports: | ||
| # 13308 - MySQL service | ||
| # 5001 - Videx service | ||
| # 1234 - GDB debug server | ||
| # 22 - SSH service | ||
| EXPOSE 13308 5001 1234 22 | ||
|
|
||
| CMD ["/etc/init.d/ssh", "start"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| #!/usr/bin/env bash | ||
| SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" | ||
| source "${SCRIPT_DIR}/config.sh" | ||
|
|
||
| # Error handling | ||
| set -e # Exit on error | ||
| set -x # Print commands for debugging | ||
|
|
||
| # Start SSH service | ||
| /etc/init.d/ssh restart | ||
|
|
||
| # Copy videx to storage | ||
| if [ -d "$MYSQL_HOME/storage/videx" ]; then | ||
| echo "Videx directory already exists. Skipping copy." | ||
| else | ||
| echo "Copying videx to $MYSQL_HOME/storage..." | ||
| cp -r "$VIDEX_HOME/src/mysql/videx" "$MYSQL_HOME/storage" | ||
| fi | ||
|
|
||
| BOOST_DIR=$MYSQL_HOME/boost | ||
|
|
||
| # Create necessary directories | ||
| mkdir -p "$BOOST_DIR" | ||
| mkdir -p "$MYSQL_BUILD_DIR"/{etc,build,lib64,data,log} | ||
|
|
||
| # Copy my.cnf to build directory | ||
| if [ -f "$SCRIPT_DIR/my.cnf" ]; then | ||
| cp "$SCRIPT_DIR/my.cnf" "$MYSQL_BUILD_DIR/etc/my.cnf" | ||
| else | ||
| echo "Warning: my.cnf not found!" | ||
| exit 1 | ||
| fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,167 @@ | ||
| # CLion GDB Remote Debugging VIDEX Guide | ||
|
|
||
| This guide shows how to mount the `MySQL8` and `videx` projects into a container using Docker, creating a debugging environment isolated from the host environment. The code in the container can be affected by local modifications in real-time, facilitating development and debugging. | ||
|
|
||
| **Note: This guide is currently only applicable for macOS.** | ||
|
|
||
| ## 1. Preparation | ||
|
|
||
| Please follow the instructions in **Section 1 Preparation** in the [installation guide](https://github.com/bytedance/videx/blob/main/doc/installation.md). | ||
|
|
||
| ## 2. Build and Start the Debug Container | ||
|
|
||
| ### 2.1 Build Docker Image | ||
|
|
||
| ```bash | ||
| cd $VIDEX_HOME | ||
| docker build -t videx_debug:latest -f build/Dockerfile.debug_env . | ||
| ``` | ||
|
|
||
| ### 2.2 Start the Container and Mount the Project | ||
|
|
||
| ```bash | ||
| docker run -dit \ | ||
| --name clion-videx-gdb \ | ||
| -p 2222:22 -p 13308:13308 -p 5001:5001 -p 1234:1234 \ | ||
| -v $MySQL8_HOME:/root/mysql_server \ | ||
| -v $VIDEX_HOME:/root/videx_server \ | ||
| videx_debug:latest \ | ||
| sleep infinity | ||
| ``` | ||
|
|
||
| ### 2.3 Run the Debug Script in the Container | ||
|
|
||
| **Required for the First Time.** | ||
| Enter the container and execute the compile command: | ||
|
|
||
| ```bash | ||
| cd /root/videx_server/build | ||
| chmod +x *.sh | ||
| ./debug.sh | ||
| ``` | ||
|
|
||
|  | ||
|
|
||
| ## 3. Open Project in CLion | ||
|
|
||
| Open the local `mysql_server` project. | ||
|
|
||
| ## 4. Configure Remote Development Environment | ||
|
|
||
| ### 4.1 Configure SSH Connection | ||
| 1. Go to `Settings` → `Tools` → `SSH Configurations` | ||
| 2. Add a new configuration: | ||
| - Host: localhost | ||
| - Port: 2222 | ||
| - User name: root | ||
| - Password: root | ||
| 3. Click `Test Connection` to verify the connection | ||
|
|
||
|  | ||
|
|
||
| ### 4.2 Configure Remote Toolchain | ||
| 1. Go to `Settings` → `Build, Execution, Deployment` → `Toolchains` | ||
| 2. Add a new Remote Host toolchain: | ||
| - Name: Remote Host | ||
| - C++ Compiler: /usr/local/gcc-9.3.0/bin/g++ | ||
|
|
||
|  | ||
|
|
||
| ## 5. Configure Remote Path Mapping | ||
|
|
||
| 1. Go to `Settings` → `Build, Execution, Deployment` → `Deployment` | ||
| 2. Add a new deployment configuration: | ||
| - Type: SFTP | ||
| - SSH Configuration: Use the SSH configured in the previous step | ||
| - Root path: /root | ||
| - Select `Use rsync for download/...` | ||
| 3. In the Mappings tab: | ||
| - Local path: Set to the mysql_server path on **your local host** | ||
| - Deployment path: /mysql_server | ||
|
|
||
|  | ||
|
|
||
|  | ||
|
|
||
| ## 6. Configure CMake Profile | ||
| 1. Go to `Settings` → `Build, Execution, Deployment` → `CMake` | ||
| 2. Add a new CMake Profile: | ||
| - Build type: Debug | ||
| - Toolchain: Remote Host | ||
| 3. CMake options: | ||
|
|
||
| ```bash | ||
| -DWITH_DEBUG=ON -DCMAKE_BUILD_TYPE=Debug -DBUILD_CONFIG=mysql_release -DFEATURE_SET=community -DCMAKE_INSTALL_PREFIX=. -DMYSQL_DATADIR=./data -DSYSCONFDIR=./etc -DWITH_BOOST=/root/mysql_server/boost -DDOWNLOAD_BOOST=ON -DWITH_ROCKSDB=OFF -DDOWNLOAD_BOOST_TIMEOUT=3600 -DWITH_VIDEX_STORAGE_ENGINE=1 | ||
| ``` | ||
|
|
||
| 4. Build directory: mysql_build_output/build | ||
|
|
||
|  | ||
|
|
||
| ## 7. Configure and Start MySQL Service | ||
|
|
||
| ### 7.1 Initialize mysqld | ||
|
|
||
| 1. Go to `Run/Debug Configurations` | ||
| 2. Search mysqld and add a new instance | ||
| - Name: mysqld-init | ||
| - Target: mysqld | ||
| 3. Set the program arguments and working directory | ||
| 4. Click ▶️ to run the mysqld-init | ||
|
|
||
| ```bash | ||
| --defaults-file=./etc/my.cnf --initialize-insecure --user=root --basedir=./ --datadir=./data | ||
| ``` | ||
|
|
||
|  | ||
|
|
||
| ### 7.2 Start mysqld | ||
|
|
||
| 1. Go to `Run/Debug Configurations` | ||
| 2. Search mysqld and add a new instance | ||
| - Name: mysqld-run | ||
| - Target: mysqld | ||
| 3. Set the program arguments and working directory | ||
| 4. Click ▶️ to run the mysqld-run | ||
|
|
||
| ```bash | ||
| --defaults-file=./etc/my.cnf --user=root --basedir=./ --datadir=./data --socket=./mysql_80.sock --port=13308 | ||
| ``` | ||
|
|
||
|  | ||
|
|
||
| ## 8. Create Debug User | ||
|
|
||
| Execute in the container terminal: | ||
|
|
||
| ```bash | ||
| /root/mysql_server/mysql_build_output/build/runtime_output_directory/mysql \ | ||
| -h127.0.0.1 -uroot -P13308 \ | ||
| -e "CREATE USER 'videx'@'%' IDENTIFIED WITH mysql_native_password BY 'password'; \ | ||
| GRANT ALL ON *.* TO 'videx'@'%'; FLUSH PRIVILEGES;" | ||
| ``` | ||
|
|
||
| Test in your local terminal: | ||
|
|
||
| ```bash | ||
| mysql -h127.0.0.1 -P13308 -uvidex -ppassword -e 'SELECT 1;' | ||
| ``` | ||
|
|
||
| You will get the following result if all goes well | ||
|
|
||
| <img src="debug_img/local_check.png" alt="local terminal check" width="600"/> | ||
|
|
||
| ## 9. Start Videx Server | ||
| ```bash | ||
| cd /root/videx_server | ||
| python3.9 -m pip install -e . --use-pep517 | ||
|
|
||
| cd src/sub_platforms/sql_opt/videx/scripts | ||
| python3.9 start_videx_server.py --port 5001 | ||
| ``` | ||
|
|
||
| ## 10. Start GDB Debugging | ||
|
|
||
| Switch to mysqld-run, and click debug | ||
|
|
||
|  |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Trailing whitespace: Line 120 has trailing whitespace after "Videx service" that should be removed for consistency.