Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build p4c from custom fork #249

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions dash-pipeline/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ DOCKER_SAITHRIFT_BLDR_IMG ?=chrissommers/dash-saithrift-bldr:220819
# Base image with test frameworks, DASH client libs not installed
DOCKER_SAITHRIFT_CLIENT_BLDR_IMG ?=chrissommers/dash-saithrift-client-bldr:220819

# P4c builder - p4c build dependencies, custom p4c fork
DOCKER_P4C_BLDR_IMG ?=chrissommers/dash-p4c-bldr:221007
Copy link
Collaborator

@chrispsommers chrispsommers Nov 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, following #225 docker images are defined in .env files under dockerfiles/ and are included into the Makefile. You should sync to main, thanks.


# Client image built locally with saithrift client libs + PTF & Pytest frameworks
# TODO: consider some other tagging scheme
DOCKER_SAITHRIFT_CLIENT_IMG ?= local/dash-saithrift-client:latest
Expand Down Expand Up @@ -412,6 +415,19 @@ run-saithrift-client-bash:
# --build-arg available_processors=$(shell nproc) \
# dockerfiles

# Builder, has base packages to make client docker
docker-dash-p4c-bldr:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You also need to make target docker-pull-dash-p4c-bldr which will be called in CI scripts. (We docker pull deliberately to make a failure more obvious.)
More importantly, you need to manually build and push this image to dockerhub for CI to pass, because the Dockerfile which uses this as a base cannot be found outside your PC environment, hence this CI failure:
https://github.com/Azure/DASH/actions/runs/3208306232/jobs/5244034990#step:3:570
I just made a new repo chrissommers/dash-p4c-bldr and added you as a collaborator. You can use this until we migrate to ACR.

docker build \
-f dockerfiles/Dockerfile.p4c-bldr \
-t $(DOCKER_P4C_BLDR_IMG) \
--build-arg user=$(DASH_USER) \
--build-arg group=$(DASH_GROUP) \
--build-arg uid=$(DASH_UID) \
--build-arg guid=$(DASH_GUID) \
--build-arg hostname=$(DASH_HOST) \
--build-arg available_processors=$(shell nproc) \
./tests

docker-dash-p4c:
docker build \
-f dockerfiles/Dockerfile.p4c-bmv2 \
Expand Down
113 changes: 113 additions & 0 deletions dash-pipeline/dockerfiles/Dockerfile.p4c-bldr
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# It uses grpc 1.43.2 as base to ensure right lib versions.
FROM chrissommers/dash-grpc:1.43.2 as grpc
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use sonicdash.azurecr.io/dash-grpc:1.43.2


# https://hub.docker.com/r/p4lang/behavioral-model
FROM p4lang/behavioral-model:no-pi
LABEL maintainer="SONIC-DASH Community"
LABEL description="DASH pipeline P4 Behavioral Model and toolchain"

# Configure make to run as many parallel jobs as cores available
ARG available_processors
ARG MAKEFLAGS=-j$available_processors


# Select the compiler to use.
# We install the default version of GCC (GCC 9), as well as clang 8 and clang 10.
ARG sswitch_grpc=yes
ARG CC=gcc
ARG CXX=g++
ENV TZ=America/Los_Angeles
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

ENV GIT_SSL_NO_VERIFY=true

ENV PI_DEPS automake \
build-essential \
clang-8 \
clang-10 \
clang-format-8 \
g++ \
libboost-dev \
libboost-system-dev \
libboost-thread-dev \
libtool \
libtool-bin \
pkg-config \
libreadline-dev \
libpcap-dev \
libmicrohttpd-dev \
doxygen \
valgrind \
vim \
git-core \
python3 \
cmake \
python3-pip

ENV PI_RUNTIME_DEPS libboost-system1.71.0 \
libboost-thread1.71.0 \
libpcap0.8 \
python3 \
python-is-python3

RUN apt-get update && apt-get install -y --no-install-recommends $PI_DEPS $PI_RUNTIME_DEPS

COPY --from=grpc /usr/local/lib/lib*grpc*.so* \
/usr/local/lib/libabsl*.so* \
/usr/local/lib/libgpr*.so* \
/usr/local/lib/libupb*.so* \
/usr/local/lib/libre2*.so* \
/usr/local/lib/libaddress_sorting*.so* \
/usr/local/lib/libssl*.so* \
/usr/local/lib/libcrypto*.so* \
./

ENV LD_LIBRARY_PATH=/usr/local/lib

RUN apt-get install -y cmake g++ git automake libtool libgc-dev bison flex \
libfl-dev libgmp-dev libboost-dev libboost-iostreams-dev \
libboost-graph-dev llvm pkg-config python3 python3-pip \
tcpdump

RUN pip3 install ipaddr scapy ply

WORKDIR /

RUN git clone https://github.com/p4lang/PI
WORKDIR /PI/
RUN git submodule update --init --recursive
RUN apt-get update && \
./autogen.sh && \
./configure --enable-Werror --with-bmv2 --with-proto --with-cli --with-internal-rpc --with-sysrepo && \
./proto/sysrepo/install_yangs.sh && \
make && \
make install

WORKDIR /

RUN pip install jinja2

RUN git clone --recursive https://github.com/marian-pritsak/p4c.git

RUN cd p4c && \
mkdir build && \
cd build && \
cmake .. && \
make -j4 && \
make install

ARG user
ARG uid
ARG guid
ARG hostname

ENV BUILD_HOSTNAME $hostname
ENV USER $user

RUN groupadd -f -r -g $guid g$user

RUN useradd $user -l -u $uid -g $guid -d /var/$user -m -s /bin/bash

RUN echo "$user ALL=(ALL) NOPASSWD:ALL" >>/etc/sudoers

USER $user
12 changes: 6 additions & 6 deletions dash-pipeline/dockerfiles/Dockerfile.p4c-bmv2
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,23 @@ FROM amd64/ubuntu:20.04
# Need python for "p4c" wrapper & gcc for the C preprocessor
RUN apt update && apt install -y python3 gcc

COPY --from=p4lang-p4c /usr/local/lib/* /usr/local/lib/
COPY --from=chrissommers/dash-p4c-bldr:221007 /usr/local/lib/* /usr/local/lib/

COPY --from=p4lang-p4c \
COPY --from=chrissommers/dash-p4c-bldr:221007 \
/usr/lib/x86_64-linux-gnu/libboost_*so* \
/usr/lib/x86_64-linux-gnu/libgc*so* \
/usr/lib/x86_64-linux-gnu/libisl*so* \
/usr/lib/x86_64-linux-gnu/libmpc*so* \
/usr/lib/x86_64-linux-gnu/libmpfr*so* \
/usr/lib/x86_64-linux-gnu/

COPY --from=p4lang-p4c /usr/lib/gcc/x86_64-linux-gnu /usr/lib/gcc/x86_64-linux-gnu/
COPY --from=p4lang-p4c /usr/bin/cpp /usr/bin/
COPY --from=p4lang-p4c /usr/local/share/p4c/ /usr/local/share/p4c/
COPY --from=chrissommers/dash-p4c-bldr:221007 /usr/lib/gcc/x86_64-linux-gnu /usr/lib/gcc/x86_64-linux-gnu/
COPY --from=chrissommers/dash-p4c-bldr:221007 /usr/bin/cpp /usr/bin/
COPY --from=chrissommers/dash-p4c-bldr:221007 /usr/local/share/p4c/ /usr/local/share/p4c/

WORKDIR /usr/local/bin

COPY --from=p4lang-p4c \
COPY --from=chrissommers/dash-p4c-bldr:221007 \
/usr/local/bin/p4c \
/usr/local/bin/p4c-bm2-ss \
/usr/local/bin/
Expand Down