How to add experimental features to kn func CLI? #1551

aslom · 2023-02-14T14:48:19Z

aslom
Feb 14, 2023

Such as Wasm buildpacks that may not be officially supported?

lkingland · 2023-02-14T16:05:38Z

lkingland
Feb 14, 2023
Collaborator

We could build support for experimental features into the core logic of the system, hiding them behind a FUNC_ENABLE_X feature flag. But for providing custom templates which specify a custom buildpack, we could use the built-in repositories support, which would look something like the following:

$ func templates
LANGUAGE     TEMPLATE
go           cloudevents
go           http
node         cloudevents
node         http
python       cloudevents
python       http
quarkus      cloudevents
quarkus      http
rust         cloudevents
rust         http
springboot   cloudevents
springboot   http
typescript   cloudevents
typescript   http

$ func repositories add wasm https://github.com/aslom/func-wasm
Repository added

$ func templates
LANGUAGE     TEMPLATE
go           cloudevents
go           http
node         cloudevents
node         http 
python       cloudevents
python       http
quarkus      cloudevents
quarkus      http
rust         cloudevents
rust         http
rust         wasm/http    // <---  from the repo
springboot   cloudevents
springboot   http
typescript   cloudevents
typescript   http

$ mkdir f && cd $_
$ func create -l wasm/rust

The function created would have your example template and custom buildpacks, etc.

The structure of that repository would be something like:

github.com:/aslom/func-wasm
    └─ rust
        └─ http
            ├─ README.md
            ├─ handler.rs
            └─ manifest.yaml

The manifest.yaml is optional when providing your own templates, but this is where you would specify the custom buildpack.

The other way to use a custom template with custom buildpack is as-needed (no local copy):

func create -l rust --repository https://github.com/aslom/func-wasm

1 reply

aslom Feb 14, 2023
Author

Thank for detailed example. I will be taking that approach and see if I can make it work. It looks like it should be very easy for users to try

matejvasek · 2023-02-14T16:12:50Z

matejvasek
Feb 14, 2023
Collaborator

@aslom Would the project template differ from non-WASM project, or would it be identical (I mean function name package and signature)?

0 replies

matejvasek · 2023-02-14T16:13:40Z

matejvasek
Feb 14, 2023
Collaborator

If the project would be the same and only BPs differ the you might just add mention in the template README.md.
Just put few lines in https://github.com/knative/func/blob/main/templates/go/cloudevents/README.md that say WASM can be used by adding something something into func.yaml.

0 replies

aslom · 2023-02-14T16:18:30Z

aslom
Feb 14, 2023
Author

@matejvasek I think the project would be different as some libraries are not available for Wasm WASI today

4 replies

matejvasek Feb 14, 2023
Collaborator

So the function signature would be different? Is it necessary or wanted?

matejvasek Feb 14, 2023
Collaborator

I thought only deployment would be slightly different?

matejvasek Feb 14, 2023
Collaborator

I mean it would have different Start() method than let say https://github.com/knative/func/pull/1451/files#diff-bb8ad382f59edfae9297e446895a161036f39115189e67da9d9f377a684a8f82R25.

aslom Feb 28, 2023
Author

Yes as Wasm is every early the way to get HTTP APIs exposed is not easy, only subset of Go libraries can be used as multi-threading and garbage collector is not available.

aslom · 2023-02-28T14:37:52Z

aslom
Feb 28, 2023
Author

I am going to present today current status: in short I got buildpack to work and generate docker image that I could deploy. Still figuring func repository layout and what is needed to make func create to work

0 replies

aslom · 2023-03-06T21:07:40Z

aslom
Mar 6, 2023
Author

After learning buildpacks/builder/run/build/images etc. I finally managed to get func create Wasm-WAGI to work - it required rebuilding func with my docker image as trusted builder #1599

func repositories add wasm file:///Users/aslom/Documents/awsm/go/src/github.com/aslom/func-wasm/

aslom@m:~/Documents/awsm/go/src/github.com/aslom|⇒   mkdir f && cd $_
aslom@m:~/Documents/awsm/go/src/github.com/aslom/f|⇒  func create -l tinygo -t wasm/http-wagi
Created tinygo function in /Users/aslom/Documents/awsm/go/src/github.com/aslom/f

aslom@m:~/Documents/awsm/go/src/github.com/aslom/f|⇒  func build -r docker.io/aslom
🕒 Building function image

2023/03/06 15:46:51.461257 DEBUG:  Pulling image index.docker.io/aslom/tinygo-wasi-wagi-builder:latest
latest: Pulling from aslom/tinygo-wasi-wagi-builder
Digest: sha256:47883bcdd464e0563cc668954ab4cb4af5c39044ed6b2deeca662348fd226208
Status: Image is up to date for aslom/tinygo-wasi-wagi-builder:latest
2023/03/06 15:46:52.379286 DEBUG:  Selected run image aslom/tinygo-wasi-wagi-stack-run:v1
2023/03/06 15:46:52.379289 DEBUG:  Pulling image aslom/tinygo-wasi-wagi-stack-run:v1
v1: Pulling from aslom/tinygo-wasi-wagi-stack-run
Digest: sha256:062add37da1904d10778edd9b03afa5db598672d94ec9ed95f587907341640e4
Status: Image is up to date for aslom/tinygo-wasi-wagi-stack-run:v1
2023/03/06 15:46:52.999673 DEBUG:  Downloading buildpack from image: docker.io/aslom/tinygo-wasi-wagi-buildpack
2023/03/06 15:46:52.999678 DEBUG:  Pulling image docker.io/aslom/tinygo-wasi-wagi-buildpack
latest: Pulling from aslom/tinygo-wasi-wagi-buildpack
Digest: sha256:4715213f043359a59766bf6b7d50994992d59b1840535fe83e23ade64cfd0291
Status: Image is up to date for aslom/tinygo-wasi-wagi-buildpack:latest
2023/03/06 15:46:53.566374 DEBUG:  Adding buildpack aslom/tinygo-wasi-wagi version 0.0.1 to builder
2023/03/06 15:46:53.566378 DEBUG:  Setting custom order
2023/03/06 15:46:53.566381 DEBUG:  Creating builder with the following buildpacks:
2023/03/06 15:46:53.566384 DEBUG:  -> aslom/[email protected]
2023/03/06 15:46:53.566385 DEBUG:  -> aslom/[email protected]
2023/03/06 15:46:53.579446 DEBUG:  buildpack aslom/[email protected] already exists on builder and will be overwritten
  - existing diffID: sha256:7f7dc3624aa6859e6545b2293028fd6c1ce2d3e366accab5f363483c0543a0f9
  - new diffID: sha256:1ccffa5cd2dfbc70d3c787da19d489ef9fcf1d3733b2372b3a940cb6dccb1975
2023/03/06 15:46:53.579485 DEBUG:  Adding buildpack aslom/[email protected] (diffID=sha256:1ccffa5cd2dfbc70d3c787da19d489ef9fcf1d3733b2372b3a940cb6dccb1975)
2023/03/06 15:46:53.990283 DEBUG:  Pulling image quay.io/boson/lifecycle:0.13.2
0.13.2: Pulling from boson/lifecycle
Digest: sha256:b4493261d2be3e0aad701851f7a74d8ccc4dde18901d48469b77605da6fbfd65
Status: Image is up to date for quay.io/boson/lifecycle:0.13.2
2023/03/06 15:46:54.962658 DEBUG:  Using build cache volume pack-cache-aslom_f_latest-2d9a4bae2b91.build
2023/03/06 15:46:54.962695 INFO:   ===> ANALYZING
2023/03/06 15:46:54.962742 DEBUG:  Running the analyzer on OS linux with:
2023/03/06 15:46:54.962743 DEBUG:  Container Settings:
2023/03/06 15:46:54.962756 DEBUG:    Args: /cnb/lifecycle/analyzer -gid 0 -daemon -stack /layers/stack.toml -run-image aslom/tinygo-wasi-wagi-stack-run:v1 -launch-cache /launch-cache docker.io/aslom/f:latest
2023/03/06 15:46:54.962758 DEBUG:    System Envs: CNB_USER_ID=1000 CNB_GROUP_ID=1000 CNB_PLATFORM_API=0.9
2023/03/06 15:46:54.962759 DEBUG:    Image: quay.io/boson/lifecycle:0.13.2
2023/03/06 15:46:54.962759 DEBUG:    User: root
2023/03/06 15:46:54.962779 DEBUG:    Labels: map[author:pack]
2023/03/06 15:46:54.962780 DEBUG:  Host Settings:
2023/03/06 15:46:54.962781 DEBUG:    Binds: /var/run/docker.sock:/var/run/docker.sock pack-cache-aslom_f_latest-2d9a4bae2b91.launch:/launch-cache pack-layers-kcqksacvev:/layers pack-app-wswgrxkwma:/workspace
2023/03/06 15:46:54.962782 DEBUG:    Network Mode:
[analyzer] ERROR: failed to : set platform API: platform API version '0.9' is incompatible with the lifecycle

Error: failed to build the function: executing lifecycle. This may be the result of using an untrusted builder: failed with status code: 11

aslom@m:~/Documents/awsm/go/src/github.com/aslom/f|⇒  time ~/Documents/awsm/go/src/knative.dev/func/func build -v -r docker.io/aslom
Building function image
Pulling image index.docker.io/aslom/tinygo-wasi-wagi-builder:latest
latest: Pulling from aslom/tinygo-wasi-wagi-builder
Digest: sha256:47883bcdd464e0563cc668954ab4cb4af5c39044ed6b2deeca662348fd226208
Status: Image is up to date for aslom/tinygo-wasi-wagi-builder:latest
Selected run image aslom/tinygo-wasi-wagi-stack-run:v1
Pulling image aslom/tinygo-wasi-wagi-stack-run:v1
v1: Pulling from aslom/tinygo-wasi-wagi-stack-run
Digest: sha256:062add37da1904d10778edd9b03afa5db598672d94ec9ed95f587907341640e4
Status: Image is up to date for aslom/tinygo-wasi-wagi-stack-run:v1
Downloading buildpack from image: docker.io/aslom/tinygo-wasi-wagi-buildpack
Pulling image docker.io/aslom/tinygo-wasi-wagi-buildpack
latest: Pulling from aslom/tinygo-wasi-wagi-buildpack
Digest: sha256:4715213f043359a59766bf6b7d50994992d59b1840535fe83e23ade64cfd0291
Status: Image is up to date for aslom/tinygo-wasi-wagi-buildpack:latest
Adding buildpack aslom/tinygo-wasi-wagi version 0.0.1 to builder
Setting custom order
Creating builder with the following buildpacks:
-> aslom/[email protected]
-> aslom/[email protected]
buildpack aslom/[email protected] already exists on builder and will be overwritten
  - existing diffID: sha256:7f7dc3624aa6859e6545b2293028fd6c1ce2d3e366accab5f363483c0543a0f9
  - new diffID: sha256:1ccffa5cd2dfbc70d3c787da19d489ef9fcf1d3733b2372b3a940cb6dccb1975
Adding buildpack aslom/[email protected] (diffID=sha256:1ccffa5cd2dfbc70d3c787da19d489ef9fcf1d3733b2372b3a940cb6dccb1975)
Using build cache volume pack-cache-aslom_f_latest-2d9a4bae2b91.build
Running the creator on OS linux with:
Container Settings:
  Args: /cnb/lifecycle/creator -daemon -launch-cache /launch-cache -log-level debug -app /workspace -cache-dir /cache -run-image aslom/tinygo-wasi-wagi-stack-run:v1 -gid 0 docker.io/aslom/f:latest
  System Envs: CNB_PLATFORM_API=0.10
  Image: pack.local/builder/78667461657365717a61:latest
  User: root
  Labels: map[author:pack]
Host Settings:
  Binds: pack-cache-aslom_f_latest-2d9a4bae2b91.build:/cache /var/run/docker.sock:/var/run/docker.sock pack-cache-aslom_f_latest-2d9a4bae2b91.launch:/launch-cache pack-layers-rbetzeggta:/layers pack-app-csizekgpqj:/workspace
  Network Mode:
===> ANALYZING
Analyzing image "0e01e96012efb9a4ac874ed40bc11dc163d02857ad56899b26429c579ca8e1dd"
Restoring data for SBOM from previous image
Retrieving previous image SBOM layer for "sha256:4b7f88a638b4072e12b135182cf8f23171ad60879247870d3789ab12dc1fd303"
Analyzing image "637118414ebdbe2d45f98c162881a81a1b540786c73006d95033f5dac4525ec6"
===> DETECTING
======== Results ========
pass: aslom/[email protected]
Resolving plan... (try #1)
aslom/tinygo-wasi-wagi 0.0.1
===> RESTORING
Reading buildpack directory: /layers/aslom_tinygo-wasi-wagi
Restoring metadata for "aslom/tinygo-wasi-wagi:wagi" from app image
Writing layer metadata for "aslom/tinygo-wasi-wagi:wagi"
Reading buildpack directory: /layers/aslom_tinygo-wasi-wagi
Reading buildpack directory item: wagi.toml
===> BUILDING
Starting build
Running build for buildpack aslom/[email protected]
Looking up buildpack
Finding plan
Creating plan directory
Preparing paths
Running build command
--> Knative Functions Wasm TinyGo Buildpack
build_dir=/workspace bp_dir=/cnb/buildpacks/aslom_tinygo-wasi-wagi/0.0.1 layersdir=/layers/aslom_tinygo-wasi-wagi wagilayer=/layers/aslom_tinygo-wasi-wagi/wagi
--> Compiling main.go using tinygo
LLVM triple:       wasm32-unknown-wasi
GOOS:              linux
GOARCH:            arm
build tags:        tinygo.wasm wasi runtime_memhash_leveldb tinygo math_big_pure_go gc.conservative scheduler.asyncify serial.none
garbage collector: conservative
scheduler:         asyncify
cached GOROOT:     /home/tinygo/.cache/tinygo/goroot-8bed305047b96c9453a373a4b3dd9119242a53cae3f988e952c7161df043837a
--> Check wagi
total 30372
-rwxr-xr-x 1 root root 31100040 Feb  2 18:01 wagi
--> cat /layers/aslom_tinygo-wasi-wagi/launch.toml
[[processes]]
type = "web"
command = ["/usr/local/bin/wagi"]
args = ["-l", "0.0.0.0:8080", "-c", "/workspace/modules.toml"]
default = true

[[processes]]
type = "check"
command = ["ls"]
args = ["-l", "/usr/local/bin/"]


Processing layers
Updating environment
Reading output files
Updating buildpack processes
Updating process list
Finished running build for buildpack aslom/[email protected]
Copying SBOM files
Creating SBOM files for legacy BOM
Listing processes
Finished build
===> EXPORTING
Reusing layers from image with id '0e01e96012efb9a4ac874ed40bc11dc163d02857ad56899b26429c579ca8e1dd'
Reading buildpack directory: /layers/aslom_tinygo-wasi-wagi
Reading buildpack directory item: launch.toml
Reading buildpack directory item: wagi
Reading buildpack directory item: wagi.toml
Processing buildpack directory: /layers/aslom_tinygo-wasi-wagi
Processing launch layer: /layers/aslom_tinygo-wasi-wagi/wagi
Reusing tarball for layer "aslom/tinygo-wasi-wagi:wagi" with SHA: sha256:b6eed13c767392445b1f9f39b566a47f646d019f11c4d3f7b1c15979951e62de
Reusing layer 'aslom/tinygo-wasi-wagi:wagi'
Layer 'aslom/tinygo-wasi-wagi:wagi' SHA: sha256:b6eed13c767392445b1f9f39b566a47f646d019f11c4d3f7b1c15979951e62de
Found SBOM of type launch for at /layers/sbom/launch
Reusing tarball for layer "launch.sbom" with SHA: sha256:4b7f88a638b4072e12b135182cf8f23171ad60879247870d3789ab12dc1fd303
Reusing layer 'launch.sbom'
Layer 'launch.sbom' SHA: sha256:4b7f88a638b4072e12b135182cf8f23171ad60879247870d3789ab12dc1fd303
Layer 'slice-1' SHA: sha256:6160b9fe61aa63572c7b799639ac852fbc575a5774fb08114b1adb8efe3c8341
Adding 1/1 app layer(s)
Reusing tarball for layer "launcher" with SHA: sha256:6c749df2cfd63eaf8d564c60026df199deccb92225c279b7e858e7cae4dfbe14
Reusing layer 'launcher'
Layer 'launcher' SHA: sha256:6c749df2cfd63eaf8d564c60026df199deccb92225c279b7e858e7cae4dfbe14
Reusing tarball for layer "config" with SHA: sha256:4b1d140ab0768f8a799c2d53bebdfb0861b3a252ae33b8d20244aa5125959106
Reusing layer 'config'
Layer 'config' SHA: sha256:4b1d140ab0768f8a799c2d53bebdfb0861b3a252ae33b8d20244aa5125959106
Reusing tarball for layer "process-types" with SHA: sha256:abbdd8c9ab0c70103f282e37af9852450b47e88100c182d2c2f650c61bd30a17
Reusing layer 'process-types'
Layer 'process-types' SHA: sha256:abbdd8c9ab0c70103f282e37af9852450b47e88100c182d2c2f650c61bd30a17
Adding label 'io.buildpacks.lifecycle.metadata'
Adding label 'io.buildpacks.build.metadata'
Adding label 'io.buildpacks.project.metadata'
Setting CNB_LAYERS_DIR=/layers
Setting CNB_APP_DIR=/workspace
Setting CNB_PLATFORM_API=0.10
Setting CNB_DEPRECATION_MODE=quiet
Prepending /cnb/process and /cnb/lifecycle to PATH
Setting WORKDIR: '/workspace'
Setting default process type 'web'
Setting ENTRYPOINT: '/cnb/process/web'
Saving docker.io/aslom/f:latest...
*** Images (529278b32095):
      docker.io/aslom/f:latest

*** Image ID: 529278b32095ad50cb7140b2be0fe754488eb189b11c527416362047519964c1
Reading buildpack directory: /layers/aslom_tinygo-wasi-wagi
Reading buildpack directory item: launch.toml
Reading buildpack directory item: wagi
Reading buildpack directory item: wagi.toml
🙌 Function image built: docker.io/aslom/f:latest
~/Documents/awsm/go/src/knative.dev/func/func build -v -r docker.io/aslom  0.28s user 0.31s system 4% cpu 13.946 total

aslom@m:~/Documents/awsm/go/src/github.com/aslom/f|⇒  docker run --rm -p 8080:8080 --entrypoint web docker.io/aslom/f:latest
No log_dir specified, using temporary directory /tmp/.tmpuwHyQb for logs
Ready: serving on http://0.0.0.0:8080

0 replies

aslom · 2023-03-06T21:50:04Z

aslom
Mar 6, 2023
Author

Instructions and source code:
https://github.com/aslom/func-wasm
https://github.com/aslom/knative-wasm-buildpacks

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to add experimental features to kn func CLI? #1551

{{title}}

Replies: 7 comments 5 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

How to add experimental features to kn func CLI? #1551

aslom Feb 14, 2023

Replies: 7 comments · 5 replies

lkingland Feb 14, 2023 Collaborator

aslom Feb 14, 2023 Author

matejvasek Feb 14, 2023 Collaborator

matejvasek Feb 14, 2023 Collaborator

aslom Feb 14, 2023 Author

matejvasek Feb 14, 2023 Collaborator

matejvasek Feb 14, 2023 Collaborator

matejvasek Feb 14, 2023 Collaborator

aslom Feb 28, 2023 Author

aslom Feb 28, 2023 Author

aslom Mar 6, 2023 Author

aslom Mar 6, 2023 Author

aslom
Feb 14, 2023

Replies: 7 comments 5 replies

lkingland
Feb 14, 2023
Collaborator

aslom Feb 14, 2023
Author

matejvasek
Feb 14, 2023
Collaborator

matejvasek
Feb 14, 2023
Collaborator

aslom
Feb 14, 2023
Author

matejvasek Feb 14, 2023
Collaborator

matejvasek Feb 14, 2023
Collaborator

matejvasek Feb 14, 2023
Collaborator

aslom Feb 28, 2023
Author

aslom
Feb 28, 2023
Author

aslom
Mar 6, 2023
Author

aslom
Mar 6, 2023
Author