feat: scaffolding ca certs

[lkingland]

• 🎁 Scaffolded functions include root CA certs
• 🎁 adds make target for updating embedded certs

All functions which utilize scaffolding (Go for now, but will work for all) will now include root CA certs, as provided by the curl project.

[codecov]

Codecov Report

Patch coverage: 56.17% and project coverage change: +0.98 🎉

Comparison is base (8a078ce) 62.59% compared to head (fb63f3e) 63.58%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1823      +/-   ##
==========================================
+ Coverage   62.59%   63.58%   +0.98%     
==========================================
  Files         107      107              
  Lines       13645    13729      +84     
==========================================
+ Hits         8541     8729     +188     
+ Misses       4298     4159     -139     
- Partials      806      841      +35     

Flag	Coverage Δ
e2e-test	35.25% <2.24%> (-0.21%)	⬇️
e2e-test-oncluster	31.22% <2.24%> (-0.21%)	⬇️
e2e-test-oncluster-runtime	26.67% <2.24%> (?)
e2e-test-runtime-go	25.62% <2.24%> (?)
e2e-test-runtime-node	26.57% <2.24%> (?)
e2e-test-runtime-python	26.57% <2.24%> (?)
e2e-test-runtime-quarkus	26.60% <2.24%> (?)
e2e-test-runtime-springboot	25.75% <2.24%> (?)
e2e-test-runtime-typescript	26.69% <2.24%> (?)
integration-tests	51.69% <56.17%> (+3.54%)	⬆️
unit-tests-macos-latest	49.56% <56.17%> (+0.05%)	⬆️
unit-tests-ubuntu-latest	50.32% <56.17%> (+0.02%)	⬆️
unit-tests-windows-latest	49.57% <56.17%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/scaffolding/scaffold.go	69.38% <0.00%> (-9.69%)	⬇️
pkg/filesystem/filesystem.go	63.38% <40.00%> (-0.86%)	⬇️
pkg/oci/containerize.go	60.74% <60.52%> (-0.63%)	⬇️
pkg/functions/repository.go	70.42% <100.00%> (+0.18%)	⬆️

... and 6 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[matejvasek]

On similar note like #1823 (comment).
I really do not like using $(CODE) as prerequisites -- it cannot detect file deletion (only addtition/modification).
Moreover go build, go test, golangci-lint are smart enough to know when/what re-compilation/re-test is required.
Similarly I don't see much value in these stamps, what I see is possible risk of not running recipe when is should be run.
I rather run recipe that takes 500ms all the time than risking that the recipe is not done when it really needs to be.

[lkingland]

Thanks for the review @matejvasek , I have added filepath.ToSlash() to the filesystem symlink test in place of the conditionals, and removed all but the minimum makefile updates for discussion in a separate PR.

[matejvasek]

/approve
/lgtm

[matejvasek]

The windows test are failing on the symlink despite filepath.ToSlash(), it makes no sense.

[lkingland]

The windows test are failing on the symlink despite filepath.ToSlash(), it makes no sense.

Shucks. I was hoping the ToSlash would do the trick. I really don't mind if they are a few bytes different, so long as they are both symlinks.

[matejvasek]

The windows test are failing on the symlink despite filepath.ToSlash(), it makes no sense.

Shucks. I was hoping the ToSlash would do the trick.

@lkingland I think I know where the issue might be.
The unit test in GH Action also regenerates zz file on Windows.
That is IMO not good because actual app do use zz file generated on Linux (because of exec perms).
This means the GH Action test is testing something that is not really happening with actual app.

Anyway... the generate/templates/main.go scrip should also fixup links here (the same way it fixes up filepaths already here ).

I really don't mind if they are a few bytes different, so long as they are both symlinks.

But these bytes are significant.

[matejvasek]

actual app do use zz file generated on Linux

Actually I am not 100% positive about this (but if we are not doing it we should be). I am only sure that we version file generated on Linux.

[matejvasek]

@lkingland tl;dr we should not suppress the tests, it really indicates that something is wrong

We need filepath.ToSlash() in:

• zip generator
• os fs
• fs test

After these 3 changes it should work. I tested it.
I know this is bug in existing code, not it this PR, sill we should fix it.

diff --git a/generate/templates/main.go b/generate/templates/main.go
index 45ec4848..f29ceb4f 100644
--- a/generate/templates/main.go
+++ b/generate/templates/main.go
@@ -80,7 +80,7 @@ func main() {
 			if err != nil {
 				return err
 			}
-			_, err = w.Write([]byte(symlinkTarget))
+			_, err = w.Write([]byte(filepath.ToSlash(symlinkTarget)))
 			return err
 		case info.Mode()&fs.ModeType == 0: // regular file
 			f, err := os.Open(path)
diff --git a/pkg/filesystem/filesystem.go b/pkg/filesystem/filesystem.go
index d43aad75..c72cf5ee 100644
--- a/pkg/filesystem/filesystem.go
+++ b/pkg/filesystem/filesystem.go
@@ -173,7 +173,11 @@ func (o osFilesystem) Stat(name string) (fs.FileInfo, error) {
 
 func (o osFilesystem) Readlink(link string) (string, error) {
 	link = filepath.FromSlash(link)
-	return os.Readlink(filepath.Join(o.root, link))
+	t, err := os.Readlink(filepath.Join(o.root, link))
+	if err != nil {
+		return "", err
+	}
+	return filepath.ToSlash(t), nil
 }
 
 // subFS exposes subdirectory of underlying FS, this is similar to `chroot`.

[matejvasek]

/approve
/lgtm

[knative-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lkingland, matejvasek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [lkingland,matejvasek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment