Trying to clone/pull private addons repositories using https

[gabonog]

Hi everyone!
I am trying to implement a way to clone/pull private addons repositories via https securely. SSH is not an option because of technical limitations of the git server.

I tried something like this (and it worked) on repos.yaml:

./repo:
  defaults:
    depth: $DEPTH_DEFAULT
  remotes:
    origin: https://user:[email protected]/project/repo.git
  target: origin $ODOO_VERSION
  merges:
    - origin $ODOO_VERSION

But the problem here is to leave those credentials in plain text in the file. So, what I want to use is something similar to this (as suggested on point 3 here: #287 (comment)):

https://${USER}:${PASS}@git-server.tld/project/repo.git

But no idea how to implement that env vars and use them in repos.yaml. These are my failed attempts:

• In common.yaml: Add env vars in build/args.
• In common.yaml: Add env vars in environment.

Any suggestion to implement those env vars?
Thank you!

[ap-wtioit]

If your are talking about a development, i think the aggregate part is always called with setup-devel.yaml that does not extend common.yaml and therefore you will need to define USER and PASS in environment of setup-devel.yaml.

For prod.yaml and test.yaml setting them in common.yaml environment should work i think.

Also keep in mind git-aggregator is using Template.substitute, so i don't think ${USER} will work but $USER should.

[ap-wtioit]

To do build stuff in scaffolding projects place your scripts in odoo/custom/build.d, name them 099-my_script_adding_credentials to run before aggregate, 101-my_script_removing_credentials runs after aggregate. Keep mind to check environment variable AGGREGATE to have the correct behaviour for devel vs prod. Build stuff runs with root so give the created files correct permissions for odoo.

One more enhanced script that fiddles with repos.yaml and odoo users git settings can be found here: https://github.com/Tecnativa/doodba/blob/163ecb0f80b04605ed3b601bba441c7612f8564c/tests/scaffoldings/repo_merge/custom/build.d/099-git_merge_no_ff

All build scripts run in one RUN command so only one layer with the final result should be created, if you are careful that should not expose any secrets.

[gabonog]

Sounds good this last approach, but I can't make it work. So, to sum up:

File: odoo/custom/src/repos.yaml

# [...]

./repo:
  defaults:
    depth: $DEPTH_DEFAULT
  remotes:
    origin: https://[email protected]/project/repo.git
  target: origin $ODOO_VERSION
  merges:
    - origin $ODOO_VERSION

# [...]

File: odoo/custom/build.d/099-repos-add-auth

#!/bin/bash
set -e

HARDCODED_AUTH="token"

sed -i 's/\$GIT_AUTH/'"$HARDCODED_AUTH"'/g' /opt/odoo/custom/src/repos.yaml

I am trying the simplest option but, when building, it is like the file doesn't exist, showing KeyError: 'GIT_AUTH' while running opt/odoo/common/build.d/100-repos-aggregate.
Am I missing something?

Once this work, somehow I have to obtain the value from a --build-arg given on docker build command.

Again, thank you for the guidance.

[ap-wtioit]

does your odoo/custom/build.d/099-repos-add-auth file have execute permissions? chmod +x odoo/custom/build.d/099-repos-add-auth

[gabonog]

So embarrasing... It was execute permissions. Thank you!

[gabonog]

Finally my workaround is (based on #614 (reply in thread)):

• Create those 2 files but changing HARDCODED_AUTH="token" to HARDCODED_AUTH=$PIPELINE_AUTH in odoo/custom/build.d/099-repos-add-auth.
• In the pipeline, before building the image, I run a sed to replace PIPELINE_AUTH in order have the value when running the script.

Thank you @ap-wtioit for helping me so much.