Unable to clone git repo in pub get due to missing SSH key

[gwendiagram]

Flutter allows Git repos to be referenced via flutter pub get (https://flutter.dev/docs/development/packages-and-plugins/using-packages) however, when sylph running on AWS tries to flutter pub get it fails due to a missing ssh key. AWS doesn't have a place to put the SSH key (only for CodeCommit) and CodeMagic doesn't pass the ssh key when it pushes to AWS.

Logs:

Starting Flutter app co.glean.app in debug mode...
Running "flutter pub get" in flutter_app...                     
Git error. Command: git clone --mirror [email protected]:{repo}

Cloning into bare repository '/home/device-farm/flutter/.pub-cache/git/cache/glean-audio-library-3a19cf8eeb19f3e3abf0bd839accc81263b56c07'...

ssh_askpass: exec(/usr/local/libexec/ssh-askpass): No such file or directory

Host key verification failed.

fatal: Could not read from remote repository.

Please make sure you have the correct access rights

and the repository exists.

pub get failed (server unavailable) -- attempting retry 1 in 1 second...

Is this something that could be fixed within sylph?

[mmcc007]

Are you asking about support for private dart repository?

[gwendiagram]

Thanks for the quick reply, appreciated!

We weren't keen on going down the private dart repository route as it would increase complexity. We were more interested in if sylph could handle ssh keys as AWS doesn't have a place that we could pass it in.

[mmcc007]

Are you using a private git repository in your pubspec.yaml? Please confirm how you are using ssh.

If understanding correctly, it should be a matter of duplicating your ssh config on the VM in Device Farm. This would involve uploading your private ssh key to the VM and configuring ssh on the VM.

[gwendiagram]

Yup, we are using a private git repository.

Oh, interesting! OK, marvellous. I'll have a look at how I can upload my private key to the Device Farm VM and let you know how it goes.

Thanks a bunch for your help!

[mmcc007]

Uploading to the VM has to be done by Sylph. This requires adding support for passing the path to the ssh key to config file, adding the ssh key file to the uploaded bundle, then configuring ssh on the VM using the ssh key.

[gwendiagram]

OK - do you suggest that we fork the repo or is this something that could be included in the main sylph codebase?

[mmcc007]

Yep, if you get it working, send a PR. Not a great fit for Sylph since appears not much call for this feature. But if PR works, may be good to have.

[gwendiagram]

Sound, cheers mate! We'll have a look.

[mmcc007]

Cool!

BTW: Sylph is using dummy ssh keys to get to a private fastlane match git server when building iOS debug ipa in CI (travis). Haven't looked at it in a while but may be possible to take a similar approach on Device Farm VM.

https://github.com/mmcc007/sylph/tree/master/lib/resources/dummy-ssh-keys
https://github.com/mmcc007/sylph/blob/master/lib/resources/script/local_utils.sh#L48

[tim-fdc]

Hi, maybe a stupid question. Why do you run flutter pub get in the AWS? Is it due to the test dependencies only?

[mmcc007]

@tim-fdc basically yes

[tim-fdc]

I see. Was just wondering if it is possible to avoid the flutter pub get at all. For us it is a bit of a pain at the moment, because we change to "local" depdencies (via path imports) before running the tests in AWS. Not a very scalable approach in particular since we have many internal packages.

[mmcc007]

@tim-fdc May not be fully understanding how you are using local packages (in tests or in app, or both). I think, by default, all local packages are uploaded (probably only need local test packages).

If you are using no local test packages, you could try forking sylph, remove the flutter pub get and see what happens.

[tim-fdc]

@mmcc007 We will try and come back with the result.

[mmcc007]

@gwendiagram btw: as a workaround, if you edit your pubspec.yaml to use 'path' to your private packages, Sylph will copy that package into bundle for upload to Device Farm.

Ultimately, the test running on device farm does not need non-test packages. So the ultimate fix is to create a test-only pubspec.yaml on the fly (#100)

This would likely fix the case for local and private non-test packages

@gwendiagram / @tim-fdc

[ESBA-salman]

We hit this same issue on Bitrise. We forked the repo and commented the lines here:

sylph/lib/resources/script/local_utils.sh

Line 53 in 8cbe88c

cat << EOF > ~/.ssh/config

Then used the following to setup sylph in the runner/CI
pub global activate --source git <git_url>