GitLab support request #168

filipproch · 2019-12-22T17:16:27Z

Expected Behaviour

Installation should complete successfully.

Current Behaviour

Installations ends with a lot of

Failed to connect to 127.0.0.1 port 31111: Connection refused

And the OpenFaaS Cloud doesn't work (tried multiple times, used ./scripts/reset.sh every time)

Steps to Reproduce (for bugs)

ofc-bootstrap apply -f init.yaml
wait...

Context

Trying to deploy OpenFaaS cloud on my cluster

Your Environment

faas-cli version ( full output ):

CLI:
 commit:  73004c23e5a4d3fdb7352f953247473477477a64
 version: 0.11.3

Docker version docker version (e.g. Docker 17.0.05 ):

Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea
 Built:             Wed Nov 13 07:22:34 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea
  Built:            Wed Nov 13 07:29:19 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Kubernetes version kubectl version:

Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T23:42:50Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.2", GitCommit:"c97fe5036ef3df2967d086711e6c0c405941e14b", GitTreeState:"clean", BuildDate:"2019-10-15T19:09:08Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}

Are you using faas-netes or the Operator?
Operating System and version (e.g. Linux, Windows, MacOS):
MacOS
Link to your project or a code example to reproduce issue:
I just cloned the ofc-bootstrap repo and edited init.yaml
Please also follow the troubleshooting guide and paste in any other diagnostic information you have:
Cluster
Digitalocean, freshly created cluster
Registry/SCM
GitLab EE, and GitLab Container Registry deployed on DO Droplet

The errors starts right here

Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Checking if OpenFaaS GW is up.
Calling the OpenFaaS server to validate the credentials...
Cannot connect to OpenFaaS on URL: http://127.0.0.1:31111. Get http://127.0.0.1:31111/system/functions: dial tcp 127.0.0.1:31111: connect: connection refused
Deploying: function-logs.
WARNING! Communication is not secure, please consider using HTTPS. Letsencrypt.org offers free SSL/TLS certificates.

Is OpenFaaS deployed? Do you need to specify the --gateway flag?
Put http://127.0.0.1:31111/system/functions: dial tcp 127.0.0.1:31111: connect: connection refused

Deploying: git-tar.
WARNING! Communication is not secure, please consider using HTTPS. Letsencrypt.org offers free SSL/TLS certificates.

Is OpenFaaS deployed? Do you need to specify the --gateway flag?
Put http://127.0.0.1:31111/system/functions: dial tcp 127.0.0.1:31111: connect: connection refused

Before everything seems okay.

The text was updated successfully, but these errors were encountered:

alexellis · 2019-12-22T17:28:29Z

Please put the full logs into a gist and link it here.

What GitLab version is this? What you mean mean by "on DigitalOcean"? Are you using DOKS or your own kubernetes installation?

alexellis · 2019-12-22T17:28:43Z

/set title: GitLab support request

alexellis · 2019-12-22T17:28:50Z

Cc @martindekov

filipproch · 2019-12-22T17:36:07Z

GitLab version: GitLab Enterprise Edition 12.6.0-ee
Gist: https://gist.github.com/filipproch/737f44708c6ae5e733d31925a5deeaf5

filipproch · 2019-12-22T17:37:26Z

DigitalOcean - using DOKS, created new Cluster from the Web UI (same as the "Video" example shows)

alexellis · 2019-12-22T17:41:50Z

Given the error messages it looks like you've run the reset or run the tool more than once. Why don't you try running it on a fresh cluster?

filipproch · 2019-12-22T17:43:46Z

yeah I run it again now (with a reset before), to generate the log file, the first time (still with error), it was on a fresh cluster
will try again then

alexellis · 2019-12-22T17:43:48Z

Did you run through the whole troubleshooting guide already to see what might have gone wrong? Did you ensure that your cluster than meets the minimum requirements for RAM and nodes?

https://docs.openfaas.com/deployment/troubleshooting/

https://docs.openfaas.com/openfaas-cloud/self-hosted/troubleshoot/

filipproch · 2019-12-22T18:06:37Z

The same run on a fresh cluster
https://gist.github.com/filipproch/a8026157bd54d642468a31cbfca092ea

Parameters are exactly the same as in demo video (3 nodes, 4GB 2vCPU)

I looked at the troubleshooting but I am not Kubernetes expert, just starting with it and interested in OpenFaaS and this script does a lot of stuff. From what I see, for some reason, OpenFaaS doesn't start.

Also, this error is right in the beginning but I checked that the credentials are alright.

Validating registry credentials fileerror with registry credentials file. Please ensure it has been created correctly

alexellis · 2019-12-22T19:05:57Z

/msg: slack

derek · 2019-12-22T19:05:59Z

--
Join Slack to connect with the community
https://docs.openfaas.com/community

alexellis · 2019-12-22T19:07:00Z

The guides are so to don't need to be a Kubernetes expert. Perhaps try the community cluster which we host or see if someone on Slack can help you.

blaisep · 2019-12-25T07:09:53Z

@filipproch I have to say, I have made almost every mistake possible with ofc-bootstrap and I am happy to help you out....as a newcomer to k8s, I found that the path to a working ofc-boostrap might appear long and twisty.
Feel free to ping me in the Slack channel.

alexellis · 2019-12-25T21:18:52Z

Most people tend to get ofc-bootstrap working when following a tutorial or the instructions provided in a short period of time. I know a few folks have struggled, but that's the exception to the rule.

It's the Christmas holidays, but I tried OFC bootstrap today and it worked as expected with a GitHub configuration.

@viveksyngh @martindekov @Waterdrips can any of you take a look at GitLab for OFC? You can install GitLab on a VM or via helm.

viveksyngh · 2019-12-26T08:43:58Z

From the provided logs .. looks like OpenFaaS installation has failed.

Release "openfaas" does not exist. Installing it now.
 Error: Could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: webhook.cert-manager.io/v1beta1: the server is currently unable to handle the request

and that is the main cause of errors.

filipproch · 2019-12-26T08:46:33Z

Hi there, thanks to everyone for offering help and looking into this, it seems its because of new Kubernetes 1.16, after going back to 1.15 it works as expected.

alexellis · 2019-12-26T08:48:50Z

I've tested OpenFaaS Cloud with Kubernetes 1.16 and it worked for me.

@viveksyngh could you reproduce the error? Try with a fake set of GitLab URLs if you are short on time.

filipproch · 2019-12-26T09:55:13Z

Ok sorry seems false alarm, I now tested with fresh cluster and changing just Kubernetes version and still the same error. It seems before it started to work only because as part of trying to get it to work, I did many many changes (out of pure desperation) and hard to tell, which one solved it.
Unfortunately don't have more time to debug it now, since I already wasted a ton, but if I get to it later, will try to better diagnose the problem.

alexellis · 2019-12-26T10:02:19Z

I installed with GitHub yesterday and it worked first time. Unfortunately you've chosen a bad time to need help since mostly everyone is on holiday. I've asked a few people to help you, just wait please, no need to "waste a tone of time"

filipproch · 2019-12-26T10:07:48Z

I installed with GitHub yesterday and it worked first time. Unfortunately you've chosen a bad time to need help since mostly everyone is on holiday. I've asked a few people to help you, just wait please, no need to "waste a tone of time"

Thanks, understand the holidays, will watch this thread, but for me, it's the exact opposite, since I used the free time to play with/try out new stuff like openfaas.

martindekov · 2019-12-26T14:08:07Z

I am extremely tight on time, I am off from the project till 6th. Currently working on mandatory personal projects not counting the family affairs. I apologize, but I won't be able to take a look/test the Issue until the end of the holidays.

viveksyngh · 2019-12-28T17:48:57Z

I tried installing OpenFaas Cloud with ofc-bootstrap on a kubernetes cluster with version 1.16 with some dummy gitlab configuration. All ofc services got deployed correctly on running. I could not test the end to end flow but the installation was smooth.

jyksnw · 2020-01-16T13:13:17Z

I have been working through getting OpenFaaS cloud running on Civo’s managed Kubernetes and I am also running into the issue with getting the error: Could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: webhook.cert-manager.io/v1beta1: the server is currently unable to handle the request Other than deploying to Civo (k3s with traefik not added) I am using GitHub as my scm and Cloudflare for DNS. I found the following bug/issue on cert-manager that looks to be similar in nature to this one cert-manager/cert-manager#2273 some reported that helm 2.14.0 worked but newer versions did not. I have yet to try this. For those that have tested and had it work what version of helm are you running?

Waterdrips · 2020-01-16T13:16:36Z

I have been working through getting OpenFaaS cloud running on Civo’s managed Kubernetes and I am also running into the issue with getting the error: Could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: webhook.cert-manager.io/v1beta1: the server is currently unable to handle the request Other than deploying to Civo (k3s with trafiek not added) I am using GitHub as my scm and Cloudflare for DNS. I found the following bug/issue on cert-manager that looks to be similar in nature to this one jetstack/cert-manager#2273 some reported that helm 2.14.0 worked but newer versions did not. I have yet to try this. For those that have tested and had it work what version of helm are you running?

I'm using 2.16 - had to install it manually as the package manager wasn't that up to date

Waterdrips · 2020-01-18T11:21:56Z

I think this issue is related to this Helm issue: helm/helm#6361

I have raised a PR to wait for cert-manager to be ready before we move on, previously this caused the openfaas components not to be installed at all

derek bot changed the title ~~Getting Failed to connect to 127.0.0.1 port 31111: Connection refused~~ GitLab support request Dec 22, 2019

Waterdrips mentioned this issue Jan 18, 2020

Change the order of checking cert-manager is ready before moving on #179

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitLab support request #168

GitLab support request #168

filipproch commented Dec 22, 2019

alexellis commented Dec 22, 2019

alexellis commented Dec 22, 2019

alexellis commented Dec 22, 2019

filipproch commented Dec 22, 2019

filipproch commented Dec 22, 2019

alexellis commented Dec 22, 2019

filipproch commented Dec 22, 2019

alexellis commented Dec 22, 2019

filipproch commented Dec 22, 2019

alexellis commented Dec 22, 2019

derek bot commented Dec 22, 2019

alexellis commented Dec 22, 2019

blaisep commented Dec 25, 2019 •

edited

Loading

alexellis commented Dec 25, 2019

viveksyngh commented Dec 26, 2019 •

edited

Loading

filipproch commented Dec 26, 2019

alexellis commented Dec 26, 2019

filipproch commented Dec 26, 2019

alexellis commented Dec 26, 2019

filipproch commented Dec 26, 2019

martindekov commented Dec 26, 2019

viveksyngh commented Dec 28, 2019

jyksnw commented Jan 16, 2020 •

edited

Loading

Waterdrips commented Jan 16, 2020

Waterdrips commented Jan 18, 2020

GitLab support request #168

GitLab support request #168

Comments

filipproch commented Dec 22, 2019

Expected Behaviour

Current Behaviour

Steps to Reproduce (for bugs)

Context

Your Environment

alexellis commented Dec 22, 2019

alexellis commented Dec 22, 2019

alexellis commented Dec 22, 2019

filipproch commented Dec 22, 2019

filipproch commented Dec 22, 2019

alexellis commented Dec 22, 2019

filipproch commented Dec 22, 2019

alexellis commented Dec 22, 2019

filipproch commented Dec 22, 2019

alexellis commented Dec 22, 2019

derek bot commented Dec 22, 2019

alexellis commented Dec 22, 2019

blaisep commented Dec 25, 2019 • edited Loading

alexellis commented Dec 25, 2019

viveksyngh commented Dec 26, 2019 • edited Loading

filipproch commented Dec 26, 2019

alexellis commented Dec 26, 2019

filipproch commented Dec 26, 2019

alexellis commented Dec 26, 2019

filipproch commented Dec 26, 2019

martindekov commented Dec 26, 2019

viveksyngh commented Dec 28, 2019

jyksnw commented Jan 16, 2020 • edited Loading

Waterdrips commented Jan 16, 2020

Waterdrips commented Jan 18, 2020

blaisep commented Dec 25, 2019 •

edited

Loading

viveksyngh commented Dec 26, 2019 •

edited

Loading

jyksnw commented Jan 16, 2020 •

edited

Loading