Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caddy and other issues when deploying with source code #5252

Closed
s0214lx opened this issue Feb 10, 2025 · 25 comments
Closed

Caddy and other issues when deploying with source code #5252

s0214lx opened this issue Feb 10, 2025 · 25 comments

Comments

@s0214lx
Copy link

s0214lx commented Feb 10, 2025

Is there a problem with caddyfile when using source code self-hosting deployment? I modified it to the following to execute "sudo caddy run --config Caddyfile" normally. My caddy version is 2.9

{$BASE_URL}:80 {
#listen {$ADDRESS}
reverse_proxy /ws/* http://lsp:3001
# reverse_proxy /ws_mp/* http://multiplayer:3002
# reverse_proxy /api/srch/* http://windmill_indexer:8001
reverse_proxy /* http://windmill_server:8000
# tls /certs/cert.pem /certs/key.pem
}

After all preparations are ready, I visited http://localhost/ in the browser and there was no response, and an error was reported in the terminal。

2025-02-10T07:29:04.253180Z ERROR request: windmill-api/src/tracing_init.rs:37: response latency=0 status=404 method=GET uri=/ traceId="b934ff26-0c5b-4b83-9860-4da8a6b45b03"

Image

@HugoCasa
Copy link
Contributor

If you're using normal caddy you should remove the layer4 part. Apart from that it should work correctly.
Make sure to set the correct environment variable for base url.
Also make sure to build the frontend before starting/building with cargo using the appropriate features (embedding,parquet,openidconnect,jemalloc,deno_core,license,http_trigger,zip,oauth2,dind,php,mysql,mssql,bigquery,oracledb,postgres_trigger,websocket,python,smtp,csharp,static_frontend,rust)

@s0214lx
Copy link
Author

s0214lx commented Feb 11, 2025

If you're using normal caddy you should remove the layer4 part. Apart from that it should work correctly. Make sure to set the correct environment variable for base url. Also make sure to build the frontend before starting/building with cargo using the appropriate features (embedding,parquet,openidconnect,jemalloc,deno_core,license,http_trigger,zip,oauth2,dind,php,mysql,mssql,bigquery,oracledb,postgres_trigger,websocket,python,smtp,csharp,static_frontend,rust)

Thank you for your answer, but when I delete the layer4 part and execute directly, I still get an error.
This is the content of the caddyfile configuration file:

{$BASE_URL} {
        bind {$ADDRESS}
        reverse_proxy /ws/* http://lsp:3001
        # reverse_proxy /ws_mp/* http://multiplayer:3002
        # reverse_proxy /api/srch/* http://windmill_indexer:8001
        reverse_proxy /* http://windmill_server:8000
        # tls /certs/cert.pem /certs/key.pem
}

and I get the following error:
2025/02/11 01:31:04.552 ERROR http.log.error dial tcp: lookup windmill_server on 127.0.0.53:53: server misbehaving {"request": {"remote_ip": "127.0.0.1", "remote_port": "43694", "client_ip": "127.0.0.1", "proto": "HTTP/1.1", "method": "GET", "host": "localhost", "uri": "/", "headers": {"Upgrade-Insecure-Requests": ["1"], "Connection": ["keep-alive"], "Sec-Fetch-Site": ["none"], "Sec-Fetch-Dest": ["document"], "Sec-Fetch-Mode": ["navigate"], "Sec-Fetch-User": ["?1"], "User-Agent": ["Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:135.0) Gecko/20100101 Firefox/135.0"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"], "Priority": ["u=0, i"], "Accept-Language": ["zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2"], "Accept-Encoding": ["gzip, deflate, br, zstd"]}}, "duration": 1.009529318, "status": 502, "err_id": "za6zpb61x", "err_trace": "reverseproxy.statusError (reverseproxy.go:1373)"}

@s0214lx
Copy link
Author

s0214lx commented Feb 11, 2025

如果您使用的是普通的 caddy,则应删除 layer4 部分。除此之外,它应该可以正常工作。 确保为基本 URL 设置正确的环境变量。 此外,请确保在启动/使用 cargo 构建之前使用适当的功能构建前端(embedding、parquet、openidconnect、jemalloc、deno_core、license、http_trigger、zip、oauth2、dind、php、mysql、mssql、bigquery、oracledb、postgres_trigger、websocket、python、smtp、csharp、static_frontend、rust)

I tried again, deleted the layer4 part in the caddyfile file, started it, and then visited http://localhost/ in the browser, and the error was as follows:
2025/02/11 04:27:47.487 ERROR http.log.error dial tcp: lookup windmill_server: i/o timeout {"request": {"remote_ip": "127.0.0.1", "remote_port": "59452", "client_ip": "127.0.0.1", "proto": "HTTP/1.1", "method": "GET", "host": "localhost", "uri": "/", "headers": {"Sec-Fetch-User": ["?1"], "Priority": ["u=0, i"], "Sec-Fetch-Dest": ["document"], "Accept-Encoding": ["gzip, deflate, br, zstd"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8"], "Accept-Language": ["zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2"], "Connection": ["keep-alive"], "Sec-Fetch-Site": ["none"], "User-Agent": ["Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0"], "Sec-Fetch-Mode": ["navigate"], "Upgrade-Insecure-Requests": ["1"]}}, "duration": 3.002232109, "status": 502, "err_id": "dufnex8kh", "err_trace": "reverseproxy.statusError (reverseproxy.go:1373)"}

@HugoCasa
Copy link
Contributor

As you're not using the docker compose, the caddyfile does not proxy to the correct endpoints.
instead of windmill_server:8000 it should be localhost:8000

@s0214lx
Copy link
Author

s0214lx commented Feb 11, 2025

As you're not using the docker compose, the caddyfile does not proxy to the correct endpoints. instead of windmill_server:8000 it should be localhost:8000

Thank you, I made the changes according to your suggestion, the content of caddyfile is as follows, and there is no problem in executing sudo caddy run --config Caddyfile, but when I visit http://localhost, I still get an error:
ERROR request: windmill-api/src/tracing_init.rs:37: response latency=0 status=404 method=GET uri=/ traceId="c16b2b63-b5bb-4556-baaa-bea5627bfe4a"

:80 {
        bind localhost
        reverse_proxy /ws/* http://lsp:3001
        # reverse_proxy /ws_mp/* http://multiplayer:3002
        # reverse_proxy /api/srch/* http://windmill_indexer:8001
        reverse_proxy /* http://localhost:8000
        # tls /certs/cert.pem /certs/key.pem
}

@HugoCasa
Copy link
Contributor

you need to build the frontend with NODE_OPTIONS="--max-old-space-size=8192" npm run build

@s0214lx
Copy link
Author

s0214lx commented Feb 11, 2025

NODE_OPTIONS="--max-old-space-size=8192" npm run build

Image
I have executed npm run build, this is the build directory created

@HugoCasa
Copy link
Contributor

now run cargo run and make sure to have the right feature flags (including static_frontend) and it should work

@s0214lx
Copy link
Author

s0214lx commented Feb 11, 2025

now run cargo run and make sure to have the right feature flags (including static_frontend) and it should work

This is the log of my cargo execution in the backend directory. It looks fine, but when I visit http://localhost, I get a 404 error.

sun@sun-virtual-machine:~/windmill-main/backend$ env DATABASE_URL="postgres://postgres:123456@localhost:5432/windmill" RUST_LOG=info cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.23s
     Running `target/debug/windmill`
Connecting to database...
OTEL settings loaded: tracing (false), logs (false), metrics (false), endpoint (None), headers defined: (false)
2025-02-11T09:04:59.074872Z  INFO windmill-api/src/db.rs:71: Acquiring global PG lock for potential migration with pid: Some(4183)
2025-02-11T09:04:59.076128Z  INFO windmill-api/src/db.rs:91: Acquired global PG lock
2025-02-11T09:04:59.078697Z  INFO windmill-api/src/db.rs:106: Releasing PG lock
2025-02-11T09:04:59.079243Z  INFO windmill-api/src/db.rs:114: Released PG lock
2025-02-11T09:04:59.323961Z  INFO src/main.rs:416: 
##############################
Windmill Community Edition unknown-version
##############################
2025-02-11T09:04:59.324075Z  INFO src/main.rs:972: config: PATH: /home/sun/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/sun/nsjail, HOME: /home/sun
2025-02-11T09:04:59.332605Z  INFO windmill-common/src/worker.rs:708: Loading config from WORKER_GROUP: default
2025-02-11T09:04:59.334045Z  INFO src/monitor.rs:1443: Reloading worker config...
2025-02-11T09:04:59.340737Z  INFO windmill-common/src/worker.rs:307: Loaded setting custom_tags, common: ["chromium"], per-workspace: {}
2025-02-11T09:04:59.342680Z  WARN windmill-common/src/server.rs:81: SMTP not configured
2025-02-11T09:04:59.342725Z  INFO src/monitor.rs:1389: Reloading smtp config...
2025-02-11T09:04:59.343906Z  INFO src/monitor.rs:1151: Loaded saml_metadata setting to None
2025-02-11T09:04:59.344293Z  INFO src/monitor.rs:1151: Loaded scim_token setting to None
2025-02-11T09:04:59.344795Z  INFO src/monitor.rs:1151: Loaded job_default_timeout setting to None
2025-02-11T09:04:59.346183Z  INFO src/monitor.rs:1151: Loaded pip_extra_index_url setting to None
2025-02-11T09:04:59.347098Z  INFO src/monitor.rs:1151: Loaded pip_index_url setting to None
2025-02-11T09:04:59.347661Z  INFO src/monitor.rs:1151: Loaded npm_config_registry setting to None
2025-02-11T09:04:59.348415Z  INFO src/monitor.rs:1151: Loaded bunfig_install_scopes setting to None
2025-02-11T09:04:59.349041Z  INFO src/monitor.rs:1151: Loaded instance_python_version setting to None
2025-02-11T09:04:59.350038Z  INFO src/monitor.rs:1151: Loaded nuget_config setting to None
2025-02-11T09:04:59.351798Z  INFO src/monitor.rs:1400: Reloading smtp config...
2025-02-11T09:04:59.412639Z  INFO src/main.rs:965: Successfully connected to pg listen
2025-02-11T09:04:59.412830Z  INFO windmill-api/src/lib.rs:265: SMTP server not started because email domain is not set
2025-02-11T09:04:59.422142Z  INFO windmill-api/src/lib.rs:564: server started on port=8000 and addr=0.0.0.0 instance=U90TB
2025-02-11T09:05:00.399399Z  INFO src/main.rs:1060: Starting 1 workers and SLEEP_QUEUE=50ms
2025-02-11T09:05:00.408924Z  INFO windmill-worker/src/worker.rs:1099: listening for jobs, WORKER_GROUP: default, config: WorkerConfig { worker_tags: ["ansible", "bash", "bun", "csharp", "deno", "dependency", "flow", "go", "hub", "other", "php", "powershell", "python3", "rust"], priority_tags_sorted: [PriorityTags { priority: 0, tags: ["ansible", "bash", "bun", "csharp", "deno", "dependency", "flow", "go", "hub", "other", "php", "powershell", "python3", "rust"] }], dedicated_worker: None, init_bash: None, cache_clear: None, additional_python_paths: None, pip_local_dependencies: None, env_vars: "" } worker=wk-default-machine-AMCA9 hostname=sun-virtual-machine
2025-02-11T09:05:00.415856Z  INFO windmill-worker/src/worker.rs:1263: ping update, memory: container=20MB, windmill=0MB worker=wk-default-machine-AMCA9 hostname=sun-virtual-machine
2025-02-11T09:05:06.456095Z  INFO windmill-worker/src/worker.rs:1263: ping update, memory: container=20MB, windmill=0MB worker=wk-default-machine-AMCA9 hostname=sun-virtual-machine

@s0214lx
Copy link
Author

s0214lx commented Feb 11, 2025

This is always the error.
ERROR request: windmill-api/src/tracing_init.rs:37: response latency=0 status=404 method=GET uri=/ traceId="c16b2b63-b5bb-4556-baaa-bea5627bfe4a"

@s0214lx
Copy link
Author

s0214lx commented Feb 11, 2025

now run cargo run and make sure to have the right feature flags (including static_frontend) and it should work

What does 'right feature flags' refer to

@HugoCasa
Copy link
Contributor

cargo run --features embedding,parquet,openidconnect,jemalloc,deno_core,license,http_trigger,zip,oauth2,dind,php,mysql,mssql,bigquery,oracledb,postgres_trigger,websocket,python,smtp,csharp,static_frontend,rust

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

cargo run --features embedding,parquet,openidconnect,jemalloc,deno_core,license,http_trigger,zip,oauth2,dind,php,mysql,mssql,bigquery,oracledb,postgres_trigger,websocket,python,smtp,csharp,static_frontend,rust

Thank you very much, with your help I successfully started windmill, but when I wanted to create an account to override the default super administrator account, a problem occurred. The following is the log of cargo. I don't know what the permissions here refer to.

2025-02-12T04:59:59.446360Z ERROR job_postprocessing: windmill-queue/src/jobs.rs:518: job 0194f887-3a36-5a85-11f0-43c03d24728b in admins did not succeed: {"error":{"message":"error during execution of the script:\nIoErr: Permission denied (os error 13) @common.rs:563:54","name":"ExecutionErr"}} job_id=0194f887-3a36-5a85-11f0-43c03d24728b workspace_id=admins worker=wk-default-machine-pWT9T tag=deno language="deno" otel.name="job postprocessing" script_path="g/all/setup_app/Change_Account"

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

cargo run --features embedding,parquet,openidconnect,jemalloc,deno_core,license,http_trigger,zip,oauth2,dind,php,mysql,mssql,bigquery,oracledb,postgres_trigger,websocket,python,smtp,csharp,static_frontend,rust

I carefully checked the cargo run log and found three errors. I did not see any information about uv in the readme document. I installed uv according to the prompts, but the error still exists.

2025-02-12T05:33:28.448771Z ERROR windmill-worker/src/python_executor.rs:262: Cannot install python: Internal: Executable uv not found on worker. PATH: /home/sun/.bun/bin:/bin:/home/sun/.deno/bin:/home/sun/.cargo/bin:/home/sun/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/sun/nsjail @common.rs:923:16

2025-02-12T05:33:28.448867Z ERROR windmill-worker/src/python_executor.rs:236: worker_name: wk-default-machine-ehncO, w_id: , job_id: 00000000-0000-0000-0000-000000000000
 
                Error while getting python from uv, falling back to system python: InternalErrLoc { message: "Executable uv not found on worker. PATH: /home/sun/.bun/bin:/bin:/home/sun/.deno/bin:/home/sun/.cargo/bin:/home/sun/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/sun/nsjail", location: "common.rs:923:16" }

2025-02-12T05:33:28.448918Z ERROR windmill-worker/src/worker.rs:800: Cannot preinstall or find Instance Python version to worker: Internal: Executable uv not found on worker. PATH: /home/sun/.bun/bin:/bin:/home/sun/.deno/bin:/home/sun/.cargo/bin:/home/sun/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/sun/nsjail @common.rs:923:16 worker=wk-default-machine-ehncO hostname=sun-virtual-machine worker_dir=/tmp/windmill/wk-default-machine-ehncO

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

cargo run --features embedding,parquet,openidconnect,jemalloc,deno_core,license,http_trigger,zip,oauth2,dind,php,mysql,mssql,bigquery,oracledb,postgres_trigger,websocket,python,smtp,csharp,static_frontend,rust

When I log in with the default account, I cannot create a new account in the instance. It prompts me that this is not a feature of the open source version. I remember that I can create one when I use docker deployment.

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

@HugoCasa I really need your help, do you have time?

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

cargo run——功能嵌入、parquet、openidconnect、jemalloc、deno_core、license、http_trigger、zip、oauth2、dind、php、mysql、mssql、bigquery、oracledb、postgres_trigger、websocket、python、smtp、csharp、static_frontend、rust

非常感谢,在你的帮助下我成功启动了windmill,但是当我想创建一个账户覆盖默认的超级管理员账户时,出现了问题。下面是cargo的日志。不知道这里的权限指的是什么。

2025-02-12T04:59:59.446360Z ERROR job_postprocessing: windmill-queue/src/jobs.rs:518: job 0194f887-3a36-5a85-11f0-43c03d24728b in admins did not succeed: {"error":{"message":"error during execution of the script:\nIoErr: Permission denied (os error 13) @common.rs:563:54","name":"ExecutionErr"}} job_id=0194f887-3a36-5a85-11f0-43c03d24728b workspace_id=admins worker=wk-default-machine-pWT9T tag=deno language="deno" otel.name="job postprocessing" script_path="g/all/setup_app/Change_Account"

I tried to deploy again, and the error became as follows:
2025-02-12T07:48:42.928863Z ERROR job_postprocessing: windmill-queue/src/jobs.rs:518: job 0194f921-a4c8-a7d7-1b14-19a27ab508d2 in admins did not succeed: {"error":{"message":"User already exists: Internal: Not implemented in Windmill's Open Source repository @users_ee.rs:20:9","name":"Error","stack":"Error: User already exists: Internal: Not implemented in Windmill's Open Source repository @users_ee.rs:20:9\n at main (file:///tmp/windmill/wk-default-machine-416no/0194f921-a4c8-a7d7-1b14-19a27ab508d2/main.ts:19:11)\n at eventLoopTick (ext:core/01_core.js:177:7)\n at async run (file:///tmp/windmill/wk-default-machine-416no/0194f921-a4c8-a7d7-1b14-19a27ab508d2/wrapper.ts:23:20)\n at async file:///tmp/windmill/wk-default-machine-416no/0194f921-a4c8-a7d7-1b14-19a27ab508d2/wrapper.ts:29:5"}} job_id=0194f921-a4c8-a7d7-1b14-19a27ab508d2 workspace_id=admins worker=wk-default-machine-416no tag=deno language="deno" otel.name="job postprocessing" script_path="g/all/setup_app/Change_Account"

@HugoCasa
Copy link
Contributor

HugoCasa commented Feb 12, 2025

windmill depends on multiple runtimes that must be installed on the system, e.g. deno, uv, bun, etc.. and you need to specify the appropriate paths in env
i'm sorry i'm not able to help you further but i suggest looking at the Dockerfile or use the docker compose directly

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

windmill depends on multiple runtimes that must be installed on the system, e.g. deno, uv, bun, etc.. and you need to specify the appropriate paths in env i'm sorry i'm not able to help you further but i suggest looking at the Dockerfile or use the docker compose directly

OK, thank you. But I am confused why when I first logged in, I created an account to overwrite the default account, but the backend log indicated that the account already existed? And, after I logged in with the default account, I could not create a new account in the instance, and it prompted me that this was a non-open source version feature.

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

windmill 依赖于必须在系统上安装的多个运行时,例如 deno、uv、bun 等。并且您需要在 env 中指定适当的路径。 很抱歉,我无法进一步帮助您,但我建议查看 Dockerfile 或直接使用 docker compose

I have solved the runtime problems such as uv, but I can't create a user
2025-02-12T08:53:27.186581Z ERROR job_postprocessing: windmill-queue/src/jobs.rs:518: job 0194f95c-f7a8-bf81-72bb-8abcb757dd13 in admins did not succeed: {"error":{"message":"User already exists: Internal: Not implemented in Windmill's Open Source repository @users_ee.rs:20:9","name":"Error","stack":"Error: User already exists: Internal: Not implemented in Windmill's Open Source repository @users_ee.rs:20:9\n at main (file:///tmp/windmill/wk-default-machine-zVYHc/0194f95c-f7a8-bf81-72bb-8abcb757dd13/main.ts:19:11)\n at eventLoopTick (ext:core/01_core.js:177:7)\n at async run (file:///tmp/windmill/wk-default-machine-zVYHc/0194f95c-f7a8-bf81-72bb-8abcb757dd13/wrapper.ts:23:20)\n at async file:///tmp/windmill/wk-default-machine-zVYHc/0194f95c-f7a8-bf81-72bb-8abcb757dd13/wrapper.ts:29:5"}} job_id=0194f95c-f7a8-bf81-72bb-8abcb757dd13 workspace_id=admins worker=wk-default-machine-zVYHc tag=deno language="deno" otel.name="job postprocessing" script_path="g/all/setup_app/Change_Account"

@HugoCasa HugoCasa marked this as a duplicate of #5263 Feb 12, 2025
@HugoCasa HugoCasa marked this as a duplicate of #5275 Feb 12, 2025
@HugoCasa
Copy link
Contributor

because this part of the code is closed source
not sure what you're trying to achieve but probably using the binaries (which include part of the closed source code) would easier than trying to build from source: https://github.com/windmill-labs/windmill/releases
also i'll close the issue now, if you need more help try to ask the community on our discord

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

because this part of the code is closed source not sure what you're trying to achieve but probably using the binaries (which include part of the closed source code) would easier than trying to build from source: https://github.com/windmill-labs/windmill/releases also i'll close the issue now, if you need more help try to ask the community on our discord

I just want to know how to set the number of workers after the source code is deployed, just like modifying windmill_worker in the docker-compose file when using docker deployment

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

because this part of the code is closed source not sure what you're trying to achieve but probably using the binaries (which include part of the closed source code) would easier than trying to build from source: https://github.com/windmill-labs/windmill/releases also i'll close the issue now, if you need more help try to ask the community on our discord

because this part of the code is closed source not sure what you're trying to achieve but probably using the binaries (which include part of the closed source code) would easier than trying to build from source: https://github.com/windmill-labs/windmill/releases also i'll close the issue now, if you need more help try to ask the community on our discord

Does this mean I can't create new accounts in the instance unless I become an ee user?

@HugoCasa
Copy link
Contributor

  • to have more workers you need to start multiple times cargo run with MODE=worker
  • no you don't need ee you just need to use the provided docker image or binary

@s0214lx
Copy link
Author

s0214lx commented Feb 12, 2025

  • no you don't need ee you just need to use the provided docker image or binary

Thank you, but if I deploy using source code, do I need to be an EE to create a user in the instance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants