refactor: migrate install/update orchestration from Bash to Python

[moham96]

Migrate the install/update path from Bash to Python

TL;DR

Replaces the bash orchestrator (install.sh, update.sh, restart.sh,
status.sh, uninstall.sh, hiddify_installer.sh, common/utils.sh,
common/jinja.py, the per-module install.sh/run.sh files, and the
common/run.sh.j2 firewall/timezone hook) with a python package
(hiddify_manager/) driven by ./init.sh install|update|upgrade|status|menu.

Repo .sh count: 70 → ~17 (12 real + 5 thin shims). Test count:
18 → 187. 269 commits, 363 files changed, net −30k / +8k
lines. Every remaining .sh is either an entrypoint, a curl|sh
bootstrap, a vendored upstream (google-bbr), a CI helper, or a 5-line
python shim kept so commander.py's sudoers paths don't churn.

Why it's worth doing

The bash orchestrator had real, silent, in-production bugs that fell
out the moment they touched python's typed environment + tests. None of
these were intentional in the migration; they showed up because porting
forces you to actually read every code path.

Bugs found and fixed along the way

Symptom on a running box	Root cause
Nightly cert renewal hadn't been running	common/daily_actions.sh called acme.sh/run.sh, deleted two refactors ago. Cron logged "file not found" every night to a place nobody looked.
Docker bootstrap broken	docker-init.sh called install.sh + status.sh, both deleted. Any new docker container would have failed on bash: install.sh: No such file.
/usr/bin/hiddify doesn't run the menu	Symlinked to menu.sh, deleted long ago. Operators got "command not found."
Singbox crashloop on every install	Templates emit {users:[...,]} with trailing commas (legal json5). The legacy common/jinja.py re-parsed .json output via json5 and re-emitted strict JSON — my initial port skipped that step, hiddify-core fatal'd parsing.
Singbox crashloop on every install, take 2	Some xray/singbox templates exec() ls ssl/*.crt and bake the listing into JSON. Render was running before cert generation, so the listing was ls: cannot access ..., which the consumer then tried to open as a file.
WARP came up with no IPv4 address	wgcf changed its Address = X, Y format to single-line comma-separated. The legacy regex (Address = ...:...) commented out the whole line when v6 was disabled, stripping the v4 with it. WARP "worked" (interface UP, handshake OK) but the kernel had no source v4 to send from.
dnstm-router crashlooping on port 53	modules/dnstt.py always installed + started; legacy gated it on hconfigs["dnstt_enable"]. systemd-resolved owns 53 on most boxes.
ssh-liberty-bridge fighting singbox for the same port	Same pattern: legacy hardcoded install_run other/ssh 0 (disabled). My initial port ignored the flag.
hiddify-cli auto-restart loop the entire session	Same again. Plus the SUB_LINK was being built from panel_links[-1] (the public domain) instead of http://127.0.0.1:9000/... like the legacy run.sh override.
bash common/install.sh failed on every reinstall	mv /etc/cron.d/hiddify_daily_memory_release ... wasn't idempotent.
N nginx restarts per cert refresh	acme.sh --pre-hook bash prepare_acme.sh ran per challenge, and every invocation systemctl restart hiddify-nginx. A 5-domain panel → 5 nginx restarts in 30s. Now it's one.
MariaDB password embeddable into SQL	f"... IDENTIFIED BY '{pw}'" had no escaping. The current password generator is alnum so it never tripped, but the next person to change generate_random_password() would have.
WireGuard peer address math wrong for high user IDs	The bash add_number_to_ipv4 carried v4 once (octets[2..3]) and add_number_to_ipv6 never carried at all. _add_int_to_ip uses ipaddress.ip_address(...) + n, which carries through.

Several of these had been broken since before this PR's first commit.
Bash orchestration made them invisible: no tests, no type checker, errors
discarded with >/dev/null 2>&1 or || true, and silent fall-through on
missing functions.

Architectural improvements

• Tests: from 18 → 175. Every module migration ships with unit tests
  for the pure logic (template rendering, address math, port enumeration,
  cert expiry checks, JSON sanitize, lock TTL, etc.) and unittest.mock
  for the systemctl/subprocess calls. Refactoring is now safe.

• Atomic file writes for state-bearing files: current.json,
  app.cfg, /etc/wireguard/hiddifywg.conf, /etc/iptables/rules.v{4,6},
  and the self-signed certs all go through tempfile + os.replace(). A
  crashed install no longer leaves half-truncated configs.

• Idempotent operations: every _disable_legacy / _ensure_* /
  cron migration is safe to re-run.

• Real python libraries instead of shell-out where it's a clear win:

  • Self-signed certs: cryptography package (no openssl subprocess
    per domain — one process, properly tested cert expiry/key validity
    detection).
  • Address arithmetic: stdlib ipaddress (proper carry, fewer bugs than
    bash arithmetic).
  • DNS lookup: socket.getaddrinfo (no dig dependency).
  • JSON sanitize: json5 (parse with trailing commas, re-emit strict).
  • HTTP API: stdlib urllib (no jq + curl pipe).

• Service-level gates on every module that ships a systemd unit
  (hconfigs["X_enable"] checked before install + auto-stop unit when
  flag flips off). Matches what the legacy install_run other/X $(hconfig "X_enable")
  did but was inconsistently honored after the python port.

• One-process-per-install instead of N+1 forks: the install loop no
  longer launches a bash subshell per module + per install_package /
  is_installed_package call. Faster on cold installs, way less log
  noise.

Panel integration & operability (kept working end-to-end)

Beyond the install loop, the whole panel-facing surface was re-verified
and fixed where the migration had broken it:

• commander.py routes intact. The panel dispatches Apply-Configs /
  Install / Update / Restart / Status / get-cert / update-usage /
  temporary-short-link through commander.py's sudoers path. Each
  target is now a 5-line shim that exec ./init.sh <cmd> (or
  python -m hiddify_manager.modules.<x>). commander.py's Command
  enum is untouched.
• ./init.sh apply-configs — new lightweight "panel state changed"
  path (regenerate current.json → certs → render → firewall → restart),
  replacing apply_configs.sh.
• Panel live-log streaming works. init.sh tees each action to the
  log/system/<action>.log filename the panel's admin_log_api polls,
  with PYTHONUNBUFFERED=1 + stdbuf -oL so it streams line-by-line
  instead of dumping at the end, and set -o pipefail so a python
  failure isn't masked by tee.
• Progress bar restored. utils/progress emits the
  ####<pct>####<title>####<subtitle>#### markers the panel's
  result.html regex parses, so the bar + labels update during long
  actions.
• Real ACME certs fetched on install/apply for direct-mode domains
  (ported from the dropped acme.sh/run.sh), self-signed fallback via
  cryptography on failure.
• Operator menu (./init.sh menu / hiddify): keyboard shortcuts,
  a rich status table (no log-timestamp noise), browseable+tailable
  system logs, fixed "Show admin link" (was calling the destructive
  reset-owner-password), confirm-before-uninstall.

What's preserved (no panel-side churn)

• commander.py is untouched. Its Command enum still points at
  the same paths (get_cert.sh, update_usage.sh, add2shortlink.sh).
  Those files are now 5-line python shims that exec python -m hiddify_manager.modules.<x>. The panel doesn't know or care.
• Systemd unit names unchanged.
• Config layout unchanged (current.json, app.cfg, per-module
  rendered configs all in their legacy paths).
• CLI surface preserved: ./init.sh install|update|status|menu|migrate
  works the same way as before; ./init.sh upgrade [mode] is the only
  new entrypoint, replacing the curl|sh hiddify_installer.sh flow.

What's still bash, and why

File	Reason it stays
init.sh, docker-init.sh	Entrypoints. Must be bash to bootstrap python.
common/download.sh, common/docker-installer.sh	curl|sh first-install bootstrap. Runs before python exists on the box.
common/google-bbr.sh	Vendored upstream (Teddysun BBR installer, 371 lines). Not ours to port.
acme.sh/get_cert.sh, hiddify-panel/update_usage.sh, nginx/add2shortlink.sh	5-line exec /...python -m ... shims so commander.py's sudoers paths stay valid.
operations/lxd/* (×3)	LXD container provisioning — separate operational concern.
.github/release_message.sh	GitHub Actions release-notes helper. CI-only.

How the install path looks now

./init.sh install
  └── hiddify_manager.manager.run_install()
      ├── for module in [common, redis, mysql, panel, nginx, haproxy,
      │                  acme.sh, speedtest, dnstt, telegram, ssfaketls,
      │                  ssh, warp, xray, hiddify-cli, wireguard, singbox]:
      │   ├── installer.install_module(module)
      │   │   └── hiddify_manager.modules.<module>.install()
      │   │       └── (apt + binary download + render + systemctl)
      │   └── if module == "hiddify-panel":
      │       └── manager._render_all_templates()
      │           ├── ensure_self_signed_cert for every domain
      │           ├── utils.template.render_tree(PROJECT_ROOT, configs)
      │           └── common.apply_runtime_config(configs)
      │               └── (timezone, firewall, sshd audit, auto-update cron)
      └── all modules log to /opt/hiddify-manager/log/system/*.log

Every module is independently importable and independently testable.

How to verify

# 1. unit tests on darwin/linux (no root needed):
python -m pytest tests/

# 2. smoke install on an Ubuntu box:
./init.sh install
systemctl list-units --state=failed   # expected: 0

# 3. check service health:
./init.sh status                       # rich colour table

# 4. menu:
./init.sh menu                         # keyboard shortcuts work

Risk

• Bootstrap path (download.sh) rewritten — first-install via
  curl … | bash needs a smoke test on a fresh server. The reinstall
  / update / status / restart / uninstall paths have all been exercised
  on a running dev box throughout the PR.
• CREATE_EASYSETUP_LINK env var (from the deleted download_install_easylink.sh)
  isn't honored yet. Easy add if anyone notices.
• hiddify_installer.sh's whiptail progress UI is gone. Modern CLI users
  haven't missed it; the python output is colorised via rich.

Commit shape

Every behavioural change is in its own atomic commit (218 commits total),
with the new module + tests landing first, the orchestrator wiring next,
and the bash deletion last. Easy to bisect, easy to revert any one piece
if something surfaces in production.

[moham96]

I have a lot of improvements and fixes. queued but all depend on this big PR since it makes the code more maintainable, i made it as a draft to discuss it further, maybe the best way moving forward is to pull it into a separate branch instead of dev