No packages installed in the shell

[purefn]

Using the latest master, 299de57, when I run kubernix without any package parameters kubectl isn't available. Even when I try and add packages with -p kubectl kubernetes-helm, they aren't available in the shell.

$ KUBERNIX_LOG_LEVEL=debug sudo kubernix -p kubernetes-helm kubectl --log-level debug 
[DEBUG] Nix environment not found, bootstrapping one
[DEBUG] Adding additional packages: ["kubernetes-helm", "kubectl"]
[DEBUG] Using default overlay
[DEBUG] Loading kernel module 'overlay'
[DEBUG] Loading kernel module 'br_netfilter'
[DEBUG] Loading kernel module 'ip_conntrack'
[DEBUG] Enabling sysctl 'net.bridge.bridge-nf-call-ip6tables'
[DEBUG] Enabling sysctl 'net.bridge.bridge-nf-call-iptables'
[DEBUG] Enabling sysctl 'net.ipv4.conf.all.route_localnet'
[DEBUG] Enabling sysctl 'net.ipv4.ip_forward'
[DEBUG] Using cluster CIDR 10.10.0.0/24
[DEBUG] Using service CIDR 10.10.1.0/24
[DEBUG] Using CRI-O (0) CIDR 10.10.2.0/24
[DEBUG] Creating CA certificates
[DEBUG] CA certificates created
[DEBUG] Creating certificate for tealc
[DEBUG] Certificate created for tealc
[DEBUG] Creating certificate for admin
[DEBUG] Certificate created for admin
[DEBUG] Creating certificate for kubernetes
[DEBUG] Certificate created for kubernetes
[DEBUG] Creating certificate for kube-controller-manager
[DEBUG] Certificate created for kube-controller-manager
[DEBUG] Creating certificate for kube-proxy
[DEBUG] Certificate created for kube-proxy
[DEBUG] Creating certificate for kube-scheduler
[DEBUG] Certificate created for kube-scheduler
[DEBUG] Creating certificate for service-account
[DEBUG] Certificate created for service-account
[DEBUG] Creating kubeconfig for tealc
[DEBUG] Kubeconfig created for tealc
[DEBUG] Creating kubeconfig for kube-proxy
[DEBUG] Kubeconfig created for kube-proxy
[DEBUG] Creating kubeconfig for kube-controller-manager
[DEBUG] Kubeconfig created for kube-controller-manager
[DEBUG] Creating kubeconfig for kube-scheduler
[DEBUG] Kubeconfig created for kube-scheduler
[DEBUG] Creating kubeconfig for admin
[DEBUG] Kubeconfig created for admin
[DEBUG] No previous run file '/tmp/kubernix/kubernix-run/etcd/run.yml' found, writing new one
[DEBUG] Waiting for process 'etcd' (etcd) to become ready with pattern: 'ready to serve client requests'
[DEBUG] No previous run file '/tmp/kubernix/kubernix-run/crio/tealc/run.yml' found, writing new one
[DEBUG] Waiting for process 'CRI-O' (crio) to become ready with pattern: 'Sandboxes:'
[DEBUG] Found pattern 'Sandboxes:' in line 'time="2020-05-06 10:44:53.737198851-07:00" level=debug msg="Sandboxes: []" file="server/server.go:409"'
[DEBUG] No previous run file '/tmp/kubernix/kubernix-run/kubelet/tealc/run.yml' found, writing new one
[DEBUG] Waiting for process 'Kubelet' (kubelet) to become ready with pattern: 'Successfully registered node'
[DEBUG] Found pattern 'ready to serve client requests' in line '2020-05-06 10:44:54.496595 I | embed: ready to serve client requests'
[DEBUG] No previous run file '/tmp/kubernix/kubernix-run/apiserver/run.yml' found, writing new one
[DEBUG] Waiting for process 'API Server' (kube-apiserver) to become ready with pattern: 'etcd ok'
[DEBUG] Found pattern 'etcd ok' in line '[+]etcd ok'
[DEBUG] Creating API Server RBAC rule for kubelet
[DEBUG] Found pattern 'Successfully registered node' in line 'I0506 10:44:57.829200  116450 kubelet_node_status.go:73] Successfully registered node tealc'
[DEBUG] No previous run file '/tmp/kubernix/kubernix-run/proxy/run.yml' found, writing new one
[DEBUG] Waiting for process 'Proxy' (kube-proxy) to become ready with pattern: 'Caches are synced'
[DEBUG] API Server RBAC rule created
[DEBUG] No previous run file '/tmp/kubernix/kubernix-run/scheduler/run.yml' found, writing new one
[DEBUG] No previous run file '/tmp/kubernix/kubernix-run/controllermanager/run.yml' found, writing new one
[DEBUG] Waiting for process 'Scheduler' (kube-scheduler) to become ready with pattern: 'Serving securely'
[DEBUG] Waiting for process 'Controller Manager' (kube-controller-manager) to become ready with pattern: 'Serving securely'
[DEBUG] Found pattern 'Serving securely' in line 'I0506 10:44:58.957975  116722 secure_serving.go:178] Serving securely on [::]:10259'
[DEBUG] Found pattern 'Serving securely' in line 'I0506 10:44:59.101301  116724 secure_serving.go:178] Serving securely on [::]:10257'
[DEBUG] Found pattern 'Caches are synced' in line 'I0506 10:44:59.159248  116663 shared_informer.go:230] Caches are synced for endpoints config'
[DEBUG] Waiting for coredns to be ready
[DEBUG] coredns status not available (0/60s)
[DEBUG] coredns status not available (2/60s)
[DEBUG] coredns status not available (4/60s)
[DEBUG] coredns status not available (6/60s)
[DEBUG] coredns status not available (8/60s)
[DEBUG] coredns status: 0/1 (10/60s)
[DEBUG] coredns status: 0/1 (12/60s)
[DEBUG] coredns status: 0/1 (14/60s)
[DEBUG] coredns status: 0/1 (16/60s)
[DEBUG] coredns status: 0/1 (18/60s)
[DEBUG] coredns status: 0/1 (20/60s)
[DEBUG] coredns status: 0/1 (22/60s)
[DEBUG] coredns status: 0/1 (24/60s)
[DEBUG] coredns status: 0/1 (26/60s)
[DEBUG] coredns status: 0/1 (28/60s)
[DEBUG] coredns status: 0/1 (31/60s)
[DEBUG] coredns status: 0/1 (33/60s)
[DEBUG] coredns status: 0/1 (35/60s)
[DEBUG] coredns status: 0/1 (37/60s)
[DEBUG] coredns status: 0/1 (39/60s)
[DEBUG] coredns status: 0/1 (41/60s)
[DEBUG] coredns status: 1/1 (43/60s)
[DEBUG] coredns ready
[  51s] █████████████████████████ 22/22 Everything is up and running
[INFO ] Spawning interactive shell
[INFO ] Please be aware that the cluster stops if you exit the shell

[root@tealc:/tmp/kubernix/kubernix-run]# kubectl
The program ‘kubectl’ is currently not installed. It is provided by
several packages. You can install it by typing one of the following:
  nix-env -iA nixos.kubectl
  nix-env -iA nixos.kubernetes
  nix-env -iA nixos.openshift

[root@tealc:/tmp/kubernix/kubernix-run]# helm
The program ‘helm’ is currently not installed. It is provided by
several packages. You can install it by typing one of the following:
  nix-env -iA nixos.helm
  nix-env -iA nixos.kubernetes-helm

[root@tealc:/tmp/kubernix/kubernix-run]# which kubectl
which: no kubectl in (/run/wrappers/bin:/root/.nix-profile/bin:/etc/profiles/per-user/root/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin)

$  cat nix/default.nix
let
  pkgs = import ./nixpkgs.nix { overlays = [(import ./overlay.nix)]; };
  packages = with pkgs; [
    cacert
    cfssl
    cni-plugins
    conmon
    conntrack-tools
    cri-o
    cri-tools
    etcd
    iproute
    iptables
    kmod
    kubernetes
    kubectl
    podman
    runc
    socat
    sysctl
    utillinux
  ] ++ [ kubernetes-helm kubectl ];
in packages

Am I missing something?

[saschagrunert]

Hey, thank you for the issue report. That is really strange, If I run the latest master on my machine, then:

> which kubectl
/nix/store/nswg7yxsp97vny77r74c62s5xkmac4zk-kubernetes-1.18.1/bin/kubectl

Can you clone the repo and do a simple make run after removing the kubernix-run directory?

[purefn]

Same result

$ sudo make run
[sudo] password for rwallace: 
nix run -f nix/build.nix -ik SSH_AUTH_SOCK -c cargo build --release
[0.5 MiB DL]
    Updating crates.io index
    Updating git repository `https://github.com/clap-rs/clap`
  Downloaded serde_yaml v0.8.11
  Downloaded unicode-width v0.1.7
  Downloaded crossbeam-channel v0.4.2
  Downloaded base64 v0.12.1
  Downloaded lazy_static v1.4.0
  Downloaded parking_lot v0.10.2
  Downloaded syn v1.0.19
  Downloaded heck v0.3.1
  Downloaded nix v0.17.0
  Downloaded quote v1.0.4
  Downloaded toml v0.5.6
  Downloaded hostname v0.3.1
  Downloaded serde v1.0.106
  Downloaded atty v0.2.14
  Downloaded signal-hook v0.1.14
  Downloaded serde_json v1.0.52
  Downloaded bitflags v1.2.1
  Downloaded termcolor v1.1.0
  Downloaded vec_map v0.8.2
  Downloaded proc-macro2 v1.0.12
  Downloaded indexmap v1.3.2
  Downloaded anyhow v1.0.28
  Downloaded console v0.11.2
  Downloaded textwrap v0.11.0
  Downloaded proc-mounts v0.2.4
  Downloaded strsim v0.10.0
  Downloaded ipnetwork v0.16.0
  Downloaded log v0.4.10
  Downloaded os_str_bytes v2.3.0
  Downloaded getset v0.1.0
  Downloaded indicatif v0.14.0
  Downloaded proc-macro-error v0.4.12
  Downloaded term_size v1.0.0-beta1
  Downloaded rayon v1.3.0
  Downloaded rand v0.7.3
  Downloaded libc v0.2.69
  Downloaded ryu v1.0.4
  Downloaded regex v1.3.7
  Downloaded maybe-uninit v2.0.0
  Downloaded itoa v0.4.5
  Downloaded match_cfg v0.1.0
  Downloaded serde_derive v1.0.106
  Downloaded cfg-if v0.1.10
  Downloaded autocfg v1.0.0
  Downloaded terminal_size v0.1.12
  Downloaded unicode-segmentation v1.6.0
  Downloaded parking_lot_core v0.7.2
  Downloaded void v1.0.2
  Downloaded crossbeam-utils v0.7.2
  Downloaded lock_api v0.3.4
  Downloaded unicode-xid v0.2.0
  Downloaded term_size v0.3.2
  Downloaded partition-identity v0.2.8
  Downloaded signal-hook-registry v1.2.0
  Downloaded yaml-rust v0.4.3
  Downloaded number_prefix v0.3.0
  Downloaded dtoa v0.4.5
  Downloaded linked-hash-map v0.5.3
  Downloaded termios v0.3.2
  Downloaded getrandom v0.1.14
  Downloaded rand_chacha v0.2.2
  Downloaded crossbeam-deque v0.7.3
  Downloaded rand_core v0.5.1
  Downloaded smallvec v1.4.0
  Downloaded scopeguard v1.1.0
  Downloaded arc-swap v0.4.6
  Downloaded version_check v0.9.1
  Downloaded rayon-core v1.7.0
  Downloaded either v1.5.3
  Downloaded regex-syntax v0.6.17
  Downloaded proc-macro-error-attr v0.4.12
  Downloaded err-derive v0.2.4
  Downloaded ppv-lite86 v0.2.6
  Downloaded crossbeam-queue v0.2.1
  Downloaded rustversion v1.0.2
  Downloaded num_cpus v1.13.0
  Downloaded synstructure v0.12.3
  Downloaded crossbeam-epoch v0.8.2
  Downloaded proc-macro-error v1.0.1
  Downloaded syn-mid v0.5.0
  Downloaded memoffset v0.5.4
  Downloaded proc-macro-error-attr v1.0.1
   Compiling proc-macro2 v1.0.12
   Compiling unicode-xid v0.2.0
   Compiling libc v0.2.69
   Compiling syn v1.0.19
   Compiling version_check v0.9.1
   Compiling autocfg v1.0.0
   Compiling cfg-if v0.1.10
   Compiling lazy_static v1.4.0
   Compiling serde v1.0.106
   Compiling maybe-uninit v2.0.0
   Compiling rustversion v1.0.2
   Compiling scopeguard v1.1.0
   Compiling getrandom v0.1.14
   Compiling bitflags v1.2.1
   Compiling unicode-width v0.1.7
   Compiling regex-syntax v0.6.17
   Compiling unicode-segmentation v1.6.0
   Compiling ryu v1.0.4
   Compiling rayon-core v1.7.0
   Compiling arc-swap v0.4.6
   Compiling smallvec v1.4.0
   Compiling linked-hash-map v0.5.3
   Compiling ppv-lite86 v0.2.6
   Compiling anyhow v1.0.28
   Compiling nix v0.17.0
   Compiling log v0.4.10
   Compiling vec_map v0.8.2
   Compiling dtoa v0.4.5
   Compiling os_str_bytes v2.3.0
   Compiling number_prefix v0.3.0
   Compiling termcolor v1.1.0
   Compiling itoa v0.4.5
   Compiling either v1.5.3
   Compiling strsim v0.10.0
   Compiling void v1.0.2
   Compiling match_cfg v0.1.0
   Compiling base64 v0.12.1
   Compiling lock_api v0.3.4
   Compiling heck v0.3.1
   Compiling yaml-rust v0.4.3
   Compiling crossbeam-utils v0.7.2
   Compiling memoffset v0.5.4
   Compiling crossbeam-epoch v0.8.2
   Compiling indexmap v1.3.2
   Compiling regex v1.3.7
   Compiling quote v1.0.4
   Compiling proc-macro-error-attr v1.0.1
   Compiling proc-macro-error-attr v0.4.12
   Compiling proc-macro-error v0.4.12
   Compiling proc-macro-error v1.0.1
   Compiling crossbeam-queue v0.2.1
   Compiling crossbeam-channel v0.4.2
   Compiling crossbeam-deque v0.7.3
   Compiling terminal_size v0.1.12
   Compiling num_cpus v1.13.0
   Compiling term_size v0.3.2
   Compiling termios v0.3.2
   Compiling signal-hook-registry v1.2.0
   Compiling parking_lot_core v0.7.2
   Compiling atty v0.2.14
   Compiling term_size v1.0.0-beta1
   Compiling hostname v0.3.1
   Compiling rand_core v0.5.1
   Compiling textwrap v0.11.0
   Compiling console v0.11.2
   Compiling signal-hook v0.1.14
   Compiling parking_lot v0.10.2
   Compiling rand_chacha v0.2.2
   Compiling rayon v1.3.0
   Compiling indicatif v0.14.0
   Compiling rand v0.7.3
   Compiling syn-mid v0.5.0
   Compiling synstructure v0.12.3
   Compiling serde_derive v1.0.106
   Compiling err-derive v0.2.4
   Compiling clap_derive v3.0.0-beta.1 (https://github.com/clap-rs/clap#9d03c849)
   Compiling getset v0.1.0
   Compiling partition-identity v0.2.8
   Compiling proc-mounts v0.2.4
   Compiling clap v3.0.0-beta.1 (https://github.com/clap-rs/clap#9d03c849)
   Compiling serde_json v1.0.52
   Compiling toml v0.5.6
   Compiling ipnetwork v0.16.0
   Compiling serde_yaml v0.8.11
   Compiling kubernix v0.2.0 (/tmp/kubernix)
    Finished release [optimized] target(s) in 1m 49s
sudo -E target/release/kubernix 
[  34s] █████████████████████████ 22/22 Everything is up and running
[INFO ] Spawning interactive shell
[INFO ] Please be aware that the cluster stops if you exit the shell

[root@tealc:/tmp/kubernix/kubernix-run]# kubectl
The program ‘kubectl’ is currently not installed. It is provided by
several packages. You can install it by typing one of the following:
  nix-env -iA nixos.kubectl
  nix-env -iA nixos.kubernetes
  nix-env -iA nixos.openshift

Is it relevant that I'm on NixOS?

[purefn]

If you are using NixOS, I wonder if maybe it is available in your shell because it is installed as a system package?

[saschagrunert]

Unfortunately I cannot spin-up a NixOS environment right now. But yeah, it could be related to the fact that you're using NixOS.

[afreakk]

Im using nixos and had the same issue, but when i did sudo -E kubernix i got the packages.
(Tested on newest master)

[saschagrunert]

Im using nixos and had the same issue, but when i did sudo -E kubernix i got the packages.
(Tested on newest master)

Yeah it seens to be the env which gets cleared out.