HyperKit Driver

[dlorenc]

This is a tracking issue for possibly creating a hyperkit driver, instead of the xhyve one we're currently using.

[dlorenc]

I was able to get hyperkit to boot minikube, however it looks like hyperkit has dropped support for running the vmnet framework in host mode, which means we can't get an IP address that the host can connect to.

[copumpkin]

This is probably relevant too once we have it working: https://blog.docker.com/2017/05/user-guided-caching-in-docker-for-mac/

Edit: erm, I guess osxfs isn't open-source 😦

[scottrigby]

@copumpkin Also, osxfs has this long standing issue for projects with many directories and files (such as PHP Symfony or Drupal). This is one of the main reasons I have been using the xhyve driver with Minikube for local development on K8S.

See this PR for more context: helm/charts#1057 CC @lypht @rabellamy @michelleN @ryanj @tomdavidson. You know, we may want to make some master issue to track the work related to K8S local file mounting (generally for local development, but really for any purpose).

[tomdavidson]

@dlorenc doesnt hyperkit rely on vpnkit for host network connectivity?

@scottrigby I think moving forward with hyperkit may eventually solve the "host volume mounting" fracture because I think one of Moby's goals to reduce the explicit requirement for Linux kernel features. The history of host volume mounting is rocky with lots of shims and bandaids and only worked in certain configurations (and still does not work completely with Docker for Windows). Prior to Docker for OSX with xhyve didn't OSX users have to result to nfs shares or rsync? This is an underlying OS and hypervisor combo specific problem and not unique to Docker, k8s, or minikube. For example, create a xhyve or vbox machine on your OSX laptop, you don't need docker, use whatever guest OS you want, but do configure the host volume mounting. Run a nodemon or some file watch in the guest and then modify the files from the host ... the guest will not get the updates without polling.

I think eventually for a Moby based solution is a long way off and minikube, such as helm/charts#1057, might have be better suited with a volume sync sidecar and a kubectl proxy / sync wrapper ( ... that wouldn't work with hyperkit unless there was networking ).

[aaron-prindle]

A hyperkit driver was mentioned as a priority for the minishift team as well. Linking the relevant meeting doc here:
https://docs.google.com/document/d/1zauvKVsxUNR9r8WhYE8dpCGeRnt7Tc6R67JxrCgtY6I/edit#

[copumpkin]

@tomdavidson I thought the 9p filesystem was to avoid the NFS stuff, but haven't looked recently. I've definitely seen support for it in hyperkit.

Also, in case anyone was wondering, if we don't require a distinct IP for the VM (and are okay with just mapping ports to localhost), we can actually run a completely unprivileged hyperkit VM, so no need for suid bits or any of that annoyance.

[sampowers]

How could the ports get mapped to localhost? I'm interested in getting that behavior on osx with Services running in minikube. But some process would have to handle proxying those connections to the vm. I think i have observed VPNKit handling this on Docker for Mac. But couldn't find details on how to do something similar with minikube, and am not (yet?) aware of anything else that handles that use case well. Including binding to priv ports like 443. Thanks On Jun 21, 2017 3:23 PM, "Daniel Peebles" <notifications@github.com> wrote: @tomdavidson <https://github.com/tomdavidson> I thought the 9p filesystem was to avoid the NFS stuff, but haven't looked recently. I've definitely seen support for it in hyperkit. Also, in case anyone was wondering, if we don't require a distinct IP for the VM (and are okay with just mapping ports to localhost), we can actually run a completely unprivileged hyperkit VM, so no need for suid bits or any of that annoyance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1543 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGDkj12HRdGeQff7dB5b9s-oTRU4MeUks5sGZf6gaJpZM4Nr7fl> .

[copumpkin]

@sampowers my understanding stems mostly from moby/hyperkit#136 (comment), which suggests that HyperKit can run with no superuser privileges when using VPNKit (instead of vmnet), and VPNKit would only need privileges if opening ports <1024. So if you were willing to map all ports above 1024 and handle the mapping yourself on localhost, you could run the whole shebang with no setuid.

[dlorenc]

Just spent some more time playing with this. I got the docker-machine-driver-xhyve code booting with hyperkit, but now I'm stuck trying to get the IP/mac address back out. The existing code shells out to xhyve with a special flag to print the mac address for a given UUID, but hyperkit doesn't have that feature.

[dlorenc]

Ref machine-drivers/docker-machine-driver-xhyve#189

[r2d4]

I think we can close this, since we have this merged in tree now.