-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Distributing PyD-produced binaries #54
Comments
Ugh, I don't want to think about this now, dmd is the only compiler that is rigorously capable of compiling pyd right now (well, 6-8 months ago when I last checked), and distributing dmd's phobos might run into legal issues. Or it might not, idk. But suppose we lived in a perfect world: what do we want when we have just written the python extension
and it just works, on windows, mac, as many flavors of linux as is reasonable, without needing to set off a local build. We want others to be able to use So back in the real world, when you have If it were possible to compile with dmd v2.72.0 and target runtime 2.68.2, then it might be reasonable to specify a standard runtime level that pyd wheels should target. Then option 4 becomes a bit more viable. Assuming the glibc compatibility constraint is met. Next question: what happens when Next question: what happens when |
Regarding your symbol conflict concerns: this issue is easy to solve at least on Linux/OS X with Licensing concerns could be avoided by forcing developers to build code with LDC, it keeps up with DMD's innovations quite well. The real issue is avoiding two copies of runtime loaded at the same time. And for that I find your idea of targeting a specific runtime quite reasonable. If |
Handy. I would prefer ldc be the standard but for some dumb blockers (#43). pyd extensions are still going to depend on phobos, and packages depending on different versions of phobos are still going to prevent them from being used side by side, unless symbols of dependencies of dependencies get namespaced, no idea if that's how things work or not. Failing that, we could work towards making pyd less dependent on phobos. Pyd uses a lot of metaprogramming apis, and it also provides conversion support for a bunch of phobos types. The former would have to be pulled into pyd, the latter might have to be dropped. Not liking the sound of any of this. |
Some homework for me:
|
Another question that comes to mind: how do things get distributed in d-land? Dub didn't do binary distributions last I checked, so I would assume you're left to your distro's package manager for *nixen and nothing or chocolatey on windows. |
Removing phobos from dependencies doesn't make any sense. I guess binary distributions in d-land don't exist a) for the same reason this discussion is taking place; b) because |
Hello,
I figured this issue tracker is a good place to spark this discussion.
Let's imagine that someone fiddles enough with PyD to create something useful. How is the produced library to be distributed? The options are seemingly as follows:
libphobos2.so
into the package, making its size rather huge-fPIC
libphobos2-{dmd,ldc,gdc}.so
package to Anaconda/PyPIAs far as I understand, though, options 2 and 3 are fine only as long as the user doesn't load two such packages simultaneously because of GC and such.
Option 1 is rather outrageous these days, now that even
numpy
andscipy
finally havemanylinux1
wheels so that one doesn't need to compile them. That's not the biggest issue, though: the nightmare will really begin when different packages start popping up with different D frontend version requirements.Option 4 looks the most reasonable; with regard to PyPI it feels somewhat of an abuse, though I suppose we're to see more and more such unintended usage in the future.
There is a small catch, though:
manylinux1
PEP puts requirements on the system library versions, namely, the binary must be compatible with the rather old CentOS 5.11 which is still in use on many clusters;druntime
currently relies onqsort_r
from newer glibc 2.8. That said, I've got a patch to handle this (required versions can be checked by runningreadelf -V libphobos2.so | grep 'Name:'
).Any thoughts?
The text was updated successfully, but these errors were encountered: