py3 fixalltests #110
py3 fixalltests #110
Conversation
wthie
commented
Dec 3, 2020
wthie
commented
- Almost all of the porting work was done by Markus Demleitner
- This pull request was specifically created at the request of the community, it does not adhere to the development standards used in the twisted universe. This draft PR is way too big and messy, my work was more like a last ditch effort to see the light at the other end of the tunnel. The common understanding is that this PR would serve as the basis for a number of more specific and more finely tuned PR's.
- Spidermonkey needs to be installed for the JS tests. Experiments with node.js instead ended with a few tests depending too much on Spidermonkey's features. That's why I dropped the idea of using node.js alternatively.
- subunit needs to be installed, the tests will inform you to do so
- Initial porting of non trivial code has shown that the flatteners are a major 'piece de resistance' with the suggestion of replacing them with twisted/web flatteners
The main pain are adapters for built-in classes and microdom flatteners, which are currently not loaded (and hence, only 33 tests succeed).
The plan here is to leave encoding of flattened thing to whatever delivers the content (or to write, as applicable).
Also, various minor changes; these suffice to make test_url tests pass.
(and migrating to context managers where appropriate for good measure).
This also fixes failed tests for http range.
(this also fixes the flatstan unit tests).
This fixes some unittests from test_newflat.
…haviour. This makes all flat_new tests pass.
This is a consequence of microdom missing in the python3 twisted packages; it might be a better idea to just raise exceptions when they're used, but then people will, in effect, probably get better error messages this way.
Right now, the plan is to have request.url be a string, only converted to a bytestring when twisted requires that. Let's see if we can get away with that.
There is one behavioural change: invalid language preference levels were sorted to the front of the preference list so far. Now, they're at the end.
Also, fixing some additional instances where upstream components either want bytes (e.g., twisted.log) or strings (e.g., cgi.FieldStorage). Oh boy, this doesn't look good. Consequently, the testutils.Request accumulator is now bytes. This means that lots of test assertions had to be changed.
This make test_compress pass.
But I've never really used guard, and I'd need to understand more of it to work out what's what here. So, I'm at 12 successful tests, 63 errors, 15 failures.
To enable that, re-adding a json module. In contrast to the original nevow.json, this is a wrapper around the built-in json module with extra behaviour for nevow classes; also, this uses loads and dumps rather than parse and serialize as before for consistency with builtin json. Dropped support for unsupported in json, dropped support for embedding executable javascript in json. I suspect we'll need this if people actually use athena. Does anyone? Json will no longer interpret \x escapes (it's not RFC 8259, and I hope it's not used anywhere). Consequently, restored and fixed test_json.
(twisted.web.Request has that, too) Forcing testutil.FakeRequest.args to have byte string keys now (twisted.web.Request has that, too) Using byte strings in testutils.Request for prepath too, now (again, we must due to twisted.web.Request). Starting a document to finally try and make sense of str vs. byte. Updating test_rend (and, minimally, rend.py) for that plan.
the docs/strings_and_bytes plan. This commit has all expected nevow.tests pass except for test_nit, and test_consolejstest (both of which look a bit odd).
|
Hi there @wthie! Any chance you could break this up into ... a lot more PRs, so folks could possibly review them? As a monolith this is enormous. Hopefully stuff like Regarding this:
The more of nevow that we can replace with |
|
Assuming that the test coverage for nevow is 100% making all tests pass under Python3 ideally would give one a library which serves the same purpose as the same library did under Python2. This was the leanest effort assumption underlying this work, although finding out that porting actual production code does not really work, the same as with a lot of the examples in nevow which would need attention and rewriting. I had Markus Demleitner's fork around for quite some time after he had the parts crossed over he needed for his production code. With Covid-19 I saw a chance to set apart some time to work on the nevow/athena section. After a few stabs and getting frustrated I started over to stubbornly make the tests work one by one with the result being this PR, ugly as hell, mechanically done! I apologize for this but it is about as far as I can go right now. I now see a chance of succeeding with porting my productive code. After that I believe I'll be in a much better position to tackle replacing similar functionality code in nevow with twisted code. This is possibly also the position of https://github.com/BMtthws as well as it was the one of https://github.com/msdemlei All this sounds pretty lame and it is an excuse but I promise I'll be back! |
|
My plan to help here is to replicate as many of the porting changes as possible with automated tools like |
…rrently some weird bytes to str transitions necessary
