UTF-8 encoded data should not by encoded with latin-1 #476
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Can you show an example url and a stack trace? generally speaking a utf8 path that is properly encoded will work fine with webob. The Latin-1 encoding is used within PEP3333 to marshal bytes around as Unicode code points but they are rarely interpreted that way. That being said in HTTP a lot of things like headers don’t support utf8. |
|
@mmerickel Thank you for your quick response, I will get back to this the upcoming week. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I had problems non ascii letters like umlauts and ß in the requests url path, even though they where uri encoded and also decoded properly before the latin-1 encode step in line 120.
I know this might be a naive fix, maybe you can come up with a better overall approach or guide me towards it. I also don't know if utf-8 encoded uris are out of scope for this library.