Trying to input chinese character crash with ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: Incorrect string value: '\xE9\x80\x89\xE8\xAF\xB6...' for column 'value' at row 1

[allan-simon]

I'm using an instance made with docker and https://registry.hub.docker.com/u/tvelocity/etherpad-lite/

when I try to input some chinese characters , the instance crash with this error message in the logs

[2015-05-06 07:33:10.540] [ERROR] console - Error: ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: Incorrect string value: '\xE9\x80\x89\xE8\xAF\xB6...' for column 'value' at row 1
    at Query.Sequence._packetToError (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/protocol/sequences/Sequence.js:48:14)
    at Query.ErrorPacket (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/protocol/sequences/Query.js:83:18)
    at Protocol._parsePacket (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/protocol/Protocol.js:271:23)
    at Parser.write (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/protocol/Parser.js:77:12)
    at Protocol.write (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/protocol/Protocol.js:39:16)
    at Socket.<anonymous> (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/Connection.js:82:28)
    at Socket.emit (events.js:95:17)
    at Socket.<anonymous> (_stream_readable.js:748:14)
    at Socket.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:410:10)
    --------------------
    at Protocol._enqueue (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/protocol/Protocol.js:135:48)
    at Connection.query (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/mysql/lib/Connection.js:185:25)
    at /opt/etherpad-lite/src/node_modules/ueberDB/mysql_db.js:214:18
    at /opt/etherpad-lite/src/node_modules/ueberDB/node_modules/async/lib/async.js:462:21
    at /opt/etherpad-lite/src/node_modules/ueberDB/node_modules/async/lib/async.js:194:13
    at /opt/etherpad-lite/src/node_modules/ueberDB/node_modules/async/lib/async.js:97:13
    at Array.forEach (native)
    at _forEach (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/async/lib/async.js:26:24)
    at async.forEach (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/async/lib/async.js:96:9)
    at _asyncMap (/opt/etherpad-lite/src/node_modules/ueberDB/node_modules/async/lib/async.js:193:9)

which seems to be a MySQL error, so I'm not sure which one is to blame ?

[JohnMcLear]

It might be that the database encoding is incorrect. Please test on http://beta.etherpad.org

[webzwo0i]

You seem to rely on mysql's default values when creating the database. epl needs binary-collation and at least utf8, so can you show variables where Variable_name like 'character_set%' or Variable_name like 'collation%'; and show full columns from store; and see if its all utf8 and utf8_bin like described in https://github.com/ether/etherpad-lite/wiki/How-to-use-Etherpad-Lite-with-MySQL

[allan-simon]

ok thanks for your feedback i'm going to check that tomorrow morning and tell you the result.

[JohnMcLear]

I put a patch in upstream tvelocity/dockerfiles#7

[JohnMcLear]

Btw @webzwo0i if I alter database to use utf8bin then new tables created will also use utf8bin right?

[MoGray]

You should be able to do it to just one table if you want like this,

ALTER TABLE table_name
[[DEFAULT] CHARACTER SET charset_name]
[COLLATE collation_name]

If you wanted you could do it for your entire database by,

ALTER DATABASE database_name DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci;

[allan-simon]

@JohnMcLear thanks for the upstream patch