Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UTF-8 support in mysql requires mysql_enable_utf8mb4, not mysql_enable_utf8 #101

Open
yahermann opened this issue Jan 15, 2022 · 0 comments

Comments

@yahermann
Copy link

yahermann commented Jan 15, 2022

According to:
https://metacpan.org/pod/DBD::mysql#mysql_enable_utf8mb4
DBD::mysql requires setting mysql_enable_utf8mb4 to support 4-byte UTF-8.

MySQL is a bit quirky in that utf8 is not real UTF-8. To get real UTF-8, you need utf8mb4:
https://mathiasbynens.be/notes/mysql-utf8mb4

Shared/lib/Dancer/Plugin/Database/Core.pm, appears to be setting mysql_enable_utf8 which only provides 3-byte UTF-8:

# If the app is configured to use UTF-8, the user will want text from the
# database in UTF-8 to Just Work, so if we know how to make that happen, do
# so, unless they've set the auto_utf8 plugin setting to a false value.
my $app_charset = $settings->{charset} || "";
my $auto_utf8 = exists $settings->{auto_utf8} ?  $settings->{auto_utf8} : 1;

if (lc $app_charset eq 'utf-8' && $auto_utf8) {
    # The option to pass to the DBI->connect call depends on the driver:
    my %param_for_driver = (
        SQLite => 'sqlite_unicode',
        mysql  => 'mysql_enable_utf8',   #################################################
        Pg     => 'pg_enable_utf8',
    );

    my $param = $param_for_driver{$driver};

    if ($param && !$settings->{dbi_params}{$param}) {
        $logger->(
            debug => "Adding $param to DBI connection params"
                . " to enable UTF-8 support"
        );
        $settings->{dbi_params}{$param} = 1;
    }
}

and to add insult to injury, adds a warning if we override to mysql_enable_utf8mb4 without disabling auto_utf8, and on top of that, it doesn't Just Work as advertised, it Just Works but not Always 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant