Bug when source file name is too long

[sampaccoud]

When the source file name is too long, I get a database error that the thumbnail name is too long:

  File "/usr/local/lib/python3.7/site-packages/easy_thumbnails/models.py", line 18, in get_file
    obj, created = self.get_or_create(**kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 588, in get_or_create
    return self.create(**params), True
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 740, in save
    force_update=force_update, update_fields=update_fields)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 778, in save_base
    force_update, using, update_fields,
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 921, in _do_insert
    using=using, raw=raw,
  File "/usr/local/lib/python3.7/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.7/site-packages/sentry_sdk/integrations/django/__init__.py", line 489, in execute
    return real_execute(self, sql, params)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.DataError: value too long for type character varying(255)

I tried configuring the THUMBNAIL_NAMER option to shorten the thumbnails name but Django filer overrides the get_thumbnail_name method and breaks this feature (known issue #1093).

I don't really care how my thumbnails are named but we should make sure it does not cause a database error.

I don't know what is the solution for this:

• increase field max length in database?
• limit the length of the source file name?
• improve filer's get_thumbnail_name method but how?
• any other solution?

[bartoszcholewa]

I've had exactly the same issue in my Django project with django-filer file manager.
Error is caused by this SQL query:

INSERT INTO "easy_thumbnails_thumbnail" ("storage_hash", "name", "modified", "source_id") VALUES ('f9bde26a1556cd667f742bd34ec7c55e', 'filer_public_thumbnails/filer_public/1e/ec/1eec4da6-782c-4251-a65d-1624827036ca/melchior_wankowicz_przekazuje_weteranowi_powstania_styczniowego_mamertowi_wandalliemu_klucz_do_domu_w_osiedlu_zasluzonych_im_aleksandry_pilsudskiej_1936_fot_polona.jpg__80x80_q85_crop_subsampling-2.jpg', '2022-01-25T20:25:27.332494+00:00'::timestamptz, 956) RETURNING "easy_thumbnails_thumbnail"."id"

This is my solution:

1. settings.py: - set custom upload name function (default is filer.utils.generate_filename.randomized

FILER_STORAGES = {
    'public': {
        'main': {
            'UPLOAD_TO': 'appname.utils.filer_name_validation'
        }
    },
    'private': {
        'main': {
            'UPLOAD_TO': 'appname.utils.filer_name_validation'
        }
    }
}

2. appname/utils.py - create utils.py file in django root app, rewrite filer's randomized function for cutting too long names:

from filer.utils.generate_filename import randomized

def filer_name_validation(instance, filename):
    name = randomized(instance, filename)
    extension = name.split('.')[-1]
    name = name[:100]
    name = name[:len(name) - len(extension) - 1] + '.' + extension
    return name

That's it. I've chosen to cut the string to 100 characters, because you have to consider the absolute path in name,
so path + filename[:100] + extension should be enough to fit max_lenght=255 in filer model.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This will now be closed due to inactivity, but feel free to reopen it.

[PeterW-LWL]

This is still a problem, as it will sometimes cause the filer admin detail page and filer admin list page to become inaccessible.

[bartTC]

I'd like to reopen the ticket as well. It's a recurring issue.