Skip to content

Message dumping is slow for big files #869

@ghost

Description

Currently all the dump files are valid JSON files, which means that for every message to be dumped, this happens:

  • Read and parse the full dump-file
  • Append the new dump
  • Re-write the entire file

Obviously, this becomes incredibly slow for bigger files.

One possibility for fixing this is to work line-based. So, it's not a valid JSON file anymore (at least with more than one line), but each line is on itself. Consequently, only one line needs to be appended. That's faster and less error-prone.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions