UnicodeEncodeError: 'charmap' codec can't encode character '\uf0df' in position 28689: character maps to <undefined>

[giuliastro]

Traceback

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Software\Docling\venv\Scripts\docling.exe_main.py", line 7, in
File "C:\Software\Docling\venv\Lib\site-packages\typer\main.py", line 338, in call
raise e
File "C:\Software\Docling\venv\Lib\site-packages\typer\main.py", line 321, in call
return get_command(self)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Software\Docling\venv\Lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Software\Docling\venv\Lib\site-packages\typer\core.py", line 665, in main
return _main(
^^^^^^
File "C:\Software\Docling\venv\Lib\site-packages\typer\core.py", line 197, in _main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "C:\Software\Docling\venv\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Software\Docling\venv\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Software\Docling\venv\Lib\site-packages\typer\main.py", line 703, in wrapper
return callback(**use_params)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Software\Docling\venv\Lib\site-packages\docling\cli\main.py", line 389, in convert
export_documents(
File "C:\Software\Docling\venv\Lib\site-packages\docling\cli\main.py", line 112, in export_documents
conv_res.document.save_as_markdown(
File "C:\Software\Docling\venv\Lib\site-packages\docling_core\types\doc\document.py", line 1942, in save_as_markdown
fw.write(md_out)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.2288.0_x64__qbz5n2kfra8p0\Lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\uf0df' in position 28689: character maps to

Steps to reproduce

launching Dockling command line on a PDF

Docling version

2.1

Python version

3.12.8

[cau-git]

@giuliastro Could you please provide us a sample PDF which causes this problem? We need one to investigate this problem.

[cau-git]

Possibly related to #598

[giuliastro]

Hi @cau-git, thank you.
Unfortunately, they are undisclosed docs and I cannot share them.
I agree with you that it seems like an encoding problem, as shown in #598, all my files are indeed UTF-8.
Probably setting the right encoding before reading / writing files would solve the problem, I must try this.