ModuleNotFoundError: No module named 'magic'

[sekharmalla]

Hey there team,

I am super grateful for all the hard work that you are doing for the Open Source LLM community,

I will come to the point, I was trying to follow 2024-04-04-build-rag-with-python tutorial on building rag with python and after everything is done and tried running python3 import.py then there's magic library error I paste below.

Traceback (most recent call last):
  File "Documents/videoprojects/2024-04-04-build-rag-with-python/import.py", line 2, in <module>
    from utilities import readtext, getconfig
  File "Documents/videoprojects/2024-04-04-build-rag-with-python/utilities.py", line 1, in <module>
    import re, os, requests, magic, ollama, string, configparser
ModuleNotFoundError: No module named 'magic'

If you can help me address this issue or find a workaround for me, I am would be more happy :) I am really excited to seeing this work.

Thanks

[jeugregg]

look at : https://github.com/ahupp/python-magic

you need to install also for example :
OSX
When using Homebrew: brew install libmagic
When using macports: port install file

for other hardware, to can see there.

[sekharmalla]

Thanks @jeugregg

I've tried that as well,

after it said it's already installed, I tried running python3 import.py and then it gave me the same error again.

Please do let me know if there are any other steps that I am missing.

Thanks again for the help.

[jeugregg]

Yes, I still an issue, so I have to update with that : brew tap libmagic/users
And it worked for me.

[Freffles]

If you're on Windows (like me) and using python (like me) and have spent 2 hours messing around with this (like me) you'll be glad to know that this is the solution to your nighmares: https://pypi.org/project/python-magic-win64/.

You will need to change code to: from winmagic import magic

[sekharmalla]

Thanks @Freffles

I am not ok windows, I am on macbook, are there any workarounds?

[Freffles]

Sorry, I don't know about Mac. I'm sure there must be other methods that can be used, even just for the sake of completing this specific task. If all the files are html, just hard code that in.

I stuck at it because I'm playing around with the idea of different chunking strategies for different source data types. Even when I got magic working, I found it was not particularly accurate.