macos build instructions added

.gitignore

@@ -5,8 +5,7 @@
 **/.DS_Store
 
 # deployment cache
-build/
-dist/
-hooks-files/
-warn_processing.py
-VCToolbox.spec
+dist/build/
+dist/dist/
+dist/hooks/
+dist/VCToolbox.spec

README.md

@@ -12,6 +12,8 @@ Available OS releases:
 
 - MacOS (For now (1.05) release isn't done)
 
+If you want to distribute for other platforms follow instructions in `dist/BUILD.md`.
+
 ## Installation (Development build)
 
 1. Install Requirements (Python 3.7+ were tested ok)

dist/BUILD.md

@@ -0,0 +1,59 @@
+# Toolbox package building guide
+
+We use OS dependent utilite PyInstaller as package building tool. Result of steps below is a platform dependent for single application. It can be used directly on same platform as guide instructions are completed. For other platforms build on virtual machines or containers. [Read more about](https://pyinstaller.readthedocs.io/en/stable/usage.html) build tool.
+
+**Note:** using of python virtual environment is highly recommended:
+```
+# creation
+virtualenv ~/install_env
+source ~/install_env/bin/activate
+
+# after use (delete if needed)
+deactivate
+rm -rf ~/install_env
+```
+
+The procedure of distribution of this repo toolbox consists of several steps (steps performed from `dist/` directory).
+
+## MacOS
+
+1. Install all required packages from requirements.txt.
+```bash
+pip -r ../requirements.txt
+```
+
+2. Trying first build which will definetely fail, but provide us with important warning log.
+```bash
+pyinstaller --name="VCToolbox" --windowed --add-data="../config.yaml:./" --add-data="../datasets/*:datasets/" --hidden-import=typing_extensions -y --onefile ../app.py
+```
+
+**Note:** `--onefile` is optional and not fully supported option with PySide6 (main Qt for Python package used in toolbox).
+
+After command above, there would be two new directories ('build/' and 'dist/'). Actually we don't need 'dist/' folder, because it stores our releasing app, but generated 'build/VCToolbox/warn-VCToolbox.txt' will be used for next stages.
+
+3. Define installation hooks for not found modules. For this run `warn_processing.py` file:
+```bash
+python3 warn_preprocessing.py
+```
+
+This stage is cumbersome and depends on previous one. Some important modules that weren't found (such as librosa, etc) already added in script. After the command, there will be generated `hooks/` directory with files defining hooks for PyInstaller. Tree will look like:
+
+```
+.
+├── BUILD.md
+├── VCToolbox.spec
+├── build
+├── dist
+├── hooks
+└── warn_processing.py
+```
+
+4. Build with hooks for not found packages:
+```bash
+pyinstaller --name="VCToolbox" --windowed --hidden-import=typing_extensions -y --additional-hooks-dir=hooks --onefile ../app.py
+```
+
+If everything done right, there would be an executable file in dist/ folder.
+
+5.(optional) For distribution purposes move app binary into root folder, because it uses `config.yaml` and `datasets/` pathes. Then zip into archive, or use other utility for `.app` and `.dmg` creation.
+

dist/warn_processing.py

@@ -0,0 +1,87 @@
+### Define next the folder to create the hooks files and the warning file to read the modules from
+output_hooks_dir = 'hooks'
+warning_file = 'build/VCToolbox/warn-VCToolbox.txt'
+
+import re
+import os
+import shutil
+
+shutil.rmtree(output_hooks_dir, ignore_errors=True)
+os.makedirs(output_hooks_dir, exist_ok=True)
+
+with open(warning_file) as file:
+    files_content = file.readlines()
+
+clean_content = []
+for line in files_content:
+    if re.search('missing module named',line):
+        temp_line = re.sub('.*imported by ','',line)
+        temp_line = re.sub('\n',', ',temp_line)
+        clean_content.append(temp_line)
+clean_content = list(set(clean_content))
+joined_content = ''.join(clean_content)
+clean_content = list(set(joined_content.split('), ')))
+
+modules_toplevel = []
+for line in clean_content:
+    if re.search('top-level',line):
+        temp_mod = re.sub(' \(.*','',line)
+        temp_mod = re.sub('\..*','',temp_mod)
+        modules_toplevel.append(temp_mod)
+modules_toplevel = list(set(modules_toplevel))
+
+modules_conditional = []
+for line in clean_content:
+    if re.search('conditional',line):
+        temp_mod = re.sub(' \(.*','',line)
+        temp_mod = re.sub('\..*','',temp_mod)
+        modules_conditional.append(temp_mod)
+modules_conditional = list(set(modules_conditional))
+
+modules_delayed = []
+for line in clean_content:
+    if re.search('delayed',line):
+        temp_mod = re.sub(' \(.*','',line)
+        temp_mod = re.sub('\..*','',temp_mod)
+        modules_delayed.append(temp_mod)
+modules_delayed = list(set(modules_delayed))
+
+modules_optional = []
+for line in clean_content:
+    if re.search('optional',line):
+        temp_mod = re.sub(' \(.*','',line)
+        temp_mod = re.sub('\..*','',temp_mod)
+        modules_optional.append(temp_mod)
+modules_optional = list(set(modules_optional))
+
+all_modules = modules_toplevel + modules_conditional + modules_delayed + modules_optional
+all_modules = list(set(all_modules))
+
+print(all_modules)
+print('Number of found modules:', len(all_modules))
+
+### Optional: remove any of the modules
+remove_list = [
+    '/usr/local/lib/python3',
+    '/Users/sotomi/envs/pyinstaller-env/lib/python3',
+    'zipimport',
+    'test',
+]
+add_list = [
+    'sacremoses',
+    'librosa',
+]
+for pkg in remove_list:
+    if pkg in all_modules:
+        all_modules.remove(pkg)
+for pkg in add_list:
+    if pkg not in all_modules:
+        all_modules.append(pkg)
+
+print('Total number of requested modules:', len(all_modules))
+
+### Optional: Change all_modules by any of the other lists, e.g. modules_toplevel
+for module in all_modules:
+    output_content = 'from PyInstaller.utils.hooks import collect_all\n\ndatas, binaries, hiddenimports = collect_all(\''+module+'\')'
+    with open(output_hooks_dir+'/hook-'+str(module)+'.py', 'w') as f:
+        f.write(output_content)