Skip to content

Commit f3dc137

Browse files
authoredOct 23, 2020
Bug patches and features
Changes being made: * Patch a handful of existing bugs present in the previous version of the script * Prevent the script from generating strm files for deleted items - deleted items will now be ignored. * Introduce a flag to be able to see realtime updates being pushed to the console (and being able to turn this off) * Introduce a flag to customise the title of the root directory being created by the script. * Update readme with the changes being made. * Additional minor changes to the readme to explain stuff in a more simple manner.
2 parents c8b2785 + f2c8b7b commit f3dc137

File tree

5 files changed

+358
-108
lines changed

5 files changed

+358
-108
lines changed
 

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# The virtual environment.
22
.venv/
3+
venv/
34

45
# Cache files stored by VS Code.
56
.vscode/

‎readme.md

+167-54
Original file line numberDiff line numberDiff line change
@@ -35,25 +35,33 @@
3535
<!-- TABLE OF CONTENTS -->
3636

3737
## Table of Contents
38-
- [Table of Contents](#table-of-contents)
3938
- [About The Project](#about-the-project)
4039
- [What is a strm file?](#what-is-a-strm-file)
4140
- [Pre-Requisites](#pre-requisites)
4241
- [Setup](#setup)
42+
- [What do the setup scripts do?](#what-do-the-setup-scripts-do)
4343
- [Usage](#usage)
4444
- [Generating strm files](#generating-strm-files)
45-
- [Where are the strm files stored?](#where-are-the-strm-files-stored)
45+
- [Where are the strm files placed?](#where-are-the-strm-files-placed)
4646
- [Custom Arguments](#custom-arguments)
47-
- [Scanning a particular folder](#scanning-a-particular-folder)
48-
- [Modifying the directory where strm files are generated](#modifying-the-directory-where-strm-files-are-generated)
47+
- [Scanning Selective Folders](#scanning-selective-folders)
48+
- [Custom Destination Directory](#custom-destination-directory)
49+
- [Realtime Updates](#realtime-updates)
50+
- [Custom Name for Root Directory](#custom-name-for-root-directory)
51+
- [Resources](#resources)
4952
- [Getting Folder ID's](#getting-folder-ids)
50-
- [Example; Using Custom Arguments](#example-using-custom-arguments)
51-
- [How is this script better than the existing add-on?](#how-is-this-script-better-than-the-existing-add-on)
52-
- [How does this script work](#how-does-this-script-work)
53+
- [Destination Directory vs Root Directory](#destination-directory-vs-root-directory)
54+
- [Examples](#examples)
55+
- [Custom destination](#custom-destination)
56+
- [Scanning a folder selectively](#scanning-a-folder-selectively)
57+
- [Custom root directory](#custom-root-directory)
58+
- [Miscellaneous Examples](#miscellaneous-examples)
59+
- [Is this script better than the existing add-on?](#is-this-script-better-than-the-existing-add-on)
60+
- [How does this script work](#how-does-this-script-work)
5361
- [Advanced Setup](#advanced-setup)
5462
- [Roadmap](#roadmap)
5563
- [A list of *possible* improvements;](#a-list-of-possible-improvements)
56-
- [Contributing](#contributing)
64+
- [Contributions](#contributions)
5765
- [License](#license)
5866
- [Acknowledgements](#acknowledgements)
5967
- [Just some fun](#just-some-fun)
@@ -65,13 +73,13 @@
6573

6674
A simple python script to complement the functionality of [Google Drive AddOn](https://kodi.tv/addon/music-add-ons-picture-add-ons-plugins-video-add-ons/google-drive) for Kodi.
6775

68-
For an add-on that [claims to be "*extremely fast*"](https://github.com/cguZZman/plugin.googledrive), I found the add-on to be quite slow (*and unreliable*). It crashes way too frequently, is slow, and gets stuck way too often (this happened frequently enough, to the point I decided to make this project just to solve this problem).
76+
For an add-on that [claims to be "*extremely fast*"](https://github.com/cguZZman/plugin.googledrive#google-drive-kodi-addon), I found the add-on to be quite slow (*and unreliable*). It crashes way too frequently, is slow, and gets stuck way too often (this happened frequently enough, to the point I decided to make this project just to solve this problem).
6977

7078
The main purpose of this project is to generate strm files for media file(s) present in a directory on Google Drive - and achieve this more reliably than the Kodi AddOn.
7179

7280
### What is a strm file?
7381

74-
In simple terms, *strm* files refer to files having an extension of `.strm`. They usually created and used by multimedia applications (Kodi and Plex being the main examples). They are used to store URL(s).
82+
In simple terms, *strm* files refer to files having an extension of `.strm`. They usually created and used by multimedia applications (Kodi and Plex being the main examples). They are used to store URL pointing to the actual media file(s) stored on in a remote server/cloud.
7583

7684
This URL is then used by the application to stream the actual media file when required.
7785

@@ -84,30 +92,31 @@ This URL is then used by the application to stream the actual media file when re
8492
## Setup
8593
1. Create a [Google Project](http://console.developers.google.com/) and [enable Google Drive API](https://developers.google.com/drive/api/v3/enable-drive-api).
8694
2. Once you enable the Drive API for your project, setup credentials required to use this API.
87-
3. Download the credentials as a JSON file and rename the file as `credentials.json`.
88-
4. Move the `credentials,json` file into the same directory containing the python script.
95+
3. Download the credentials as a JSON file and rename this file as `credentials.json`.
96+
4. Move the `credentials.json` file into the same directory containing the python script.
8997
5. Install the python dependencies.
9098
<br> `pip install -r requirements.txt`
9199
6. Install [Kodi AddOn for Google Drive](https://kodi.tv/addon/music-add-ons-picture-add-ons-plugins-video-add-ons/google-drive) in your Kodi installation.
92100
7. Login to the add-on. Make sure that you use the same Google account while logging into this script and the Google add-on.
93101

94102
Alternatively, for the fourth step, you can directly use the `setup.sh` or `setup.bat` scripts and have them create an efficient setup for you.
95103

96-
> #### What do the setup scripts do?
97-
>
98-
> The setup script will;
99-
> * Update `pip`
100-
> * Install `virtualenv`
101-
> * Create a virtual environment named `.venv`
102-
> * Install required packages into this virtual environment.
104+
#### What do the setup scripts do?
103105

104-
While anyone is free to use the direct setup files, do note that they are primarily for my use (to quickly setup a system for usage/testing/debugging) - and are not guaranteed to work in your system.
106+
The setup script will;
107+
* Update `pip`
108+
* Install `virtualenv`
109+
* Create a virtual environment named `.venv`
110+
* Activate this virtual package
111+
* Install required python packages into this virtual environment.
112+
113+
While anyone is free to use the direct setup files, do note that they are primarily for my use (to quickly setup a system for usage/testing/debugging) - and are *not guaranteed* to work in your system.
105114

106115
## Usage
107116

108117
Executing the python script directly (without custom arguments) will fetch a list of all teamdrives to which the account has access -- allowing you to select a teamdrive from this list which will then be scanned for media files.
109118

110-
Alternatively, you can pass [custom arguments](#custom-arguments) to the script to scan a particular folder, or to modify the directory where the strm files are stored after being generated.
119+
Alternatively, you can pass [custom arguments](#custom-arguments) to the script to modify the default behaviour.
111120

112121
Once the script finishes generating strm files after a scan, add the resulting [directory as a source](https://kodi.wiki/view/Adding_video_sources) in your Kodi installation. From there on, Kodi will treat these `.strm` files as actual media files and will be able to scan/play them normally.
113122

@@ -121,66 +130,170 @@ Once you're comfortable with the usage of this script, you might want to take a
121130

122131
Running the python script without any custom arguments will fetch a list of all team-drives that are connected to the account, selecting a teamdrive from this list will make the script scan the contents of the particular teamdrive.
123132

124-
### Where are the strm files stored?
133+
### Where are the strm files placed?
125134

126135
By default, the strm files generated after a scan are stored in the **working directory**. Use `pwd` in Unix-based systems, or `cd` in Windows get the location of current working directory.
127136

128-
Look for a directory with the **same name as the source** (the Google folder/teamdrive) scanned.
137+
By deafult, the strm files are placed inside a directory with the **same name as the source** (the Google folder/teamdrive) scanned.
138+
139+
Note: Take a look at the flags for [custom destination directory](#custom-destination-directory) and [custom root directory](#custom-name-for-root-directory) to modify the directory inside which strm files are being generated.
129140

130141
## Custom Arguments
131142

132-
Using custom arguments, you can control the folder that is being scanned, and/or modify the directory in which the strm files are being placed after a scan.
143+
Using custom arguments, you can control the folder that is being scanned, modify the directory in which the strm files are being placed after a scan, modify the name of the root directory being used, choose if you want the script to run in silent mode or not and more.
144+
145+
Note: These flags can be passed only if the script is being executed from the terminal.
146+
147+
#### Scanning Selective Folders
148+
149+
**Flag:** `--source=<folder-id>`
150+
**Value Expected:** ID of an existing folder on Google Drive.
151+
152+
By default, the script will prompt you to select a teamdrive as the source - the selected teamdrive will be completely scanned by the script. This might not always be desired.
153+
154+
This flag allows you to scan the contents of any particular directory as needed, i.e. with the help of this flag instead of scanning a complete teamdrive (or the main drive), an individual folder can be selectively scanned using its ID.
155+
156+
> Note: Check [Getting the Folder ID](#getting-folder-ids) section to get the ID of a folder in Google Drive.
157+
158+
Side Note: Drive API treats teamdrives as folders, meaning that this flag can also be used to scan a teamdrive with it's ID.
159+
160+
[>> Using this Flag; Example](#scanning-a-folder-selectively)
161+
162+
#### Custom Destination Directory
163+
164+
**Flag:** `--dest="</path/to/destination>"`
165+
**Expected Value:** Path to an ***existing*** directory.
133166

134-
This enables scanning a particular folder in a teamdrive instead of scanning the entire teamdrive, or to scan the contents of the *My Drive* section.
167+
This flag is used to decide the destination directory in which the root directory (containing the strm files) will be placed.
135168

136-
Note: These flags can only be passed if you execute the script from the terminal.
169+
[>> Resource: Destination Directory vs Root Directory](#destination-directory-vs-root-directory)
137170

138-
### Scanning a particular folder
171+
Make sure that path supplied as a value with this flag belongs to an **existing directory**. If the path contains spaces, wrap it inside double quotes.
139172

140-
Add `--source=<folder-id>` flag while executing the python script to force it to scan a particular directory. [Example](#example-using-custom-arguments)
173+
[>> Using this Flag; Example](#custom-destination)
141174

142-
> Note: Check [Getting the Folder ID](#getting-folder-ids) section to get the folder id for a particular folder in Google Drive.
175+
#### Realtime Updates
143176

177+
**Flag:** `--updates=<value>`
178+
**Expected Value:** `on` OR `off`
144179

145-
### Modifying the directory where strm files are generated
180+
This is an optional flag to see real-time progress as the script is scanning items from the source. The value of `off` implies no updates are to be printed to the console - effectively running the script silently.
146181

147-
Add `--dest=<destination>` flag while executing the python script to store the generated strm files in a particular directory. Make sure that the destination points to an existing directory.
182+
Having realtime updates might not always be desirable (especially if script is being run in the backgrond - me for one). Setting the value of this flag to `off` will run the script in silent mode - under silent mode, the only time this script will print to the console will be to notify that a scan has completed.
148183

149-
Inside the destination directory, this script will make a directory with the same name as the source folder on Google Drive. If the destination contains a space, wrap it inside the double quotes before adding it as a flag.
184+
By default, the value of this flag is `on`, i.e. realtime progress will be displayed on the console.
185+
186+
[>> Using this Flag; Example](#miscellaneous-examples)
187+
188+
#### Custom Name for Root Directory
189+
190+
**Flag:** `--rootname=<root-directory>`
191+
**Expected Value:** Name for the root directory
192+
193+
Optional flag to modify the name of root directory that is created by this script during runtime.
194+
195+
[>> Resource: Destination Directory vs Root Directory](#destination-directory-vs-root-directory)
196+
197+
Be default, the name of the teamdrive/folder being scanned will be used as the name of the root directory.
198+
199+
Important: If a directory with the same name as the root directory already exists inside destination directory, this script will ***erase the existing root directory***.
200+
201+
[>> Using this Flag; Example](#custom-root-directory)
202+
203+
## Resources
150204

151205
#### Getting Folder ID's
152206

153207
You can get the id for a particular folder by opening the folder and copying the id from the url.
154208

155209
For example, in the URL `https://drive.google.com/drive/folders/0AOC6NXsE2KJMUk9PTA`, the folder-id is `0AOC6NXsE2KJMUk9PTA`.
156210

157-
This method can also be used to get the ID of a particular TeamDrive and use it as a [custom argument](#scanning-a-particular-folder) to directly scan the teamdrive.
211+
This method can also be used to get the ID of a particular TeamDrive and use it as a [custom argument](#scanning-selective-folders) to directly scan the teamdrive.
212+
213+
214+
#### Destination Directory vs Root Directory
215+
216+
In basic terms, when the script scans a folder on drive, it creates a directory which will contain all the strm files being generated.
158217

159-
#### Example; Using Custom Arguments
160-
161-
Running the following command will scan the folder `0AOC6NXsE2KJMUk9PVA` and generate strm files inside the directory `/home/kodi libraries` instead of resorting to the defaults.
218+
This directory that has been created by the script, during the runtime is the ***Root Directory***. The parent directory that contains the root directory is termed as the ***Destination Directory***.
162219

163-
`python strm-generator.py --source=0AOC6NXsE2KJMUk9PVA --dest="/home/kodi libraries"`
220+
The path being used as the parent/destination directory should point to an already existing directory. On the other hand, if a directory with the same name as root directory is already present inside the destination, it will be wiped off.
221+
222+
As an example, take a look at this directory structure;
223+
224+
``` bash
225+
│── E:
226+
│ ├── Kodi Library
227+
│ │ ├── Downloaded Media
228+
│ │ │ ├── new-file.mp4.strm
229+
│ │ │ ├── old-file-02.mp4.strm
230+
│ │ ├── Collection
231+
│ │ │ ├── movie-file.mkv.strm
232+
│ ├── STRM Files
233+
│ │ ├── Content Media
234+
│ │ │ ├── movie-01.mkv.strm
235+
│ │ │ ├── movie-02.mp4.strm
236+
│ │ │ ├── trailer.mkv.strm
237+
```
238+
239+
From above, strm files are present inside three directories, namely, `Downloaded Media`, `Collection` and `Content Media` - these are the *Root Directories*. The directories containing these root directories then become the `Destination Directory`. Meaning that, there are two destination directories, `Kodi Library` and `STRM Files`.
240+
241+
242+
## Examples
243+
244+
#### Custom destination
245+
Running the following command will generate the root directory (containing strm files) at the destination `/home/kodi library`. Make sure that this is a valid path and points to a directory.
246+
247+
`python strm-generator.py --dest="/home/kodi library"`
248+
249+
Note that since the path contains space, it has been wrapped in double spaces.
250+
251+
#### Scanning a folder selectively
252+
253+
Running the following command will scan a folder with the ID `0AOC6NXsE2KJMUk9PVA` on Google Drive. Make sure that the account you signed into the script with has access to this folder.
254+
255+
`python strm-generator.py --source=0AOC6NXsE2KJMUk9PVA --dest="/home/kodi library"`
256+
257+
Note the additional destination flag in the command, instead of placing the results in the working directory, the script will now place them inside `/home/kodi library`.
258+
259+
#### Custom root directory
260+
261+
Running the following command will generate strm files inside a directory named `new root directory`
262+
263+
`python strm-generator.py --source=0AOC6NXsE2KJMUk9PVA --dest="/home/kodi library" --rootname="new root directory"`
264+
265+
The strm files will be present in a directory named `new root directory` (this is the root directory). The root directory by itself will be present inside `"/home/kodi library"` (this being the destination directory).
266+
267+
### Miscellaneous Examples
268+
269+
* Scanning a particular folder on drive, with custom destination and root directories, plus no updates to the console (using all the flags at once).
270+
> `python strm-generator.py --source=0AOC6NXsE2KJMUk9PVA --dest="/home/kodi libraries" --rootname="Staging; Media" --updates=off`
271+
272+
* Scanning a particular folder with no updates
273+
> `python strm-generator.py --source=0AOC6NXsE2KJMUk9PVA --updates=off`
274+
275+
* Scanning a folder selectively with custom root directory
276+
> `python strm-generator.py --source=0AOC6NXsE2KJMUk9PVA --rootname="Media"`
277+
278+
* Scanning a folder with custom destination directory and no updates on console
279+
> `python strm-generator.py --source=0AOC6NXsE2KJMUk9PVA --dest="/home/Videos" --updates=off`
164280
165-
If you're unsure about using custom arguments, feel free to just replace the values in the command above with **valid** values, and you're good to go!
166281

167282
---
168283

169-
### How is this script better than the existing add-on?
284+
## Is this script better than the existing add-on?
170285

171-
Based on the tests I have run upto now, this script didn't crash or lag, and was quite a bit faster than the add-on. Honestly, I'm not sure about why this happens. While the add-on is [open source](https://github.com/cguZZman/plugin.googledrive), I'm yet to take a look at the source code and determine why.
286+
Based on the tests I have run, this script didn't crash or lag, and was quite a bit faster than the add-on. Honestly, I'm not sure why this happens - while the Google Drive add-on is [open source](https://github.com/cguZZman/plugin.googledrive), I'm yet to take a proper look at it.
172287

173288
What I do know is, this script is as simple as it can be. There is no clever hack, or any trick present. Yet, all the tests I ran prove that this script is somehow better than the add-on (and no, this is **definitely not** because of any hardware limitation at my end).
174289

175290
It is possible that this difference is being caused by Kodi (or the integration between the add-on and Kodi). Regardless, I noticed a flaw, and wrote a simple script to overcome it effectively.
176291

177-
## How does this script work
292+
### How does this script work
178293

179294
The basic functioning of this script revolves around traversing a directory on Google Drive, iterating through each file present in this directory and generating an equivalent `.strm` file for every media file. An important part of this is to be able to recognize media files.
180295

181-
The most important step while creating a strm file is to ensure that the contents of the file are stored in such a manner that they can be parsed by the Google Drive add-on.
182-
183-
The last step above ensures that this script does not need to handle the part of using these strm files to stream the video. Once this script generates the strm files, everything else is being handled by the Kodi add-on, this includes parsing the strm file to get the actual URL, streaming contents from the URL and everything else.
296+
The most important step while creating a strm file is to ensure that the contents of the file are stored in such a manner that they can be parsed by the Google Drive add-on. This ensures that this script does not need to handle the part of using these strm files to stream the video. Once an strm file has been generated, everything else is being handled by the Kodi add-on, this includes parsing the strm file to get the actual URL and streaming media from the URL.
184297

185298
## Advanced Setup
186299

@@ -194,9 +307,7 @@ Whenever I open up Kodi, it automatically scrapes the new files, and adds them t
194307

195308
If needed, the systemd service can be modified to run the script multiple times, scanning different sources each time to be able to scale my current setup to span across multiple teamdrives/folders, all this without requiring any sort of input.
196309

197-
Windows users can achieve the same functionality as systemd.service using [Windows Task Scheduler](https://en.wikipedia.org/wiki/Windows_Task_Scheduler).
198-
199-
In case someone wants to replicate a similar setup as mine, here are some useful links to help you get started.
310+
Windows users can achieve the same functionality as systemd.service using [Windows Task Scheduler](https://en.wikipedia.org/wiki/Windows_Task_Scheduler). In case someone wants to replicate a similar setup as mine, here are some useful links to help you get started.
200311

201312
- [Creating a systemd.service](https://medium.com/@benmorel/creating-a-linux-service-with-systemd-611b5c8b91d6)
202313
- [Using Windows Task Scheduler](https://windowsreport.com/schedule-tasks-windows-10)
@@ -213,10 +324,11 @@ Based on the tests I've run, this script is ~20% faster than the Google Drive Ad
213324
> *For the curious, yes, I ran the test on the exact same source without any modifications. The results are as accurate as possible.*
214325
215326
#### A list of *possible* improvements;
216-
- Improvement(s) on the GUI - updating progress on the terminal while the script is running.
217-
- Multithreading (still not sure on this one) - despite GIL, the execution speed can be improved to some extent.
327+
- Improvement(s) on the GUI - ~~updating progress on the terminal while the script is running~~ added in [e451e08
328+
](https://github.com/demon-rem/kodi-strm/commit/e451e087adcc3bc116f1718e0bc5a9100b81555a).
329+
- Multithreading (still not sure on this one) - despite GIL, the execution speed can be improved to *some* extent.
218330

219-
## Contributing
331+
## Contributions
220332

221333
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions made are **extremely appreciated**.
222334

@@ -230,9 +342,10 @@ Contributions are what make the open source community such an amazing place to l
230342
Distributed under the MIT License. See [`LICENSE`](./LICENSE). for more information.
231343

232344
## Acknowledgements
233-
* [Img Shields](https://shields.io)
234-
* [Google API Python Client](https://github.com/googleapis/google-api-python-client)
235-
* [Coloroma](https://github.com/tartley/colorama)
345+
* [Img Shields](https://shields.io): for the badges being used in this readme.
346+
* [Google API Python Client](https://github.com/googleapis/google-api-python-client): a python wrapper to work with Drive API.
347+
* [Coloroma](https://github.com/tartley/colorama): Used to print the names of teamdrives in separate colors while selecting a source in interactive mode.
348+
* [Reprint](https://github.com/Yinzo/reprint): Used to provide updates on the console.
236349
* Special credits to [Satan](https://github.com/not-satan) for being annoying enough to force me to work on this repo as a priority.
237350

238351
<br>
@@ -246,4 +359,4 @@ Distributed under the MIT License. See [`LICENSE`](./LICENSE). for more informat
246359
[license]: https://img.shields.io/github/license/demon-rem/kodi-strm?style=for-the-badge
247360
[latest-release]: https://img.shields.io/github/v/release/demon-rem/kodi-strm?style=for-the-badge
248361
[release-date]: https://img.shields.io/github/release-date/demon-rem/kodi-strm?style=for-the-badge
249-
[issues-url]: https://img.shields.io/github/issues-raw/demon-rem/kodi-strm?style=for-the-badge
362+
[issues-url]: https://img.shields.io/github/issues-raw/demon-rem/kodi-strm?style=for-the-badge

‎requirements.txt

+2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
astroid==2.4.2
22
autopep8==1.5.4
3+
backports.shutil-get-terminal-size==1.0.0
34
cachetools==4.1.1
45
certifi==2020.6.20
56
chardet==3.0.4
@@ -22,6 +23,7 @@ pyasn1-modules==0.2.8
2223
pycodestyle==2.6.0
2324
pylint==2.6.0
2425
pytz==2020.1
26+
reprint==0.5.2
2527
requests==2.24.0
2628
requests-oauthlib==1.3.0
2729
rsa==4.6

‎setup.bat

+11-9
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,23 @@
1-
Upgrading pip
1+
@echo off
2+
3+
echo Upgrading pip
24
python -m pip install --upgrade pip
35

4-
Installing virtual-environment
6+
echo Installing virtual-environment
57
pip install --upgrade virtualenv
68

7-
Creating a virutal environment
9+
echo Creating a virutal environment
810
virtualenv .venv
911

1012
REM Activating the virtual environment.
1113
.venv\Scripts\activate
1214

13-
Installing the required packages
15+
echo Installing the required packages
1416
pip install -r requirements.txt
1517

16-
Setup executed successfully.
17-
18-
Activate the virtual-environment using
19-
`.venv\Scripts\activate`
18+
echo Setup executed successfully.
2019

21-
Once done, follow the rest of instructions in the setup.
20+
echo Activate the virtual-environment using
21+
echo `.venv\Scripts\activate`
22+
echo(
23+
echo Once done, follow the rest of instructions in the setup.

‎strm-generator.py

+177-45
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
import sys
44
from os import getcwd, mkdir, system
5-
from shutil import rmtree
6-
from os.path import exists, isdir, join
5+
from os.path import basename, dirname, exists, isdir, join
76
from pickle import dump as dump_pickle
87
from pickle import load as load_pickle
98
from re import match, sub
9+
from shutil import rmtree
1010
from time import sleep
1111
from typing import Dict, List, Optional
1212

@@ -15,9 +15,12 @@
1515
from google.oauth2.credentials import Credentials
1616
from google_auth_oauthlib.flow import InstalledAppFlow
1717
from googleapiclient.discovery import Resource, build
18+
from reprint import output
1819

19-
files = 0
20-
directories = 0
20+
files_scanned = 0
21+
directories_scanned = 0
22+
files_skipped = 0
23+
bytes_scanned = 0
2124

2225

2326
def authenticate() -> Resource:
@@ -60,6 +63,42 @@ def authenticate() -> Resource:
6063
return service
6164

6265

66+
def update(files: int, directories: int, skipped: int, size: int, out_stream):
67+
"""
68+
Prints updates to the screen.
69+
70+
Parameters
71+
-----------
72+
files: Integer containing the number of files that have been scanned. \n
73+
directories: Integer containing number of directories that have been scanned. \n
74+
skipped: Integer containing the number of files that have been skipped. \n
75+
size: Integer containing the raw size of files scanned (in bytes). \n
76+
out_stream: A dictionary object to which new lines are to be written as needed. \n
77+
"""
78+
79+
# The value of `size` will be an integer containing the raw size of the file(s) traversed
80+
# in bytes. Starting by converting this into a readable format.
81+
82+
# An array size units. Will be used to convert raw size into a readble format.
83+
sizes = ['B', 'KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB']
84+
85+
counter = 0
86+
while size >= 1024:
87+
size /= 1024
88+
counter += 1
89+
90+
# Forming a string containing readable size, this will be used to directly convert th e
91+
readable_size = '{:.3f} {}'.format(
92+
size,
93+
sizes[counter]
94+
)
95+
96+
out_stream[2] = f'Directories Scanned: {directories}'
97+
out_stream[3] = f'Files Scanned: {files}'
98+
out_stream[4] = f'Bytes Scanned: {readable_size}'
99+
out_stream[5] = f'Files Skipped: {skipped}'
100+
101+
63102
def select_teamdrive(service: Resource) -> str:
64103
"""
65104
Allows the user to select the teamdrive for which strm files are to be generated.
@@ -140,7 +179,32 @@ def select_teamdrive(service: Resource) -> str:
140179
print(f'\t{Fore.RED}Incorrect input detected. {Style.RESET_ALL}\n')
141180

142181

143-
def walk(origin_id: str, service: Resource, cur_path: str, item_details: Dict[str, str]):
182+
def shrink_path(full_path: str, max_len: int = 70) -> str:
183+
"""
184+
Shrinks the path name to fit into a fixed number of characters.
185+
186+
Parameters
187+
-----------
188+
full_path: String containing the full path that is to be printed. \n
189+
max_len: Integer containing the maximum length for the final path. Should be
190+
more than 10 characters. Default: 15 \n
191+
192+
Returns
193+
--------
194+
String containing path after shrinking it. Will be atmost `max_len` characters
195+
in length.
196+
"""
197+
198+
if len(full_path) <= max_len:
199+
# Directly return the path if it fits within the maximum length allowed.
200+
return full_path
201+
202+
allowed_len = max_len - 6
203+
return f'{full_path[:int(allowed_len / 2)]}......{full_path[-int(allowed_len / 2):]}'
204+
205+
206+
def walk(origin_id: str, service: Resource, orig_path: str, item_details: Dict[str, str], out_stream,
207+
push_updates: bool, drive_path='~'):
144208
"""
145209
Traverses directories in Google Drive and replicates the file/folder structure similar to
146210
Google Drive.
@@ -156,57 +220,97 @@ def walk(origin_id: str, service: Resource, cur_path: str, item_details: Dict[st
156220
Every directory present inside this directory will be traversed, and a `.strm` file will be generated
157221
for every video file present inside this directory. \n
158222
service: Instance of `Resource` object used to interact with Google Drive API. \n
223+
orig_path: Path to the directory in which strm files are to be placed once generated. This directory
224+
will be made by THIS method internally. \n
225+
item_details: Dictionary containing details of the directory being scanned from Google drive. \n
226+
out_stream: Dictioanry object to which the output is to be written to once (if updates are needed). \n
227+
push_updates: Boolean indicating if updates are to be pushed to the screen or not. \n
159228
"""
160229

161-
global files, directories
230+
global files_scanned, directories_scanned, bytes_scanned, files_skipped
162231

163232
if not isinstance(origin_id, str) or not isinstance(service, Resource):
164233
raise TypeError('Unexpected argument type')
165234

166-
# print(item_details)
167-
if item_details['mimeType'] != 'application/vnd.google-apps.folder':
168-
raise TypeError('Expected the id for a directory')
169-
170235
# Updating the current path to be inside the path where this directory is to be created.
171-
cur_path = join(cur_path, item_details['name'])
236+
cur_path = join(orig_path, item_details['name'])
172237

173-
# Creating the root directory for this search.
174-
try:
175-
mkdir(cur_path)
176-
except:
177-
pass
238+
# Creating the root directory.
239+
mkdir(cur_path)
178240

179241
page_token = None
180242

243+
if push_updates:
244+
out_stream[0] = f'Scanning Directory: {shrink_path(drive_path)}/'
245+
out_stream[1] = '\n' # Blank line
246+
181247
while True:
182248
result = service.files().list(
249+
# Getting the maximum number of items avaiable in a single API call
250+
# to reduce the calls required.
183251
pageSize=1000,
184252
pageToken=page_token,
185-
fields='files(name, id, mimeType, teamDriveId)',
253+
254+
# The fields that are to be included in the response.
255+
fields='files(name, id, mimeType, teamDriveId, size)',
256+
257+
# Getting itesm from all drives, this allows scanning team-drives too.
186258
supportsAllDrives=True,
187259
includeItemsFromAllDrives=True,
188-
q=f"'{origin_id}' in parents"
260+
261+
# Skipping trashed files and directories
262+
q=f"'{origin_id}' in parents and trashed=false"
189263
).execute()
190264

191-
# print(f'Found {len(result["files"])}')
192265
for item in result['files']:
193266
if item['mimeType'] == 'application/vnd.google-apps.folder':
194-
# If the current object is a folder, recursively calling the same method.
195-
directories += 1
196-
walk(item['id'], service, cur_path, item)
267+
# If the current object is a folder, incrementing the folder count and recursively
268+
# calling the same method over the new directory encountered.
269+
directories_scanned += 1
270+
271+
walk(
272+
origin_id=item['id'],
273+
service=service,
274+
orig_path=cur_path,
275+
item_details=item,
276+
out_stream=out_stream,
277+
push_updates=push_updates,
278+
drive_path=f'{join(drive_path, item["name"])}'
279+
)
197280
elif 'video' in item['mimeType'] or match(r'.*\.(mkv|mp4)$', item['name']):
198-
try:
199-
file_content = f'plugin://plugin.googledrive/?action=play&item_id={item["id"]}'
200-
if 'teamDriveId' in item:
201-
# Adding this part only for items present in a teamdrive.
202-
file_content += f'&item_driveid={item["teamDriveId"]}&teamDriveId={item["teamDriveId"]}'
203-
204-
file_content += f'&content_type=video'
205-
with open(join(cur_path, item['name']+'.strm'), 'w+') as f:
206-
f.write(file_content)
207-
files += 1
208-
except:
209-
pass
281+
# Scanning the file, and creating an equivalent strm file if the file is a media file
282+
# Since the mime-type of files in drive can be modified externally, scanning the file
283+
# as a media file if the file has an extension of `.mp4` or `.mkv`.
284+
285+
# Forming the string that is to be placed inside the strm file to ensure that the file
286+
# can be used by the drive add-on.
287+
file_content = f'plugin://plugin.googledrive/?action=play&item_id={item["id"]}'
288+
if 'teamDriveId' in item:
289+
# Adding this part only for items present in a teamdrive.
290+
file_content += f'&item_driveid={item["teamDriveId"]}&teamDriveId={item["teamDriveId"]}'
291+
292+
file_content += f'&content_type=video'
293+
with open(join(cur_path, item['name']+'.strm'), 'w+') as f:
294+
f.write(file_content)
295+
296+
# Updating the counter for files scanned as well as bytes scanned.
297+
files_scanned += 1
298+
bytes_scanned += int(item['size'])
299+
else:
300+
# Skipping the file if the file is not a video file (i.e. mime type does not match), and the
301+
# file does not have an extension of `.mp4` or `.mkv`. Updating the counter to increment the
302+
# number of files that have been skipped from the scan.
303+
files_skipped += 1
304+
305+
if push_updates:
306+
# Updating counter on the screen if updates are to be pushed to the screen.
307+
update(
308+
files=files_scanned,
309+
directories=directories_scanned,
310+
skipped=files_skipped,
311+
size=bytes_scanned,
312+
out_stream=out_stream
313+
)
210314

211315
if 'nextPageToken' not in result:
212316
break
@@ -218,11 +322,17 @@ def walk(origin_id: str, service: Resource, cur_path: str, item_details: Dict[st
218322

219323
destination = getcwd()
220324
source = None
325+
updates = True # True by default.
326+
dir_name = None # The name of the directory to store the strm files in.
221327

222328
# Pattern(s) that are to be used to match against the source argument. The group is important since
223329
# this pattern is also being used to extract the value from argument.
224330
pattern_source = r'^--source=(.*)'
225-
patter_dest = r'^--dest="?(.*)"?'
331+
pattern_output = r'^--updates="?(off|on)"?$'
332+
333+
# TODO: Might want to differentiate between platforms here. Especially for custom directories.
334+
pattern_custom_dir = r'^--rootname="?(.*)"?$'
335+
pattern_dest = r'^--dest="?(.*)"?$'
226336

227337
# Looping over all arguments that are passed to the script. The first (zeroe-th) value shall be the
228338
# name of the python script.
@@ -239,15 +349,22 @@ def walk(origin_id: str, service: Resource, cur_path: str, item_details: Dict[st
239349

240350
# Extracting id from the argument using substitution. Substituting everything from the
241351
# argument string except for the value :p
242-
source = sub(pattern_source, r'\1', sys.argv[i])
243-
elif match(patter_dest, sys.argv[i]):
352+
source = match(pattern_source, sys.argv[i]).groups()[0]
353+
elif match(pattern_dest, sys.argv[i]):
244354
# Again, extracting the value using regex substitution.
245-
destination = sub(patter_dest, r'\1', sys.argv[i])
355+
destination = match(pattern_dest, sys.argv[i]).groups()[0]
246356

247357
if not isdir(destination):
248358
print(f'Error: `{sys.argv[i]}` is not a directory.\n')
249359
exit(10) # Force quit.
250-
360+
elif match(pattern_output, sys.argv[i]):
361+
# Switching the updates off if the argument has been passed.
362+
# Since the default value is to allow updates, no change is required incase
363+
# updates are explicitly being allowed.
364+
if match(pattern_output, sys.argv[i]).groups()[0] == 'off':
365+
updates = False
366+
elif match(pattern_custom_dir, sys.argv[i]):
367+
dir_name = match(pattern_custom_dir, sys.argv[i]).groups()[0]
251368
else:
252369
print(f'Unknown argument detected `{sys.argv[i]}`')
253370
exit(10) # Non-zero exit code to indicate error.
@@ -265,17 +382,32 @@ def walk(origin_id: str, service: Resource, cur_path: str, item_details: Dict[st
265382
if 'teamDriveId' in item_details and item_details['id'] == item_details['teamDriveId']:
266383
# If the source is a teamdrive, attempting to fetch details for the teamdrive instead.
267384
item_details = service.drives().get(
268-
teamDriveId=item_details['teamDriveId']
385+
driveId=item_details['teamDriveId']
269386
).execute()
270387

388+
if dir_name is not None:
389+
# If the name of the root directory is not set, using the name of the teamdrive/drive.
390+
item_details['name'] = dir_name
391+
271392
# Clearing the destination directory (if it exists).
272-
final_path=join(destination, item_details['name'])
393+
final_path = join(destination, dir_name)
273394
if isdir(final_path):
274395
rmtree(final_path)
275396

276-
print(f'\n\tSaving the output at `{destination}`')
397+
print() # Empty print.
277398

278399
# Calling the method to walk through the drive directory.
279-
walk(source, service, destination, item_details)
280-
281-
print(f'Completed. Traversed through {directories} directories and {files} files.')
400+
with output(output_type='list', initial_len=6, interval=0) as out_stream:
401+
# Creating the output object here to ensure that the same object is being used
402+
# for updates internally.
403+
404+
walk(
405+
origin_id=source,
406+
service=service,
407+
orig_path=destination,
408+
item_details=item_details,
409+
out_stream=out_stream,
410+
push_updates=updates
411+
)
412+
413+
print(f'\n\tTask completed. Saved the output at `{final_path}`')

0 commit comments

Comments
 (0)
Please sign in to comment.