Skip to content

A script that allows to download files (ex. PDFs) and videos (as mp4s and mp3s) from various sources (CSV, Trello,...) and platforms (YouTube, Vimeo,...).

Notifications You must be signed in to change notification settings

gnutix/videos-downloader

Repository files navigation

Videos Downloader

This script automates and simplify extracting resources (PDFs, audios/video files, ...) from a source (Trello board, CSV file, ...) and downloading them on your computer.

Scrutinizer Code Quality

Requirements

  1. Install youtube-dl python package : https://github.com/rg3/youtube-dl#installation
  2. Install Composer : https://getcomposer.org/download/
  3. Install the vendors :
composer install

Running the application

Copy the default configuration file and adapt the values as needed:

cp config/app.yml.dist config/app.yml

To check the configuration files validity, you can run :

bin/yaml-lint

To get a list of options for running the script, run :

bin/app --help

And finally to run the script itself :

bin/app

Congratulations! The downloads/ folder should be full of files now... ;)

Examples of configuration files

Here's one I use to download songs from my Trello repertoire board, along with tabs :

path_part:
    path: '/home/gnutix/Music/Repertoire'
    priority: -255

sources:
    -
        Extension\Trello\Trello:
            board_id: MkYHGxzY
            card_properties:
                - '[desc]'
                - '[attachments][%index%][url]'
processors:
    -
        Extension\File\File:
            extensions: '(?:pdf|mp3)'
    -
        Extension\YouTubeDl\YouTubeDl: ~

Here's another I use to download files from a website for which I had a paid account and extracted a list of URLs (in data.csv) containing the videos :

enabled: false

path_part:
    path: '/home/gnutix/Downloads/QLRR'
    priority: -255

sources:
    -
        Extension\CSV\CSV:
            base_url: 'https://qlrr.fr/6/qlrr/'
            resources:
                - '/home/gnutix/Downloads/QLRR/data.csv'
processors:
    -
        Extension\File\File:
            extensions: '(?:pdf|mp4)'
    -
        Extension\YouTubeDl\YouTubeDl:
            referer: 'https://qlrr.fr/6/'
            download_files:
                video: mp4

Some (mostly debug related) tips for analyzing the downloaded files

Display the number of files by extension :

export EXTENSION="mp3" # or mp4, pdf, ...
find downloads/ -name "*.${EXTENSION}" | wc -l

Display the sum of all files sizes by extension :

export EXTENSION="mp3" # or mp4, pdf, ...
find downloads/ -name "*.${EXTENSION}" -exec du -b {} \; | awk '{total+=$1}END{print total}' | numfmt --to=iec-i

List the 30 most heavy files by extension :

export EXTENSION="mp3" # or mp4, pdf, ...
ls -1Rs downloads/ | grep '.${EXTENSION}' | sed -e "s/^ *//" | grep "^[0-9]" | sort -nr | head -n30

About

A script that allows to download files (ex. PDFs) and videos (as mp4s and mp3s) from various sources (CSV, Trello,...) and platforms (YouTube, Vimeo,...).

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages