MB Utils

A collection of utility functions and tools to simplify common Python development tasks. This package provides various helper functions for logging, file operations, S3 interactions, and more.

Features

• Logging: Easy-to-use logging setup with file handlers
• File Operations: Path checking and validation
• Decorators: Useful decorators for deprecation warnings and retry logic
• Image Verification: Validate and verify image files
• S3 Integration: Simplified AWS S3 file and directory operations
• Utilities: Various helper functions including timing and batching

Installation

Install the package using pip:

pip install mb_utils

Usage

Logging

from mb_utils.src.logging import logger

logger.info("This is an info message")
logger.error("This is an error message")

Path Checking

from mb_utils.src.path_checker import check_path

if check_path("/path/to/check"):
    print("Path exists!")

Retry Decorator

from mb_utils.src.retry_decorator import retry

@retry(max_retries=3, delay=1)
def might_fail():
    pass

S3 Operations

from mb_utils.src.s3 import upload_file, download_file, upload_dir, download_dir

# Upload a single file
upload_file('local_file.txt', 'bucket-name', 'remote_file.txt')

# Download a file
download_file('bucket-name', 'remote_file.txt', 'local_file.txt')

Available Modules

Module	Description	Import Path
logging	Logger setup with file handlers	from mb_utils.src.logging import logger
path_checker	Path validation utilities	from mb_utils.src.path_checker import *
deprecated	Function deprecation decorator	from mb_utils.src.deprecated import deprecated_func
verify_image	Image verification	from mb_utils.src.verify_image import verify_image
retry_decorator	Retry mechanism for functions	from mb_utils.src.retry_decorator import retry
s3	AWS S3 operations	from mb_utils.src.s3 import *
extra	Additional utilities	from mb_utils.src.extra import *
profiler	Code profiling utilities	from mb_utils.src.profiler import *
video_extract	Video processing and frame extraction	from mb_utils.src.video_extract import *

Video Extraction

The video_extract module provides utilities for working with video files, including frame extraction and YouTube video downloading.

Extract Frames from Video

from mb_utils.src.video_extract import write_vid_to_img

# Extract frames from a video file
write_vid_to_img(
    url="/path/to/video.mp4",
    folder="./output_frames"  
)

Download YouTube Videos

from mb_utils.src.video_extract import download_yt_vid

# Download highest resolution
video_path = download_yt_vid(
    url="https://www.youtube.com/watch?v=example",
    folder="./videos",
    file_name="my_video.mp4",
    res='high'  # or 'low' for lowest resolution
)

USe mb_shorts/mb_ffmpeg for all YT/video processing

Profiling

The profiler module provides utilities for performance analysis of your Python code.

Function Profiling with SnakeViz

from mb_utils.src.profiler import run_with_snakeviz

@run_with_snakeviz(save_only=False)  # Opens SnakeViz automatically
def process_data(data):
    pass

# Or use as context manager
with run_with_snakeviz():
    pass

Line-by-Line Profiling

from mb_utils.src.profiler import line_profile

@line_profile
def process_item(item):
    result = item * 2
    return result

Simple Timing

from mb_utils.src.profiler import time_it

@time_it
def expensive_operation():
    pass

Memory Profiling

from mb_utils.src.profiler import MemoryProfiler

def process_large_data():
    with MemoryProfiler() as mem:
        data = [0] * 10**6

Comprehensive Profiling

from mb_utils.src.profiler import profile_function

@profile_function(
    enable_line_profiling=True,
    enable_memory_profiling=True
)
def analyze_data(data):
    # Will run with multiple profiling tools
    pass

Included Scripts

• verify_images_script: Utility script for image verification

License

This project is licensed under the MIT License - see the LICENSE file for details.