Data Science Projects with Keras

This repository contains a collection of data science projects that utilize artificial neural networks implemented in Keras. The projects are based on use cases from Udacity, Coursera, and Kaggle.

The repository also introduces a minimal package called ds_boost, initially implemented as a helper for this repository.

The repository is currently being used to update Python versions and packages (such as TensorFlow), test the effects on older code, and find out the best practices for migrating to newer versions of Python and packages. (e.g. TensorFlow 2.0+, pandas 2.0+, etc.)

This repository was initially developed in early 2018 and has since been updated with the best practices I've learned as a lead data scientist implementing solutions in a production environment. However, please note that this repository is being superseded by a new one that will contain LLM-based generative AI and containerized solutions. The new repository is currently in a private repository and will be published in the future. The new repository will be more class-oriented and will include smart ML-based models with explainability, confidence intervals, and generative AI-augmented dashboards.

Status September 2023: Currently updating the notebooks to Python 3.11, Pandas 2.1.1 and TensorFlow 2.13.0. The current pip package and most scenarios are still under development.

Scenarios

Classification models

• Enron Scandal Identifies Enron employees who may have committed fraud *

• Property Maintenance Fines Predicts the probability of a set of blight tickets to be paid on time

• Sentiment IMDB Predicts positive or negative sentiments from movie reviews (NLP)

• Spam detector Predicts the probability that a given email is a spam email (NLP)

• Student Admissions Predicts student admissions to graduate school at UCLA

• Titanic Predicts survival probabilities from the sinking of the RMS Titanic

Regression models

• Bike Rental Predicts daily bike rental ridership *

• House Prices Predicts house sales prices from Ames Housing database *

• Simple tickets Predicts the number of tickets requested by different clients

Recurrent models

• Machine Translation Translates sentences from English to French (NLP)

• Simple Stock Prediction Predicts Alphabet Inc. stock price

• Text generator Creates an English language sequence generator (NLP)

Social network models

• Network Predicts missing salaries and new email connections from a company's email network

* tested OK with Python 3.11, Pandas 2.1.1 and TensorFlow 2.13.0. The rest of the notebooks are being updated to the latest versions of Python and packages.

Setup & Usage

Python 3.11+ required

Basic Usage

1. Install the package developed as a helper for this repo:

pip install ds-boost

2. Download and Open the desired Jupyter Notebook

Development Mode

1. Clone the repository using git:

git clone https://github.com/angelmtenor/data-science-keras.git

2. Enter to the root path of the repo and use or create a new conda environment:

$ conda create -n dev python=3.11 -y && conda activate dev

3. In the root folder of the cloned repository, install all the required dev packages and the ds-boost mini package (Make required):

make setup

To install tensorflow with GPU support, follow the instructions of this guide: Install TensorFlow GPU.

Development Tools Required:

A Container/Machine with Conda, Git and Poetry as closely as defined in .devcontainer/Dockerfile:

• This Dockerfile contains a non-root user so the same configuration can be applied to a WSL Ubuntu Machine and any Debian/Ubuntu CLoud Machine (Vertex AI workbench, Azure VM ...)
• In case of having an Ubuntu/Debian machine with non-root user (e.g.: Ubuntu in WSL, Vertex AI VM ...), just install the tools from non-root user (no sudo)* section of .devcontainer/Dockerfile (sudo apt-get install <software> may be required)
• A pre-configured Cloud VM usually has Git and Conda pre-installed, those steps can be skipped
• The development container defined in .devcontainer/Dockerfile can be directly used for a fast setup (Docker required). With Visual Studio Code, just open the root folder of this repo, press F1 and select the option Dev Containers: Open Workspace in Container. The container will open the same workspace after the Docker Image is built.

Contributing

Check out the contributing guidelines. Here we encourage a simple manual execution of the pre-commit hooks before pushing any changes to the repository.

make qa

License

ds_boost was created by Angel Martinez-Tenor. It is licensed under the terms of the MIT license.

Credits

ds_boost was created from a Data Science Template developed by Angel Martinez-Tenor. The template was built upon py-pkgs-cookiecutter [template] (https://github.com/py-pkgs/py-pkgs-cookiecutter)