Skip to content

Sion1225/Study-Deeplearning-NLP

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

130 Commits
.ipynb_checkpoints
.ipynb_checkpoints
 
 
DataSet
DataSet
 
 
Models
Models
 
 
Presentation
Presentation
 
 
Attention mechanism with BiLSTM.ipynb
Attention mechanism with BiLSTM.ipynb
 
 
Auto Differentiation&Linear Regression.ipynb
Auto Differentiation&Linear Regression.ipynb
 
 
BERT_Study.ipynb
BERT_Study.ipynb
 
 
BIO NER using BiLSTM & F1-score.ipynb
BIO NER using BiLSTM & F1-score.ipynb
 
 
BIO_tagging.ipynb
BIO_tagging.ipynb
 
 
Bag of Words.ipynb
Bag of Words.ipynb
 
 
Bilingual Evaluation Understudy Score.ipynb
Bilingual Evaluation Understudy Score.ipynb
 
 
Book Recommendation System using Document Embedding.ipynb
Book Recommendation System using Document Embedding.ipynb
 
 
Byte Pair Encoding.ipynb
Byte Pair Encoding.ipynb
 
 
Character-Level Neura Machine Translation with Seq2Seq.ipynb
Character-Level Neura Machine Translation with Seq2Seq.ipynb
 
 
Character_level_RNNLM.ipynb
Character_level_RNNLM.ipynb
 
 
Cosine Similarity.ipynb
Cosine Similarity.ipynb
 
 
Embedding Visualization.ipynb
Embedding Visualization.ipynb
 
 
IMDB Movie Review Sentiment Analysis (Binary).ipynb
IMDB Movie Review Sentiment Analysis (Binary).ipynb
 
 
IMDB Sentiment Classification with 1D CNN.ipynb
IMDB Sentiment Classification with 1D CNN.ipynb
 
 
IntegerEncording&Padding&One_Hot_Encording.ipynb
IntegerEncording&Padding&One_Hot_Encording.ipynb
 
 
Keras Word Embedding.ipynb
Keras Word Embedding.ipynb
 
 
Learning with Wikipedia.ipynb
Learning with Wikipedia.ipynb
 
 
Logic gate.ipynb
Logic gate.ipynb
 
 
Logistic Regression.ipynb
Logistic Regression.ipynb
 
 
Movie recommendation.ipynb
Movie recommendation.ipynb
 
 
Multi Linear&Logistic Regression.ipynb
Multi Linear&Logistic Regression.ipynb
 
 
MultiLayer Perceptron.ipynb
MultiLayer Perceptron.ipynb
 
 
NER using BiLSTM-CRF.ipynb
NER using BiLSTM-CRF.ipynb
 
 
Part-of-speech Tagging using Bi-LSTM.ipynb
Part-of-speech Tagging using Bi-LSTM.ipynb
 
 
Practice GloVe.ipynb
Practice GloVe.ipynb
 
 
Pre-trained GloVe.ipynb
Pre-trained GloVe.ipynb
 
 
README.md
README.md
 
 
Recurrent Neural Network.ipynb
Recurrent Neural Network.ipynb
 
 
Regular Expression.ipynb
Regular Expression.ipynb
 
 
SentencePiece.ipynb
SentencePiece.ipynb
 
 
Sigmoid Function.ipynb
Sigmoid Function.ipynb
 
 
Softmax Regression practicing_the classification of irises.ipynb
Softmax Regression practicing_the classification of irises.ipynb
 
 
Splitting Data.ipynb
Splitting Data.ipynb
 
 
TF-IDF.ipynb
TF-IDF.ipynb
 
 
Text Generation using GRU.ipynb
Text Generation using GRU.ipynb
 
 
Text Generation using RNN.ipynb
Text Generation using RNN.ipynb
 
 
Transformer.ipynb
Transformer.ipynb
 
 
Word-Level Seq2Seq Translator.ipynb
Word-Level Seq2Seq Translator.ipynb
 
 
Word2Vec with gensim.ipynb
Word2Vec with gensim.ipynb
 
 
preprocessing.ipynb
preprocessing.ipynb
 
 

Repository files navigation

Study-Deeplearning-NLP

Study Deep learning & Machine learning for NLP

This repository is for record my studies and make enable study anywhere for myself.

These studies are based on [유원준/안상준, "딥 러닝을 이용한 자연어 처리 입문", 2022] plus [Saito Goki, "Deep Learning from Scratch"]. but is not same in all part. And started from 22.09.27 .

Siwon Seo, Tokai-university

Contents

Text preprocessing

  • Tokenization
  • Cleaning & Normalization
  • Stemming & Lemmatization
  • Stopword
  • Integere Encoding
  • Padding
  • One-Hot Encoding
  • Splitting Data

Language Model

  • Statistical Language Model
  • N-gram Language Model
  • Perplexity (PPL)

Count based word Representation

  • Bag of Words (BoW)
  • Document-Term Matrix (DTM)
  • TF-IDF (Term Frequency-Inverse Document Frequency)

Vector Similarity

  • Cosine Similarity
  • Euclidean distance, Jaccard similarity

Machine Learning (Deep Learning basic)

  • Linear Regression
  • Automatic differentiation
  • Logistic Regression
  • Softmax Regression
  • etc..

Deep Learning

  • Perceptron
  • Feed-Forward Neural Network (FFNN)
  • Fully-connected layer (FC)
  • Activation Function
  • Loss function
  • Batch Size
  • Optimizer
  • Back-Propagation
  • Overfitting
  • Gradient Vanishing & Exploding

Recurrent Neural Network (RNN)

  • RNN
  • LSTM
  • GRU
  • BiLSTM (BiGRU)

Word Embedding

  • Sparse Representation
  • Dense Representation & Word Embedding
  • Word2Vec
  • GloVe
  • FastText
  • ELMo

CNN

  • CNN
  • 1D CNN
  • Character Embedding

Tagging Task

  • Part of speech Tagging
  • Named Entity Recognition (NER)
  • BIO

====================================

Subword Tokenizer

  • Byte Pair Encoding (BPE)
  • Sentence Piece
  • Subword Text Encoder

Sequence to Sequence

Attention Mechanism

Transformer

About

Study Deep learning & Machine learning for NLP

Resources

Readme
Activity

Stars

0 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages