11 - Project - Natural Language Processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bbac054e",
   "metadata": {},
   "source": [
    "# Project - Natural Language Processing\n",
    "- Can you determine who tweeted this?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1c81218",
   "metadata": {},
   "source": [
    "### Description\n",
    "- We will analyze a collection of tweets from one tweet account\n",
    "- Can we figure out the person behind the account?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74bc044f",
   "metadata": {},
   "source": [
    "### Step 1: Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96cab55a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from nltk import word_tokenize, ngrams\n",
    "from collections import Counter\n",
    "import nltk\n",
    "nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32280f22",
   "metadata": {},
   "source": [
    "### Step 2: Import data\n",
    "- Use Pandas [read_csv](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) method to read files/tweets.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2708ecdc",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85cfe9d1",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "37099a8c",
   "metadata": {},
   "source": [
    "### Step 3: Convert content to a list of content\n",
    "- Use list on the column **content**\n",
    "    - You can also apply [to_list()](https://pandas.pydata.org/docs/reference/api/pandas.Series.to_list.html) on the column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26de946c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1d7e107",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "24c9b883",
   "metadata": {},
   "source": [
    "### Step 4: Create a corpus\n",
    "- Create an empty list called **corpus**\n",
    "- Iterate over **content**\n",
    "    - Extend **corpus** with all words in lowercase if any character is alpha in the word.\n",
    "        - HINT: To lowercase, call **lower()** on the word.\n",
    "        - HINT: To check if any character is alhpa, use **any(c.isalpha() for c in word)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a62294a5",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc328dcc",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "9e69e419",
   "metadata": {},
   "source": [
    "### Step 5: Check corpus\n",
    "- Find the length of the corpus\n",
    "- Look at the first 10 words in the corpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fa6fcbd",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f4762ac",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "6ed7bb50",
   "metadata": {},
   "source": [
    "### Step 6: Display all 3-grams\n",
    "- Use **Counter(ngrams(corpus, 3))** and assign it to a variable\n",
    "- List the 10 most common 3-grams\n",
    "    - HINT: call **most_common(10)** on the result from **Counter(...)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72a7be3e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "665a8df2",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "95494b28",
   "metadata": {},
   "source": [
    "### Step 7 (Optional): Pretty print\n",
    "- Iterate over the result with a for-loop\n",
    "    - HINT: Each loop gives a **ngram** and **frequency**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "468ffcb8",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "615dfde1",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b48d23ca",
   "metadata": {},
   "source": [
    "### Step 8 (Optional): Try it with 4-grams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d498871",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c18a607e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4d7869f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}