249763NLEQuestion2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/YashMagane/Natural-Language-Engineering/blob/main/249763NLEQuestion2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Natural Language Engineering G5119\n",
        "## Computer Based Examination, 2023\n",
        "\n",
        "Remember, you can add cells and change their type (between code and text/markdown) as required to answer the questions."
      ],
      "metadata": {
        "id": "0sTS7cMjiUHb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# update your candidate number here\n",
        "candidate_number = 249763"
      ],
      "metadata": {
        "id": "AXFCZsF0iUkz"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Question 2 (50 marks)\n",
        "\n",
        "This Question is about POS Tagging.\n",
        "\n"
      ],
      "metadata": {
        "id": "grPE5uwpVI65"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7KspiAfrVF_n"
      },
      "outputs": [],
      "source": [
        "### do not change the code in this cell\n",
        "# make sure you run this cell\n",
        "tagged_sentences=[\"john_N loves_V chocolate_N\",\n",
        "                  \"bob_N hates_V meetings_N\",\n",
        "                  \"alice_N likes_V fred_N\"]"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "a)\n",
        "\n",
        "i) Use the `split` method to break each sentence in `tagged_sentences` into `word_tag` tokens. The result should be a list of lists and be named `tagged_words` For example, `[\"dogs_N bark_V\"]` should become `[[\"dogs_N\",\"bark_V\"]]`. (4 marks)\n",
        "\n"
      ],
      "metadata": {
        "id": "LgT47zd8WJNm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tagged_words = []\n",
        "for sentence in tagged_sentences:\n",
        "    words = sentence.split()\n",
        "    tagged_words.append(words)\n",
        "print(tagged_words)"
      ],
      "metadata": {
        "id": "CkoECVuwdxrJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b3b69d6c-f658-49a2-f3f3-81d06fa4ee57"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[['john_N', 'loves_V', 'chocolate_N'], ['bob_N', 'hates_V', 'meetings_N'], ['alice_N', 'likes_V', 'fred_N']]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "ii) Use the `split` method again to break each token in `tagged_words` into `(word,tag)` tuples. The result should be a list of lists of tuples and be named `words_and_tags` For example, `[[\"dogs_N\", \"bark_V\"]]` should become `[[(\"dogs\",\"N\"), (\"bark\",\"V\")]]`. (6 marks)\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "_0cSNormdyz-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "words_and_tags = []\n",
        "for sentence in tagged_words:\n",
        "    tuples = []\n",
        "    for token in sentence:\n",
        "        word, tag = token.split(\"_\")\n",
        "        tuples.append((word, tag))\n",
        "    words_and_tags.append(tuples)\n",
        "print(words_and_tags)"
      ],
      "metadata": {
        "id": "36U7HvzfeyB5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fb55a086-47cc-4746-fa87-6510c0929320"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[('john', 'N'), ('loves', 'V'), ('chocolate', 'N')], [('bob', 'N'), ('hates', 'V'), ('meetings', 'N')], [('alice', 'N'), ('likes', 'V'), ('fred', 'N')]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "iii) Calculate the counts for all tags in `words_and_tags`. The result should be a dictionary called `tag_counts`. For example, the tag \"V\" occurs 3 times, so `tag_counts[\"V\"]` will be `3`. (4 marks)"
      ],
      "metadata": {
        "id": "RIuwCLsPeyvv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tag_counts = {}\n",
        "for sentence in words_and_tags:\n",
        "    for word, tag in sentence:\n",
        "        if tag not in tag_counts:\n",
        "            tag_counts[tag] = 1\n",
        "        else:\n",
        "            tag_counts[tag] += 1\n",
        "print(tag_counts)"
      ],
      "metadata": {
        "id": "f62Qza4XfbGj",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "321d4a7e-c103-4aa8-9147-01280cfed995"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'N': 6, 'V': 3}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "iv) Calculate the tag counts for all words in `words_and_tags` and call the result `word_tag_counts`. This should be a dictionary of dictionaries with the outer dictionary having words as keys and the values being themselves dictionaries that contain the tag counts for that word. For example, `word_tag_counts[\"john\"][\"N\"]=1`. (6 marks)"
      ],
      "metadata": {
        "id": "RdPWLXiNfbjA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "word_tag_counts = {}\n",
        "for sentence in words_and_tags:\n",
        "    for word, tag in sentence:\n",
        "        if word not in word_tag_counts:\n",
        "            word_tag_counts[word] = {}\n",
        "        if tag not in word_tag_counts[word]:\n",
        "            word_tag_counts[word][tag] = 1\n",
        "        else:\n",
        "            word_tag_counts[word][tag] += 1\n",
        "print(word_tag_counts)"
      ],
      "metadata": {
        "id": "xQU64ejVg5_0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6b02f7d7-d0ba-4e94-b43f-7e4a0efb2cfc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'john': {'N': 1}, 'loves': {'V': 1}, 'chocolate': {'N': 1}, 'bob': {'N': 1}, 'hates': {'V': 1}, 'meetings': {'N': 1}, 'alice': {'N': 1}, 'likes': {'V': 1}, 'fred': {'N': 1}}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "b) Name three other common part-of-speech classes, and an example of a word that is found in each class. (6 marks)"
      ],
      "metadata": {
        "id": "ILO3VZ6RhHZY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Adjectives: they are used to describe characterisitics of nouns or pronouns, an example of an adjective would be smart.\n",
        "\n",
        "Adverbs: they are words in which describe the adjectives or verbs, in some cases they can describe themselves too, most adverbs end in the sufix -ly, and example of a adverb is preferably.\n",
        "\n",
        "Conjuction: these words are used to join two or more words together, they can be used to join phrases too, the most common example being the word and."
      ],
      "metadata": {
        "id": "9m5dAMi1EMZl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "c) Construct a unigram tagger by following the steps below."
      ],
      "metadata": {
        "id": "Rrr2XXy5EM-i"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "i) Calculate the tag probabilities p(t) from the `tag_counts` dictionary by dividing each tag count by the total count and put the result in a dictionary called `tag_probs`. (3 marks)"
      ],
      "metadata": {
        "id": "bIJklFHwEjNA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tag_probs = {}\n",
        "total_count = sum(tag_counts.values())\n",
        "for tag, count in tag_counts.items():\n",
        "    tag_probs[tag] = count/total_count\n",
        "print(tag_probs)"
      ],
      "metadata": {
        "id": "4GfOLhidFC41",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3a3fec44-cf1b-4717-a3f3-ec0fed7be95b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'N': 0.6666666666666666, 'V': 0.3333333333333333}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "ii) Calculate the emission probabilities p(w|t) from the `word_tag_counts` dictionary by dividing each word-tag count by the total count for each tag, and put the result in a dictionary called `word_tag_probs`. (5 marks)"
      ],
      "metadata": {
        "id": "UoB_NrryFDcA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "word_tag_probs = {}\n",
        "for word, tag_counts in word_tag_counts.items():\n",
        "    if word not in word_tag_probs:\n",
        "        word_tag_probs[word] = {}\n",
        "    for tag, count in tag_counts.items():\n",
        "        total_count = tag_counts[tag]\n",
        "        word_tag_probs[word][tag] = count / total_count\n",
        "print(word_tag_probs)"
      ],
      "metadata": {
        "id": "PZM4k95cFhgJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3df91d8c-c162-43e7-a910-361c42f01505"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'john': {'N': 1.0}, 'loves': {'V': 1.0}, 'chocolate': {'N': 1.0}, 'bob': {'N': 1.0}, 'hates': {'V': 1.0}, 'meetings': {'N': 1.0}, 'alice': {'N': 1.0}, 'likes': {'V': 1.0}, 'fred': {'N': 1.0}}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "iii) Write a function that takes a list of words and outputs the most probable tag for each word. Apply your function to the list `[\"fred\",\"likes\",\"meetings\"]`. (10 marks)"
      ],
      "metadata": {
        "id": "J9h_nl-pIqeV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def get_most_probable_tags(words, word_tag_probs, tag_probs):\n",
        "    most_probable_tags = []\n",
        "    for word in words:\n",
        "        if word in word_tag_probs:\n",
        "            tag_probs = word_tag_probs[word]\n",
        "            most_probable_tag = max(tag_probs, key=tag_probs.get)\n",
        "    return most_probable_tags"
      ],
      "metadata": {
        "id": "OQLnGU-ON5iP"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "d)\n",
        "\n",
        "i) The unigram tagger defined above ignores the order of tags and words. A better approach is to assign probabilities to sequences of tags, rather than to each tag individually. Describe the Markov assumption as it applies in this case. (2 marks)"
      ],
      "metadata": {
        "id": "DY7JDPE-N-at"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The Markov assumption is that the probaiblity of a sequence of tags relies only on the previous tag in the sequence. for example, a sequence of tags like A V N, the probability of getting N relies on the tag V, but not A. In this case, if Markov's approach is used, the model can take the order of the tags and put higher probabilities to the tags."
      ],
      "metadata": {
        "id": "yBLgfoH7O3AH"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "ii) In the case where we are trying to find the most probable sequence of tags, the simplest approach would be to calculate a probability for every possible assignment of tags to words. Why might we want to avoid doing this, and what algorithm can we use instead? (4 marks)"
      ],
      "metadata": {
        "id": "rfc8ne91O39T"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Making calculations of a probability for every possible assigmment of tags to words would be very time consuming and just not feasible if we have a large quantity of data. Viterbi's algorithm can be used to find the best tag sequence without enumerating all possibilities as it uses Markov's assumption to calculate the probability of each possible tag, by first recording the highest probability."
      ],
      "metadata": {
        "id": "K6VFkX4hS9ET"
      }
    }
  ]
}