Individual Big Data Programming Project -- University Junior Year
Before beginning my main data science project, I originally planned to analyse lego data and determine the relationship between the feature of Lego sets such as the price, number of pieces and the theme and how they correlate with each other. The analysis should be able to answer the following questions:
- How does the theme of a Lego set affect its price?
- Which Lego sets have the most pieces? Is there a common theme between the top Lego sets?
- Which are the most common themes for Lego sets?
- What are the number of Lego sets made for each theme?
- What are the top age groups for Lego sets? Is there a common theme between the top ages?
- Which Lego sets have the most minifigs? Is there a common theme between the top Lego sets?
I was able to create a scraping tool using R that scraped a total of 19,239 unique records with 16 unique attributes. Due to some changes in my aims and scope and wanting to include textual data, I decided to change the scope for my individual project and analyse lyrics instead. This scrapped data will be used later for a personal project to gain more experience in big data programming.
- tidyverse
- dplyr
- rvest