HTML Web scraping on Mars data to create a Flask web application using Python, Beautiful Soup, Splinter and MongoDB

Mission-to-Mars

Overview

To develop an app to webscrape the following information about the planet Mars from NASA Science Mars Exploration websites:

• Latest News
• Featured Image
• Facts about the planet
• Images of the hemispheres

Resources

• Data Sources:

  • Mars Planet Science
  • Space Image
  • Astropedia Search Results | GUSS Astrogeology Science Center

• Software/Libraries: Visual Studio Code 1.56.0, jupyter Notebook 6.3.0, Jupyter lab 3.0.14, Flask 1.1.2, Splinter 1.26.4, Web Drive Manager, Beautiful Soup, Pymongo, MongoDB 4.4.6, Mongo DB Compass, htmlslib, lxml.

Results:

**Task 1

• Used Chrome Developer tools to identified HTML components,
• Using Beautiful Soup/Splinter to automate a web browser and perform the scrape:
  • webscrapped and retrieved latest news about Mars, feature image and facts about the planet
  • scrapped full-resolution images of Mars’s hemispheres and the titles of those images.
• MongoDB to store the data,
• Flask to create a web application to display the data.

Initial App Design

Task 2

• Changed Bootstrap 3 components to customize the view of the page:
  • Updated the color and size of the scrape button from btn btn-warning btn-xs to btn btn-default btn-lg
  • Added the hemisphere images as thumbnails in a single row by changing the grid from col-md-6 to col-md-3.

• Used the Bootstrap 3 grid system to update the index.html file so that the website is mobile-responsive.
• Tested the responsiveness of the website using the DevTools.

iPad Pro

Pixel 2XL