v0.2

v0.2

Added features

• Database integration via Postgresql on Supabase
• Utilising private API service for whatsapp message sending
• Created two cron workflows on master branch:
  • Master branch main cron job: Executes automation every 1st day of the month. Includes database persistence to prod and sending WhatsApp messages.
  • Master branch cron job tests: Executes automation every 5 days for test purposes. Saves data to SQA schema and skips sending WhatsApp messages.

New Feature Update Limitations

• As the WhatsApp messaging API is private, other users are not able to use it, hence unable to fully use the automation, unless they edit it.
• If the API breaks, the entire messaging module of the automation is unusable.
• The cron jobs may not run immediately on schedule, as it is dependent on whether Github Actions can support the load to run or not. Reference

Cumulative Project Limitations

• Automation scrips are dependent on website HTML structure.
• Automation scripts are also dependent on code dependencies/modules.
• Whatsapp messaging via pywhatkit will actually fail in github actions, but no error logs are produced. However, this does not break the pipeline.
• As the WhatsApp messaging API is private, other users are not able to use it, hence unable to fully use the automation, unless they edit it.
• If the API breaks, the entire messaging module of the automation is unusable.
• The cron jobs may not run immediately on schedule, as it is dependent on whether Github Actions can support the load to run or not. Reference

Future works

• Add unit tests to modules.
• Organize code into relevant packages or modules wherever necessary.
• Save the screenshots to a remote bucket or storage service.

v0.1

v0.1

Features

• Basic working automated web scraping on tnb and air selangor portal
• Basic working whatsapp messaging utility
• Workflow setup for dev branch push

Limitations

• Automation scrips are dependent on website HTML structure which I am not in control of.
• Automation scripts are also dependent on code dependencies/modules which I am also not in control of.
• Whatsapp messaging actually fails in github actions, but no error logs produced because it is not yet built. However, this does not break the pipeline.

Future works

• Database integration via Postgresql on Supabase (it's free)
• Test Twilio for WhatsApp API (Free version)
• Add a pipeline for pull requests on the main branch