Note to Self

[codecakes]

FE/BE API integration work:

• Convert business cases of sales erp to OpenAPI using Swagger API and CodeGen.
• Design Data Modeling for NoSQL, SQL and TSDB requirements.
• Use FastAPI instead of Flask to generate openAPI spec Restful app.
• Write core components. TDD!
  • Use Auth0 for RBAC and user session management.
  • Do Swagger API Automation Testing.
  • Write Mock Client/Server Test coverage.
• Use Bazel if possible. And deploy to Cloud Endpoint. Test.
  • Authenticate using dummy Id using Auth0 session.
  • Make API calls.

---

BE/DB

Archive Time Series As Flat file data Older than X momnths from Firestore to Google Storage in Parquet format using PyStore.

Archived Storage: Write to Flat File.

parallelize(
for each firestore sales collections (Like <sales_collection_Appl_Goog>)
	- Connect to pystore collection using GCFS via FUSE mounted dir and fetch <store_name> 
	- fetch tail of each firestore sales collection from <store_name>
	- dump all docs from json to df after tail and older than <X> days to <store_name>
	- dump df <store_name> to pystore <store_name> collection
)

Warehouse Analytics: Read from Flat file.

parallelize(
for each firestore sales collection (Like <sales_collection_Appl_Goog>)
	- Connect to pystore collection using GCFS via FUSE mounted dir and fetch <store_name> 
	- fetch each firestore sales collection name from <store_name>
	- Load pystore collection from <store_name>
	- Load df from pystore collection
	- Dump df to BigQuery using `pandas-gbq`
)