Development

ArcGIS_Snowflake/environment.yml

@@ -0,0 +1,5 @@
+name: app_environment
+channels:
+  - snowflake
+dependencies:
+  - pydeck=*

Avalanche-Customer-Review-Analytics/Avalanche-Customer-Review-Analytics.ipynb

@@ -0,0 +1,192 @@
+{
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Streamlit Notebook",
+   "name": "streamlit"
+  },
+  "lastEditStatus": {
+   "notebookId": "2gfpag77rjklnaepw2qp",
+   "authorId": "6841714608330",
+   "authorName": "CHANINN",
+   "authorEmail": "[email protected]",
+   "sessionId": "fd937486-2fde-4160-99dc-ddfca8af4103",
+   "lastEditTime": 1743707076161
+  }
+ },
+ "nbformat_minor": 5,
+ "nbformat": 4,
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "3e3bdd35-2104-4280-a28f-e02cac177a85",
+   "metadata": {
+    "name": "md_title",
+    "collapsed": false
+   },
+   "source": "# Build a Customer Review Analytics Dashboard with Streamlit on Snowflake\n\nIn this notebook, we're performing data processing of the Avalanche customer review data. By the end of the tutorial, we'll have created a few data visualization to gain insights into the general sentiment of the products."
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3fc8fa46-8a26-43e3-a2a9-381c89eae2a7",
+   "metadata": {
+    "name": "md_about",
+    "collapsed": false
+   },
+   "source": "## Avalanche data\n\nThe Avalanche data set is based on a hypothetical company that sells winter sports gear. Holistically, this data set is comprised of the product catalog, customer review, shipping logistics and order history.\n\nIn this particular notebook, we'll use only the customer review data. We'll start by uploading customer review data in DOCX format. Next, we'll parse and reshape the data into a semi-structured form. Particularly, we'll apply LLMs for language translation and text summarization along with sentiment analysis."
+  },
+  {
+   "cell_type": "markdown",
+   "id": "03e5be91-6497-450d-97c0-ca70199b8eef",
+   "metadata": {
+    "name": "md_data",
+    "collapsed": false
+   },
+   "source": "## Retrieve customer review data\n\nFirst, we're starting by querying and parsing the content from DOCX files that are stored on the `@avalanche_db.avalanche_schema.customer-reviews` stage."
+  },
+  {
+   "cell_type": "code",
+   "id": "b45557a0-01b9-4775-9b97-28da754ec326",
+   "metadata": {
+    "language": "sql",
+    "name": "sql1",
+    "collapsed": false,
+    "codeCollapsed": false
+   },
+   "outputs": [],
+   "source": "-- Parse content from DOCX files\nWITH files AS (\n  SELECT \n    REPLACE(REGEXP_SUBSTR(file_url, '[^/]+$'), '%2e', '.') as filename\n  FROM DIRECTORY('@avalanche_db.avalanche_schema.customer_reviews')\n  WHERE filename LIKE '%.docx'\n)\nSELECT \n  filename,\n  SNOWFLAKE.CORTEX.PARSE_DOCUMENT(\n    @avalanche_db.avalanche_schema.customer_reviews,\n    filename,\n    {'mode': 'layout'}\n  ):content AS layout\nFROM files;",
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "id": "796ba2b7-2d50-4d22-911d-db20912257f5",
+   "metadata": {
+    "name": "md_sql2",
+    "collapsed": false
+   },
+   "source": "## Data reshaping\n\nWe're reshaping the data to a more structured form by using regular expression to create additional columns from the customer review `LAYOUT` column."
+  },
+  {
+   "cell_type": "code",
+   "id": "c6f47ba7-4c5a-46f1-a2eb-3533f4dcda05",
+   "metadata": {
+    "language": "sql",
+    "name": "sql2",
+    "codeCollapsed": false,
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": "-- Extract PRODUCT name, DATE, and CUSTOMER_REVIEW from the LAYOUT column\nSELECT \n  filename,\n  REGEXP_SUBSTR(layout, 'Product: (.*?) Date:', 1, 1, 'e') as product,\n  REGEXP_SUBSTR(layout, 'Date: (202[0-9]-[0-9]{2}-[0-9]{2})', 1, 1, 'e') as date,\n  REGEXP_SUBSTR(layout, '## Customer Review\\n([\\\\s\\\\S]*?)$', 1, 1, 'es') as customer_review\nFROM {{sql1}};",
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "id": "99f6b075-3d7c-4615-8414-86568a80ee20",
+   "metadata": {
+    "name": "md_sql3",
+    "collapsed": false
+   },
+   "source": "## Apply Cortex LLM on customer review data\n\nHere, we'll apply the Cortex LLM to perform the following 3 tasks:\n- Text translation is performed on foreign language text where they are translated to English.\n- Text summarization is performed on the translated text to obtain a more concise summary.\n- Sentiment score is calculated to give insights on whether the sentiment was positive or negative."
+  },
+  {
+   "cell_type": "code",
+   "id": "74be7b08-6122-4a98-b113-99ff874375e3",
+   "metadata": {
+    "language": "sql",
+    "name": "sql3",
+    "collapsed": false,
+    "codeCollapsed": false
+   },
+   "outputs": [],
+   "source": "-- Perform translation, summarization and sentiment analysis on customer review\nSELECT \n    product,\n    date,\n    SNOWFLAKE.CORTEX.TRANSLATE(customer_review, '', 'en') as translated_review,\n    SNOWFLAKE.CORTEX.SUMMARIZE(translated_review) as summary,\n    SNOWFLAKE.CORTEX.SENTIMENT(translated_review) as sentiment_score\nFROM {{sql2}}\nORDER BY date;",
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "id": "adaa0f32-5263-41ac-aa30-88cc75303d42",
+   "metadata": {
+    "name": "md_df",
+    "collapsed": false
+   },
+   "source": "## Convert SQL output to Pandas DataFrame\n\nHere, we'll convert the SQL output to a Pandas DataFrame by applying the `to_pandas()` method."
+  },
+  {
+   "cell_type": "code",
+   "id": "b88d6ae3-0de9-42c1-b48a-f2ebc4d34255",
+   "metadata": {
+    "language": "python",
+    "name": "df",
+    "codeCollapsed": false,
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": "sql3.to_pandas()",
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a3a0334d-29df-494f-982f-3e1fcd916066",
+   "metadata": {
+    "name": "md_bar",
+    "collapsed": false
+   },
+   "source": "## Bar charts\n\nHere, we're creating some bar charts for the sentiment scores.\n\n### Daily sentiment scores\n\nNote: Positive values are shown in green while negative values in red."
+  },
+  {
+   "cell_type": "code",
+   "id": "4cd85ca2-f005-4285-a633-744b12de2109",
+   "metadata": {
+    "language": "python",
+    "name": "py_bar",
+    "codeCollapsed": false,
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": "import streamlit as st\nimport altair as alt\nimport pandas as pd\n\n# Ensure SENTIMENT_SCORE is numeric\ndf['SENTIMENT_SCORE'] = pd.to_numeric(df['SENTIMENT_SCORE'])\n\n# Create the base chart with bars\nchart = alt.Chart(df).mark_bar(size=15).encode(\n    x=alt.X('DATE:T',\n            axis=alt.Axis(\n                format='%Y-%m-%d',  # YYYY-MM-DD format\n                labelAngle=90)  # Rotate labels 90 degrees\n            ),\n    y=alt.Y('SENTIMENT_SCORE:Q'),\n    color=alt.condition(\n        alt.datum.SENTIMENT_SCORE >= 0,\n        alt.value('#2ecc71'),  # green for positive\n        alt.value('#e74c3c')   # red for negative\n    ),\n    tooltip=['PRODUCT:N', 'DATE:T'] # Add tooltip\n).properties(\n    height=500\n)\n\n# Display the chart\nst.altair_chart(chart, use_container_width=True)",
+   "execution_count": null
+  },
+  {
+   "cell_type": "markdown",
+   "id": "32bcfa7b-c940-4615-94a2-373c199ede4f",
+   "metadata": {
+    "name": "md_bar_2",
+    "collapsed": false
+   },
+   "source": "### Product sentiment scores"
+  },
+  {
+   "cell_type": "code",
+   "id": "74951343-25ef-41c7-825e-4d487dc676eb",
+   "metadata": {
+    "language": "python",
+    "name": "py_product_sentiment",
+    "codeCollapsed": false
+   },
+   "outputs": [],
+   "source": "import streamlit as st\nimport altair as alt\nimport pandas as pd\n\n# Create the base chart with aggregation by PRODUCT\nbars = alt.Chart(df).mark_bar(size=15).encode(\n    y=alt.Y('PRODUCT:N', \n            axis=alt.Axis(\n                labelAngle=0,  # Horizontal labels\n                labelOverlap=False,  # Prevent label overlap\n                labelPadding=10  # Add some padding\n            )\n    ),\n    x=alt.X('mean(SENTIMENT_SCORE):Q',  # Aggregate mean sentiment score\n            title='MEAN SENTIMENT_SCORE'),\n    color=alt.condition(\n        alt.datum.mean_SENTIMENT_SCORE >= 0,\n        alt.value('#2ecc71'),  # green for positive\n        alt.value('#e74c3c')   # red for negative\n    ),\n    tooltip=['PRODUCT:N', 'mean(SENTIMENT_SCORE):Q']\n).properties(\n    height=400\n)\n\n# Display the chart\nst.altair_chart(bars, use_container_width=True)",
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "id": "d430287f-867c-484a-8e09-d9d29ca9ef3f",
+   "metadata": {
+    "language": "python",
+    "name": "py_download",
+    "codeCollapsed": false
+   },
+   "outputs": [],
+   "source": "# Download button for the CSV file\nst.subheader('Processed Customer Reviews Data')\nst.download_button(\n    label=\"Download CSV\",\n    data=df[['PRODUCT', 'DATE', 'SUMMARY', 'SENTIMENT_SCORE']].to_csv(index=False).encode('utf-8'),\n    mime=\"text/csv\"\n)",
+   "execution_count": null
+  },
+  {
+   "cell_type": "code",
+   "id": "597a05b3-0ead-4fb0-a821-d02ce6802b47",
+   "metadata": {
+    "language": "sql",
+    "name": "cell1"
+   },
+   "outputs": [],
+   "source": "",
+   "execution_count": null
+  }
+ ]
+}