Shopify Extractor

Shopify extractor for Keboola Connection. Download all objects under Orders, Products, inventory items, levels, locationsEvent and Customer hierarchies.

Data is always loaded incrementally.

Credits: Client part of this application is partially inspired by Singer.IO shopify TAP project

Table of contents:

[TOC]

Shopify setup - Prerequisites

To enable this application you need to create a custom application :

• In the left panel in Shopify select "Apps"

• Click "Develop apps" on the top of the webpage (it is next to the green "Customize your store" button)

• Click "Create an App"

• Name the app, ex. "Keboola Extractor app"

• Click "Configure Admin API Scopes"

• Enable Read access for following objects:

  • Orders : "read_orders"
  • Products : "read_products"
  • Inventory : "read_inventory"
  • Customers : "read_customers"
  • Events : "read_marketing_events"
  • Locations : "read_locations"
  • Payments : "read_shopify_payments_payouts"

• Save the scopes

• Click "Install app"

• Copy the Admin API access token, store it somewhere safe

• use this Admin API access token to authorize the app in the "Admin API access token" field

Configuration

Admin password

Admin password of your private app.

Shop name

Your shop id found in url, e.g. [shop_id].myshopify.com

Loading Options

Fetch parameter

Define which date field (updated_at or created_at) will be used for filtering while fetching data from Shopify. Any data that was updated or created in the range will be fetched.

Period from and to dates

Will fetch data filtering on the defined fetch parameter. Accepts date in YYYY-MM-DD format or dateparser string i.e. 5 days ago, 1 month ago, yesterday, etc.

Load type

The result tables will be updated based on the primary key if set to Incremental update. Full load overwrites the destination table each time.

Endpoints

Following endpoints are supported

Products

Inventory

This allows to retrieve inventory items, its' levels and locations based on related products.

NOTE this endpoint is available only if Products endpoint is checked.

To link product variant with inventory_item and inventory_level follow the diagram below, the datasets can be joined through their primary foreign/primary keys:

Orders

Download orders, order is a customer's request to purchase one or more products from a shop

Orders - transactions

Download transactions related to Orders.

Customers

The Customer endpoint download information about a shop's customer

Payments Transactions

Balance Transactions make up the canonical "statement of account", or ledger for a Shopify Payments account

Events

Downloads events related to selected Resources. These need to be selected in the resources fields

Event Types

You may download specific event types to limit the result size. Specify the event names separated with comma e.g. confirmed, create, destroy. If omitted all possible types are downloaded

Note that different resources generate different types of event. See the docs for a list of possible verbs.

Development

If required, change local data folder (the CUSTOM_FOLDER placeholder) path to your custom path in the docker-compose file:

    volumes:
      - ./:/code
      - ./CUSTOM_FOLDER:/data

Clone this repository, init the workspace and run the component with following command:

git clone [email protected]:kds_consulting_team/kds-team.ex-shopify.git
cd kds-team.ex-shopify
docker-compose build
docker-compose run --rm dev

Run the test suite and lint check using this command:

docker-compose run --rm test

Integration

SSL verifying turnoff for development

in shopify_cli.py add following lines to the top of the script

import ssl
# Add this at the top of your script
ssl._create_default_https_context = ssl._create_unverified_context

For information about deployment and integration with KBC, please refer to the deployment section of developers documentation