Skip to content

Commit

Permalink
docs: cq hub documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
koltyakov committed Nov 19, 2023
1 parent 50bdba8 commit 8a230c8
Show file tree
Hide file tree
Showing 19 changed files with 67 additions and 108 deletions.
2 changes: 0 additions & 2 deletions docs/authentication.md → docs/_conf_auth.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# SharePoint Source Plugin Authentication Options

```yaml
# sharepoint.yml
# ...
Expand Down
2 changes: 0 additions & 2 deletions docs/conf-ct.md → docs/_conf_cts.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# Configuration: Content Types based rollup

Content Types based rollup allows to fetch data from multiple lists or document libraries based on the Content Type configuration.

All items based on the parent content type are fetched from all lists and subwebs below the context site URL.
Expand Down
2 changes: 0 additions & 2 deletions docs/conf-libs.md → docs/_conf_libs.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# Configuration: Document libraries

Document listariries are the same as lists in SharePoint, but with a few differences. And it's common to expand File entity to get file metadata.

Also, a document library URI usually doesn't contain `Lists/` prefix.
Expand Down
2 changes: 0 additions & 2 deletions docs/conf-lists.md → docs/_conf_lists.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# Configuration: Lists

SharePoint lists is the main artifact for customizable data storage in SharePoint.

Lists fetching configuration follows same naming as SharePoint REST API.
Expand Down
2 changes: 0 additions & 2 deletions docs/conf-mmd.md → docs/_conf_mmd.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# Configuration: Managed Metadata

To configure managed metadata fetching, you need to provide a term set ID (GUID) and an optional alias for the table name.

```yaml
Expand Down
4 changes: 1 addition & 3 deletions docs/conf-search.md → docs/_conf_search.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
# Configuration: Search queries

Search drived data source can be user only with user associated authentication strategies. E.g. it won't work with `addin` strategy.
Search-drived datasource can be user only with user associated authentication strategies. E.g. it won't work with `addin` strategy.

```yaml
# sharepoint.yml
Expand Down
2 changes: 0 additions & 2 deletions docs/spctl.md → docs/_conf_spctl.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
### Interactive schema builder

The plugin ships with configuration utility `spctl`.

![](https://github.com/koltyakov/cq-source-sharepoint/blob/main/assets/spctl.gif?raw=true)
Expand Down
2 changes: 0 additions & 2 deletions docs/conf-uil.md → docs/_conf_uil.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# Configuration: User Information List

Quite often you'd need getting User Information List for Author and Editor fields joining. This is a special case, and we have a dedicated configuration for it.

```yaml
Expand Down
2 changes: 0 additions & 2 deletions docs/conf-ups.md → docs/_conf_ups.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# Configuration: User Profiles

User Profiles are fetched via Search API, so the search should be configured in the farm.

Search drived data source can be user only with user associated authentication strategies. E.g. it won't work with `addin` strategy.
Expand Down
15 changes: 0 additions & 15 deletions docs/_meta.json

This file was deleted.

61 changes: 46 additions & 15 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,55 @@
# SharePoint Source Plugin Configuration Reference

A single source `yml` configuration assumes fetching data from a single SharePoint site. If you need to fetch data from multiple sites, you can create multiple source configurations. Alternatevely, search based data fetching can be used for rollup scenarios grabbing data from as many sites as needed.

```yaml
# sharepoint.yml
# ...
spec:
# auth:
# lists:
# content_types:
# mmd:
# search:
# profiles:
auth:
# ...
lists:
# ...
content_types:
# ...
mmd:
# ...
search:
# ...
profiles:
# ...
```

See more details about configuration options in the following sections:
### Config: Authentication

:conf_auth

### Config: Lists

:conf_lists

### Config: Document libraries

:conf_libs

### Config: User Information List

:conf_uil

### Config: Content Types

:conf_cts

### Config: Managed Metadata

:conf_mmd

### Config: Search

:conf_search

### Config: User Profiles

:conf_ups

### Interactive Schema Builder

- [auth](authentication)
- [lists](config-lists)
- [content_types](conf-ct)
- [mmd](conf-mmd)
- [search](conf-search)
- [profiles](conf-ups)
:conf_spctl
20 changes: 11 additions & 9 deletions docs/getting-started.md → docs/getting_started.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
# Getting started guide
---
hub-title: Getting Started
---

## Install CloudQuery
### Install CloudQuery

Follow [quickstart instructions](https://www.cloudquery.io/docs/quickstart/).

## Source sample data
### Source sample data

Provision and seed some sample data. [See more](https://github.com/koltyakov/cq-source-sharepoint/blob/main/cmd/seed/README.md). Which satisfy the schema below.

## Auth configuration
### Setup authentication

```bash
# .env or env vars export
Expand All @@ -18,7 +20,7 @@ SP_SITE_URL=https://contoso.sharepoint.com/sites/site

or use "ondeman" auth.

## Source configuration
### Source configuration

```yaml
# sharepoint.yml
Expand All @@ -27,7 +29,7 @@ spec:
name: sharepoint
registry: cloudquery
path: "koltyakov/sharepoint"
version: "VERSION_SOURCE_SHAREPOINT"
version: "VERSION_SOURCE_KOLTYAKOV_SHAREPOINT"
destinations: ["sqlite"]
tables: ["*"]
spec:
Expand Down Expand Up @@ -81,7 +83,7 @@ spec:
alias: "order"
```
## Destination configuration
### Destination configuration
For the sake of simplicity, we'll use SQLite destination.
Expand All @@ -91,12 +93,12 @@ kind: destination
spec:
name: sqlite
path: cloudquery/sqlite
version: "v2.4.15"
version: "VERSION_DESTINATION_SQLITE"
spec:
connection_string: ./sp.db
```
## Run CloudQuery
### Run CloudQuery
```bash
# With auth environment variables exported
Expand Down
15 changes: 9 additions & 6 deletions docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,14 @@ The SharePoint Source plugin allows you to fetch data from SharePoint and load i
- Lists and Document Libraries data fetching
- Content Types based rollup
- User Information List data fetching
- Search Query data source
- User Profiles data source
- Managed Metadata data source
- SharePoint Online support
- SharePoint On-Premise support
- Search Query datasource
- User Profiles datasource
- Managed Metadata datasource

## Supported platforms

- SharePoint Online
- SharePoint On-Premise

![demo](https://github.com/koltyakov/cq-source-sharepoint/blob/main/assets/demo.gif?raw=true)

Expand All @@ -30,7 +33,7 @@ spec:
name: sharepoint
registry: cloudquery
path: koltyakov/sharepoint
version: "VERSION_SOURCE_SHAREPOINT"
version: "VERSION_SOURCE_KOLTYAKOV_SHAREPOINT"
destinations: ["postgresql"]
tables: ["*"]
spec:
Expand Down
7 changes: 0 additions & 7 deletions docs/tables.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/tables/content_types.md

This file was deleted.

9 changes: 0 additions & 9 deletions docs/tables/lists.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/tables/mmd.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/tables/profiles.md

This file was deleted.

7 changes: 0 additions & 7 deletions docs/tables/search.md

This file was deleted.

0 comments on commit 8a230c8

Please sign in to comment.