Skip to content

Commit

Permalink
Docs: CDP formatting and sidebar fixes (#9437)
Browse files Browse the repository at this point in the history 
* realtime edits

---------

Co-authored-by: Marcus Hof <[email protected]>
  • Loading branch information
@ivanagas @MarconLP
ivanagas and MarconLP authored Sep 27, 2024
1 parent 2c0ca85 commit a975c94
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 60 deletions.
60 changes: 33 additions & 27 deletions contents/docs/cdp/destinations/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,21 @@ title: Realtime destinations
showTitle: true
---

> Our new realtime destinations are in <b>preview</b>. You can opt in to try them out in the [Feature previews panel](https://us.posthog.com/#panel=feature-previews). Destinations require the data pipeline add-on in [your billing settings](https://us.posthog.com/organization/billing).
> Our new realtime destinations are in **preview**. You can opt in to try them out in the [Feature previews panel](https://us.posthog.com/#panel=feature-previews). Destinations require the data pipeline add-on in [your billing settings](https://us.posthog.com/organization/billing).
## Template library

### Template library
To get started, you can create a new destination from the [Data pipelines](https://us.posthog.com/pipeline/destinations) page and choose from a range of available **templates**. Depending on the destination, there are a range of configuration options, such as credentials needed or where to find certain properties from the event being processed.

To get started, you can create a new Destination from the [Pipeline destinations](https://us.posthog.com/pipeline/destinations) page and choose from a range of available <b>Templates</b>. Depending on the destination there will be a range of configuration options, such as credentials needed or where to find certain properties from the event being processed.
## Input formatting

### Input formatting
Any configuration input for a destination can have its values formatted using our [Hog](/docs/hog) language to include data from the `event`, `person` and more.

Any configuration input for a destination can have its values formatted using our Hog language to include data from the `event`, `person` and more.

You can template any input using curly brackets `{}`. Many inputs will default to a formatted value indicating a recommended configuration that you can modify if needed.
You can template any input using curly brackets `{}`. Many inputs default to a formatted value indicating a recommended configuration that you can modify if needed.

For example given the incoming event:

```typescript
```ts
{
event: {
name: "$pageview",
Expand All @@ -37,20 +36,19 @@ For example given the incoming event:
}
```

you can create a template such as this:
You can create a template such as this:

```
{event.name} was triggered by {person.properties.first_name} {person.properties.last_name}
// Outputs: $pageview was triggered by Max the Hedgehog
```


#### Global object
### Global object

Below is a the structure of the global variables available whenever templating a destination.

```typescript
```ts
{
event: {
uuid: string // The unique ID of the event
Expand Down Expand Up @@ -87,16 +85,17 @@ Below is a the structure of the global variables available whenever templating a
}
```

### Testing it out
## Testing it out

Once you have saved your destination, you can test it by invoking it with an example payload. You can choose whether to mock out the `async` calls (e.g. HTTP API calls) or not.

> WARNING: If you do not mock out the API call then it will call your destination! Make sure this is definitely okay and check your example data so that you don't mess up your production data.
> **⚠️ Warning:** If you do not mock out the API call, it will call your destination. Make sure this is definitely okay and check your example data so that you don't mess up your production data.
## Advanced - custom code

### Advanced - custom code
For most use cases, we recommend using one of the templates. These take care of a lot of the logic for you and expose simple inputs for you to configure.

Generally speaking we would recommend using one of the many templates that we are building, that take care of a lot of the logic for you, exposing simple inputs for you to configure. You can however, modify any destination by clicking the `show source code` option. From here you can modify the inputs (for example marking an input as secret so that it will be encrypted) or the code of the function itself.
You can, however, modify any destination by clicking the `show source code` option. From here you can modify the inputs (for example marking an input as secret so that it is encrypted) or the code of the function itself.

The code is written in our Hog language with further documentation of available features [here](/docs/hog). The majority of destinations are simply wrappers around `fetch` - a provided function for safely doing async, retriable HTTP calls.

Expand All @@ -112,19 +111,26 @@ if (res.status >= 400) {
}
```
#### Guidelines
- <b>Start from an existing template - </b> Check out existing templates that are close to your kind of use case and work from there
- <b>Rely on filters - </b> always offload as much as possible to the built in `filters` and `inputs`. This will make modifying your destination later much simpler (as well as being more performant)
- <b>Use `inputs` wherever possible - </b> these are great for secrets such as API credentials. You can mark them as secret and they will not be returned to the UI in the future and will be encrypted.
- <b>Keep it short -</b> We have tight controls on execution time, memory usage etc.
- <b>Do not do more than 2 `fetch` calls</b> The function will error if you do. If you have a need for more than 2 calls, please contact support.
### Guidelines
- **Start from an existing template:** Check out existing templates that are close to your kind of use case and work from there.
- **Rely on filters:** Offload as much as possible to the built in `filters` and `inputs`. This will make modifying your destination later much simpler (as well as being more performant)
- **Use `inputs` wherever possible:** These are great for secrets such as API credentials. You can mark them as secret and they will not be returned to the UI in the future and will be encrypted.
- **Keep it short:** We have tight controls on execution time, memory usage etc.
- **Do not do more than 2 `fetch` calls:** The function will error if you do. If you have a need for more than 2 calls, please [contact support](https://us.posthog.com/#panel=support%3Asupport%3Aapps%3A%3Atrue).
## FAQ
### FAQ
### How up-to-date is the person information?
#### How up-to-date is the Person information?
The `person` object contains the latest information for the [person profile](/docs/data/persons) associated with the `distinct_id` of the event **at the time the event is processed**. You should make sure to use events that occur on or after person properties would be merged (such as `$identify`) if you need this information for your event.
The `person` object will contain the latest information for the Person profile associated with the `distinct_id` of the event <b>at the time the event is processed</b>. You should make sure to use events that occur on or after person properties would be merged (such as `$identify`) if you need this information for your event
### How many events can I send to a destination?
#### How many events can I send to a destination?
There is no limit on the number of events to be processed but the system requires that the destination responds with healthy status codes (2xx) and in a timely fashion.
There is no limit on number of events to be processed but the system requires that the destination responds with healthy status codes (2xx) and in a timely fashion. If the destination performs poorly (too many errors, too slow) for a prolonged period of time your destination will be quarantined and eventually disabled. We will try to re-enable it automatically after a temporary disabled period before stopping it entirely. You then need to modify either the filters or the destination and re-enable it in the UI to try again.
If the destination performs poorly (too many errors, too slow) for a prolonged period, your destination will be quarantined and eventually disabled. We will try to re-enable it automatically after a temporary disabled period before stopping it entirely. You then need to modify either the filters or the destination and re-enable it in the UI to try again.
10 changes: 7 additions & 3 deletions contents/docs/cdp/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,20 @@ title: Data pipelines
showTitle: true
---

Data pipelines include transformations and destinations. Transformations extend PostHog's functionality by filtering and transforming data. Destinations sends PostHog data to other sources and includes [batch exports](/docs/cdp/batch-exports) which reliably send data to a destination on a schedule.
Data pipelines include transformations and destinations. Transformations extend PostHog's functionality by filtering and transforming data. Destinations send PostHog data to other sources in [realtime](/docs/cdp/destinations) or as [batch exports](/docs/cdp/batch-exports) which reliably send data to a destination on a schedule.

> Destinations require the data pipeline add-on in [your billing settings](https://us.posthog.com/organization/billing).
Destinations require the data pipeline add-on in [your billing settings](https://us.posthog.com/organization/billing).

## Use cases

Pipelines can be used for a wide variety of use cases, such as:

- **Send event data to a data warehouse.** If you have a data lake or data warehouse, you can use destinations to send PostHog event data there, while ensuring you still have that data in PostHog to perform your analytics processes.

- **Send event data via webhooks.** You can use our realtime destinations to send event data to external services through webhooks. This is useful when you want to push event data to tools like Slack, Hubspot, or Intercom.

- **Enforce event schemas.** By default, PostHog does not enforce schemas on events it receives. However, a transformation could do so, preventing ingestion of events that do not match the specified schema in order to keep your data clean and following specific guidelines you need it to follow.

- **Label events.** To facilitate sorting through your events, you can use transformations to determine arbitrary logic to label an event (e.g. by setting a `label` property). This can help you tailor your metrics in PostHog, as well as facilitate data ordering if you ever use PostHog data elsewhere.

For a full list of transformations and destinations currently available , see the [PostHog codebase](https://github.com/PostHog/posthog/blob/6762a4869c0c0ad50263ba09d493da4716603ab1/frontend/src/scenes/pipeline/utils.tsx#L40).
For a full list of transformations and destinations currently available, see the [destinations](https://us.posthog.com/pipeline/destinations) and [transformations](https://us.posthog.com/pipeline/transformations) tabs under data pipeline in-app.
44 changes: 14 additions & 30 deletions src/navs/index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2660,15 +2660,12 @@ export const docsMenu = {
children: [
{
name: 'Data pipelines (CDP)',
badge: {
title: 'Preview',
className: 'uppercase !bg-blue/10 !text-blue !dark:text-white !dark:bg-blue/50',
},
},
{
name: 'Overview',
url: '/docs/cdp',
icon: 'IconHome',
color: 'seagreen',
},
{
name: 'Realtime destinations',
Expand All @@ -2678,7 +2675,15 @@ export const docsMenu = {
className: 'uppercase !bg-blue/10 !text-blue !dark:text-white !dark:bg-blue/50',
},
icon: 'IconLive',
color: 'salmon',
children: [
{
name: 'Overview',
url: '/docs/cdp/destinations',
},
{
name: 'Destinations',
},
{
name: 'Slack',
url: '/docs/cdp/destinations/slack',
Expand Down Expand Up @@ -2777,6 +2782,7 @@ export const docsMenu = {
className: 'uppercase !bg-blue/10 !text-blue !dark:text-white !dark:bg-blue/50',
},
icon: 'IconShare',
color: 'purple',
children: [
{
name: 'Amazon S3',
Expand All @@ -2800,12 +2806,11 @@ export const docsMenu = {
},
],
},
{
name: 'Connectors',
},
{
name: 'Transformations',
url: '',
url: '/docs/cdp/geoip-enrichment',
icon: 'IconWrench',
color: 'yellow',
children: [
{
url: '/docs/cdp/geoip-enrichment',
Expand Down Expand Up @@ -2849,32 +2854,11 @@ export const docsMenu = {
},
],
},
{
name: 'More',
},
{
name: 'Troubleshooting and FAQs',
url: '/docs/cdp/common-questions',
icon: 'IconQuestion',
},
{
name: 'Build a connector',
url: '/docs/cdp/build',
icon: 'IconGraduationCap',
children: [
{
name: 'Overview',
url: '/docs/cdp/build',
},
{
name: 'Developer reference',
url: '/docs/cdp/build/reference',
},
{
name: 'Tutorial',
url: '/docs/cdp/build/tutorial',
},
],
color: 'blue',
},
],
},
Expand Down

0 comments on commit a975c94

Please sign in to comment.