Skip to content

Commit

Permalink
Merge branch 'master' into anonymous-events
Browse files Browse the repository at this point in the history
  • Loading branch information
corywatilo authored Oct 2, 2024
2 parents 081d8a3 + 42fd076 commit e60144c
Show file tree
Hide file tree
Showing 17 changed files with 190 additions and 30 deletions.
2 changes: 1 addition & 1 deletion contents/blog/how-we-improved-feature-flags-resiliency.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Contrast it with the PostHog interface not loading, where the problem is constra

Further, flags are very sensitive to latency. If it takes 5 seconds for your flags to evaluate, that holds up your customer's application for 5 seconds. You can't wait to load them asynchronously, either, as you need this result to determine what to show. Your business logic depends on the flag.

This is why the [Feature Success team](/teams/feature-success) has spent the last few months making PostHog's feature flags fast and resilient. Our goal was to ensure that:
This is why the [Feature Flags team](/teams/feature-flags) has spent the last few months making PostHog's feature flags fast and resilient. Our goal was to ensure that:

1. Neither the SDKs that query flags, nor the API interface, should go down if the PostHog interface does
2. Latency-sensitive flags resolve in 50ms or less.
Expand Down
11 changes: 4 additions & 7 deletions contents/handbook/engineering/feature-ownership.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ You can also view the list [directly in GitHub](https://github.com/PostHog/posth
| Data Warehouse | [Team Data Warehouse](teams/data-warehouse) | <span class="lemon-tag gh-tag">feature/data-warehouse</span> |
| Error tracking | [Team Error Tracking](/teams/error-tracking) | <span class="lemon-tag gh-tag">feature/error-tracking</span> |
| Events | [Team Product Analytics][Team Product Analytics] | <span class="lemon-tag gh-tag">feature/events</span> |
| Experimentation | [Team Feature Success][Team Feature Success] | <span class="lemon-tag gh-tag">feature/experimentation</span> |
| Feature Flags | [Team Feature Success][Team Feature Success] | <span class="lemon-tag gh-tag">feature/feature-flags</span> |
| Experimentation | [Team Experimentation][Team Experimentation] | <span class="lemon-tag gh-tag">feature/experimentation</span> |
| Feature Flags | [Team Feature Flags][Team Feature Flags] | <span class="lemon-tag gh-tag">feature/feature-flags</span> |
| Group Analytics | [Team Product Analytics][Team Product Analytics] | <span class="lemon-tag gh-tag">feature/group-analytics</span> |
| HogQL | [Team Product Analytics][Team Product Analytics] | <span class="lemon-tag gh-tag">feature/dashboards</span> |
| Heatmaps | [Team Replay][Team Replay] | <span class="lemon-tag gh-tag">feature/heatmaps</span> |
Expand All @@ -56,6 +56,7 @@ You can also view the list [directly in GitHub](https://github.com/PostHog/posth
| Session Analytics | [Team Product Analytics][Team Product Analytics] | <span class="lemon-tag gh-tag">feature/sessions</span> |
| Settings (personal & project) | [@benjackwhite][@benjackwhite] | <span class="lemon-tag gh-tag">feature/settings</span> |
| SSO | [@mariusandra][@mariusandra] | <span class="lemon-tag gh-tag">feature/sso</span> |
| Surveys | [Team Surveys][Team Surveys] | <span class="lemon-tag gh-tag">feature/surveys</span> |
| Toolbar | [Team Replay][Team Replay] | <span class="lemon-tag gh-tag">feature/toolbar</span> |
| Web Analytics | [Team Web Analytics][Team Web Analytics] | <span class="lemon-tag gh-tag">feature/web-analytics</span> |
| Webhook delivery service | [Team Pipeline][Team Pipeline] | <span class="lemon-tag gh-tag">feature/pipelines</span> |
Expand All @@ -73,21 +74,17 @@ Some of the features we are building may exist in other products already. It is

[@benjackwhite]: https://github.com/benjackwhite
[@EDsCODE]: https://github.com/EDsCODE
[@liyiy]: https://github.com/liyiy
[@macobo]: https://github.com/macobo
[@mariusandra]: https://github.com/mariusandra
[@neilkakkar]: https://github.com/neilkakkar
[@pauldambra]: https://github.com/pauldambra
[@rcmarron]: https://github.com/rcmarron
[@Twixes]: https://github.com/Twixes
[@yakkomajuri]: https://github.com/yakkomajuri
[@timgl]: https://github.com/timgl
[Team Product Analytics]: /teams/product-analytics
[Team Web Analytics]: /teams/web-analytics
[Team Replay]: /teams/replay
[Team Pipeline]: /teams/pipeline
[Team Infrastructure]: /teams/infrastructure
[Team Feature Success]: /teams/feature-success
[Team Feature Flags]: /teams/feature-flags
[Team Infrastructure]: /teams/infrastructure
[Team Growth]: /teams/growth
[Team ClickHouse]: /teams/clickhouse
Expand Down
2 changes: 2 additions & 0 deletions contents/handbook/engineering/posthog-com/small-teams.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ To change your team's image, send it to [#team-website-and-docs](https://posthog
4. [Create a new team on GitHub](https://github.com/orgs/PostHog/teams) and remove the new members from their previous team
5. Create the new `feature/team-{team-name}` labels [on GitHub](https://github.com/PostHog/posthog/labels)
6. On Slack, create a new channel called #team-{team-name}. Add a new People > User group with the handle @team-{team-name}-folks. Add / Remove people from other groups as necessary.
7. If there are existing forum topics or roadmap items, re-assign them to the new team.
8. Update small team names in [Ashby](https://app.ashbyhq.com/admin/organizational-settings/custom-fields/job/6897f288-b46e-4d51-9bd9-aeb20ad8e62f).


### Removing a small team
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ User interviews are powerful but slow. You can quickly narrow down the most impo
After the initial discussion, get your user on Slack for B2B or iMessage for B2C – or whatever messaging tool works best for them. This will enable you to quickly follow up with mockups or further questions during the solution validation phase.
Here's an example from Li on our [Feature Success Team](/teams/feature-success) seeking feedback on our feature flag permissioning. She received a response 3 minutes (!!) later.
Here's an example from Li on our Feature Success Team seeking feedback on our feature flag permissioning. She received a response 3 minutes (!!) later.
![Slack feedback on feature flags](https://res.cloudinary.com/dmukukwp6/image/upload/v1710055416/posthog.com/contents/images/blog/10x-engineers-do-user-interviews/feature-flags-feedback.png)
Expand Down
2 changes: 1 addition & 1 deletion contents/product-engineers/how-to-create-user-personas.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ Your user persona is the voice of your user. It should guide your product develo

Incorporate them into team meetings and encourage your team to always ask, "How will this benefit our key user personas?". By doing so, you build a culture of empathy for your users, ensuring that you make the best products for them.

For instance, at PostHog our product teams use our personas to create their team mission, values, and quarterly goals. Since our company handbook is public, you can view precisely how by visiting each team's page – e.g., our [Feature Success](/teams/feature-success#personas) and [Web Analytics](/teams/web-analytics#who-are-we-building-for) teams.
For instance, at PostHog our product teams use our personas to create their team mission, values, and quarterly goals. Since our company handbook is public, you can view precisely how by visiting each team's page – e.g., our [Experiments](/teams/experiments#personas) and [Web Analytics](/teams/web-analytics#who-are-we-building-for) teams.

## Frequently asked questions

Expand Down
4 changes: 2 additions & 2 deletions contents/product-engineers/measuring-feature-success.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ crosspost:
- Blog
---

Is my new feature working? It's the question PostHog exists to answer – we even have [an entire team dedicated to it](/teams/feature-success).
Is my new feature working? It's the question PostHog exists to answer – we even have [an entire team dedicated to it](/teams/experiments).

We define a successful new feature as one that:

Expand Down Expand Up @@ -129,7 +129,7 @@ You may have noticed that this process is relatively fluid. We don’t enforce m

This is because we bias for impact, and believe that ownership is key to building successful features. Without people or teams owning features, there is little incentive to make them successful. No process can make ownerless features successful.

We know that every team, inside and outside of PostHog, has different ways of evaluating success, it's a [key focus of ours](/handbook/strategy/overview#strategy) to build tools to help them do this. If you're curious exactly what that looks like, you can check out what the feature success team is up to on [their small team page](/teams/feature-success).
We know that every team, inside and outside of PostHog, has different ways of evaluating success, it's a [key focus of ours](/handbook/strategy/overview#strategy) to build tools to help them do this. If you're curious exactly what that looks like, you can check out what the Experiments team is up to on [their small team page](/teams/experiments).

## Further reading

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Feature Success
title: Experiments
sidebar: Handbook
showTitle: true
hideAnchor: false
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,4 @@
As always, reliability is the #1 unwritten goal: Making sure feature flags are reliable trumps every other objective.

### Objective: Make sure feature flags can handle 10x current scale

In the beginning of 2024, we hit some scaling limits on flags: It's now very expensive to run flags on Django and we'll hit some scaling limits at 5x our scale.

To get ahead of this problem, we'll rewrite our flags service to be more performant and reliable. This is a continuation of last quarter's goal, since we aren't done yet.

### Objective: No-code experiments
### Objective: No-code experiments

Last quarter we polished up our new experiment UI. This quarter, we want to address some issues that came up, and then focus on building out some more ambitious features on top of experiments.

Expand Down
26 changes: 26 additions & 0 deletions contents/teams/feature-flags/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: Feature Flags
sidebar: Handbook
showTitle: true
hideAnchor: false
template: team
---

## What this team does

We build the feature flag platform, SDKs, API, and UI to help users safely ship new features and make data-driven decisions.

Here are some of the things we own:

- **Feature flag SDKs** - SDKs in JS/TS, Python, Rust, PHP, Go, Ruby, iOS, Flutter,and Android.
- **Feature flag API** - both the soon-to-be-legacy `/decide` endpoint and the new `/flags` endpoint
- **Flag manager UI** - a way to manage your feature flags in PostHog

We write Rust, Python, and TypeScript. We're big fans of automated testing, static types, and banter.

## Slack channel

[#team-feature-flags](https://posthog.slack.com/messages/team-feature-flags)

## Feature ownership
You can find out more about the features we (and the other teams) own [here](/handbook/engineering/feature-ownership)
19 changes: 19 additions & 0 deletions contents/teams/feature-flags/objectives.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
### Q4 2024 Objectives

As always, reliability is the #1 goal: Making sure feature flags are reliable and performant trumps every other objective.

### Objective 1: Make sure the new feature flags service (the `/flags` endpoint) can handle 10x current scale with a 2x speed increase

In the beginning of 2024, we hit some scaling limits on flags: It's now very expensive to run flags on Django and we'll hit some scaling limits at 5x our scale.

To get ahead of this problem, [we decided]((https://github.com/PostHog/posthog/issues/22131)) to [rewrite our flags service](https://github.com/PostHog/posthog/issues/22131) to be more performant and reliable.

As of Q3 2024, we were at 90% feature parity with the `/decide` endpoint. Q4 will be when we roll out the new feature flag service to all users.

### Objective 2: Maintain and improve reliability of the existing feature flag platform (the `/decide` endpoint)

While we roll out the new feature flag platform, we need to maintain and improve reliability of the existing feature flag platform, which mostly consists of:

- Fixing bugs in the existing feature flag platform
- Improving alerting and monitoring of the existing feature flag platform
- Improving the documentation of the existing feature flag platform
93 changes: 93 additions & 0 deletions contents/teams/surveys/index.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: Surveys
sidebar: Handbook
showTitle: true
hideAnchor: false
template: team
---

## Values
- Fast, iterative and high output rather not slow and thoughtful - achieving this
- Feedback-driven not spec-driven - we do a decent job at this
- Missionary (we have a clear problem definition and are aligned on how impactful a solution would be) not mercenary - glimpses of this
- Collaborative not lone wolf - glimpses of this

## Personas

### Company Persona
- Primary
- Size:
- 20-75 employees
- Stage:
- Post-PMF
- Series A-D
- Customer type:
- B2B/B2C/(B2B2C)
- High expectation traits:
- Use the modern data stack
- Frontend uses typescript and react
- High-growth
- Not:
- API companies
- Shopify stores/no-code companies

### User Persona
- Primary
- Role
- Product-minded front-end engineer
- Growth engineer
- Seniority
- Decision-making seat on product
- Senior engineer
- IC
- High expectation traits
- Reads HackerNews
- Educated about the other feature flagging/experimentation tools in the space
- Needs high-reliability and high-performance
- Uses best-in-class tools such as Linear/Figma
- Secondary:
- Role:
- Product Manager
- Not:
- Role:
- Backend engineer
- Marketing

## Jobs to be done

### Surveys
- Primary
- Gather customer feedback on your website.
- Turn customer feedback into actionable insights that drive your product development.
- Secondary
- Meet your customers where they are, email or mobile surveys.
- Survey automation to take action based on survey responses.

## Feature ownership
You can find out more about the features we own [here](/handbook/engineering/feature-ownership)

## Long term vision

Imagine Bob is a product manager, and Alice is an engineer, both of whom love using PostHog.

During their weekly growth review, Posthog shows them that one of their workflows is performing 50% worse than other SaaS companies with a similar flow. They decide to build a new feature together, but they're unsure of the impact, so Bob & Alice decide to gate the feature via a feature flag.

Alice builds the feature and runs the PostHog CLI, automatically converting his feature branch to a feature-flagged version. During creation, he selects the team template they normally use, called "Autorollout based on conversion metric", using the conversion metric that Posthog suggests. The feature progressively rolls out to internal users, then to beta users, then to remaining users. If their conversion metric falls by more than 20% the feature automatically rolls back and alerts their team. Alice requests a feature flag review from Bob.

Bob checks the Posthog UI and because it's such an important feature - adds a safety condition for Sentry errors increasing by 30% and a few counter metrics. This should result in an automatic rollback as well. Bob starts the experiment.

Thankfully, nothing goes wrong when the feature is rolled out. The team is disappointed that the feature doesn't seem to move any of the core company metrics, however. This doesn't fit into either of Alice's or Bob's model, so they dig deeper why this was the case.

Before they even start, PostHog automatically does some impact analysis on their core metrics, and generates some insights into what properties are highly correlated with conversion & which aren't.

As it turns out, people in USA and India love their new feature and show a 40% increase in conversion. Other countries, especially the UK, seem to dislike it so much that it negatively affects conversion. In the end, these forces balance out, leading to similar total conversion rates.

They suspect it might have something to do with their positioning in other countries, so they run a marketing experiment using PostHog, where PostHog automatically generates recommended copy text to try out. It generates 5 variants, and they test these in all countries.

As it turns out, copy wasn't the issue, and there's no significant change here. They watch a few recordings from the experiment to confirm there's nothing off here.

Since it's not a positioning issue, Bob & Alice decide that it makes sense to introduce some personalisation, and let people opt-in to the new feature, and have it on by default for USA and India. They can customise this right from the feature flag, and set this up such that any users who opt-in on their UI automatically get the flag.

PostHog keeps analysing metrics for this flag over time, and notifies Bob and Alice when their customers behaviour change. For example, if the conversion for users in UK has taken a turn for the better, or if enterprise customers have taken a turn for the worse.

Our long term vision is to make all of this possible.
19 changes: 19 additions & 0 deletions contents/teams/surveys/objectives.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
### Objective: Surveys

Last quarter we polished up our new surveys UI.
This quarter, we want to address some issues that came up, and then focus on building out some more ambitious features on top of Surveys.

Broadly, we will:
1. Address outstanding experiments UI/UX issues
2. Build no-code experiments
3 Enterprise-3000 for surveys
* Global CSS
* Approvals
* Previews
4 Progress bars for Surveys
5 Back and Forward buttons for Multiple question surveys.
6 Backend events for survey activations.
7 Email surveys, which link out to full-screen surveys/ embed survey questions in them.
8 Work on full-screen survey forms.


4 changes: 2 additions & 2 deletions src/components/Product/AbTesting/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ const product = {
freeTier: '1,000,000 requests',
}

const team = 'Feature Success'
const teamSlug = '/teams/feature-success'
const team = 'Experiments'
const teamSlug = '/teams/experiments'

const featuresPerRow = 3
const features = [
Expand Down
4 changes: 2 additions & 2 deletions src/components/Product/FeatureFlags/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ const product = {
freeTier: '1,000,000 requests',
}

const team = 'Feature Success'
const teamSlug = '/teams/feature-success'
const team = 'Feature Flags'
const teamSlug = '/teams/feature-flags'

const featuresPerRow = 3
const features = [
Expand Down
4 changes: 2 additions & 2 deletions src/components/Product/Surveys/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ const product = {
freeTier: '250 survey responses',
}

const team = 'Feature Success'
const teamSlug = '/teams/feature-success'
const team = 'Surveys'
const teamSlug = '/teams/surveys'

const featuresPerRow = 3
const features = [
Expand Down
12 changes: 10 additions & 2 deletions src/navs/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -1012,8 +1012,12 @@ export const companyMenu = {
url: '/teams/error-tracking',
},
{
name: 'Feature Success',
url: '/teams/feature-success',
name: 'Experiments',
url: '/teams/experiments',
},
{
name: 'Feature Flags',
url: '/teams/feature-flags',
},
{
name: 'Infrastructure',
Expand All @@ -1027,6 +1031,10 @@ export const companyMenu = {
name: 'Replay',
url: '/teams/replay',
},
{
name: 'Surveys',
url: '/teams/surveys',
},
{
name: 'Web Analytics',
url: '/teams/web-analytics',
Expand Down
4 changes: 4 additions & 0 deletions vercel.json
Original file line number Diff line number Diff line change
Expand Up @@ -1346,6 +1346,10 @@
"source": "/teams/pipeline",
"destination": "/teams/cdp"
},
{
"source": "/teams/feature-success",
"destination": "/teams/feature-flags"
},
{ "source": "/docs/cdp/airbyte-export", "destination": "/docs/cdp/batch-exports" },
{ "source": "/docs/cdp/avo-inspector", "destination": "/docs/cdp/destinations/avo" },
{ "source": "/docs/cdp/customer-io", "destination": "/docs/cdp/destinations/customerio" },
Expand Down

0 comments on commit e60144c

Please sign in to comment.