Analytics: Is there a better way?

[GregTheGreek]

I'm opening this discussion as a result of #1247. Although the PR was done with good intentions some very valid points were brought up WRT where the data was being hosted, and what its being used for.

My main question is:

1. What analytics are we tracking, and what will that piece of data provide us?
2. If we need to track analytics, can we do this without a third party?

I'm not sure if we have other questions, it would be great to add more if we do, I can continue to edit the thread as needed.

cc: @Tbaut @sweetpea22 @priom

[aidanih]

I am in complete agreement in that we will not be using 3rd party hosted analytics tools. Thank you @Tbaut & @sweetpea22 for always ensuring we are living up to our mission, vision and values.

[ColinSchwarz]

I was just chatting with Priom about this. There is a self hosting option provided by posthog, his concern with using it is that it would take more time to set up and cost us more dev ops resources to maintain. IMO, this is worth the extra effort. My preference would be to launch Storage without these analytics and then take the time we need to provide a self hosted option that we can be proud of.

[GregTheGreek]

This makes sense, do we have high level what metrics we're looking for and why? I'm assuming some will help with KPIs and informing ux?

[jamesefhawkins]

Howdy - I'm from PostHog and stumbled into this discussion. A helpful framework for this generically is AARRR ("pirate" metrics):

• Acquisition (how many people are using your products - a sign if they're solving a need)
• Activation (how many people get value for the first time - are people figuring out how to use the product properly for the first time)
• Retention (are people getting enough value to come back)
• Referral (may / may not be relevant - is my product good enough to invite others to it?)
• Revenue (may / may not be relevant - how well are we monetizing our product)

Inside each of these, you can then break out into more detail. Of course, you can also answer ad hoc questions "is anyone using feature X?", "should we add support for more languages? How would that impact our conversion rate?", "is layout X better than layout Y?" (we provide feature flags integrated with our analytics to help you understand the impact of changes, or just to help with the release process)

Definitely recommend first-party usage especially for the nature of what you're working on!

[GregTheGreek]

I had an idea for a non-invasive way to track page duration. Imagine a scenario where we want to know if a certain page is confusing (maybe billing or settings). We could have a window.setTimer(..) go off after x seconds in which a small toast can appear in the bottom right corner asking the user if they're confused and to provide feedback. At this point we could store the number of times that toast triggered per page, but I see no reason to know which user it was.

Although this might not be the most intuitive way, I think using this as an example can help us find better ways to get feedback without actual tracking mouse movements etc...