Webtrekk missing hidden event contextual information

[defagos]

On mobile platforms (iOS and Android), Janis Berneker from SRF alerted us that hidden events are missing contextual information (e.g. app name, version) when received on Webtrekk. This is due to the fact that Webtrekk seems to automatically add hidden event contextual information based on the previous page view received.

We already discussed similar issues with Manuel Montanari from Webtrekk in the past. We should investigate the problem with him if he is available, as well as with the GD.

[defagos]

Here is the initial conversation I had with Janis:

Hi Samuel
I’m Janis from SRF and responsible for Digital Analytics & Data Science. I don’t know if you’re the right person to contact or if you have a ticketing system where we can post requests.

Why I’m contacting you:
We have the problem that some apps are sending hidden events before page views because the sent information is only available before the first page view. This was no problem with comScore but Webtrekk has the requirement that hidden events MUST be sent after a pageview.
Since all SRG apps are affected by this it would be great if this features (=caching hidden event and send it after the first page view) could be added directly to the library itself.

Please let me know if you’re the right person or feel free to redirect me.

Sunny greetings from Zurich
Janis

---

Janis,

Thanks for reaching out. We are the right people to contact for analytics-related issues. We also have a ticketing system, but let me answer your request by email first. I’ll discuss today with the rest of the team whether we should move the discussion to the ticketing system. We’ll send you in invite if this is the case.

I remember that we discussed specific Webtrekk hidden event behaviors with Manuel Montanari (from Webtrekk) at the time we implemented the new requirements (part of the discussion can be found on Confluence if you have access). We mostly addressed the question of dangling hidden events without page views (as we send one hidden event at application startup), but I cannot remember if we were informed of other problems related to the ordering of page views and hidden events.

Could you please provide us with more context so that we can better address this issue:

1. Did you find this issue yourself or did someone else find it? If yes, who else?
2. What is the negative impact this problem creates in your reports?
3. Are both Android and iOS apps affected in the same way? Are web measurements affected as well?
4. Can you provide us with some examples of applications which are affected?
5. Are people from the GD (Markus Gubler / David Schwelien) aware of this issue?
6. Which site identifier(s) are the apps using for TagCommander? 3666 or 3667?
7. Could you provide us with an example of the results you get and what you would have expected?

Thanks in advance for your help!

Best regards.

Samuel

---

Hi Samuel
Great, thanks for replying.

About your questions:

1. Did you find this issue yourself or did someone else find it? If yes, who else?
   I found it myself when testing a new feature of our SRF News App which has to be measured. It’s about measuring something in the menu which is rendered before loading the actual page.
2. What is the negative impact this problem creates in your reports?
   With Webtrekk a hidden event only ha a few parameters but automatically gets the parameters from previous pageview – like app name, app version and much more. As the hidden event has no previous page view we can’t assign it to a specific app etc.
3. Are both Android and iOS apps affected in the same way? Are web measurements affected as well?
   Yes, iOS and Android are affected as it’s a a Webtrekk behavior. Theoretically the same would happen in the web when a hidden event is fired before a page view. But I don’t know of any case where this happens as a page view is tracked right after loading a page.
4. Can you provide us with some examples of applications which are affected?
   SRF News App iOS/Android: “Installed Apps”-Hidden-Event, “Tooltip”-Hidden-Event (tracked only once!)
5. Are people from the GD (Markus Gubler / David Schwelien) aware of this issue?
   No, haven’t informed them yet. Markus is on holidays at the moment.
6. Which site identifier(s) are the apps using for TagCommander? 3666 or 3667?
   3666
7. Could you provide us with an example of the results you get and what you would have expected?
   I expect hidden events to be fired after page views.

Hope this helps!
Best
Janis

[defagos]

Inheriting hidden event contextual information is a problem IMHO (since some events might not be related to a page view, e.g. the initial application overlap hidden event we send).

If this is really a Webtrekk issue but we are ultimately only talking with the TagCommander agnostic service, I see following approaches:

• Add the missing context at the client level.
• Add the missing context at the TagCommander level.

We need to better understand how Webtrekk works first. I'll get in touch with the GD.

[pyby]

The Letterbox container on TagCommander (3666) has 3 tags for Webtrekk: Media (playback events), page view and hidden event. **They are totally independent today.**

The hidden event send a GET request to Webtrekk like this:
http://srgssr01.wt-eu02.net/#Webtrekk_track_id#/wt?p=#version#,#page_unique_name#,#client_time#,0&ceid=#TC_UNIQUE_ID#&x-wt-ip=#IP#&x-wt-ua=#TC_USER_AGENT#&ck1=#event_source#&ck2=#event_type#&ck3=#event_name#&ck4=#event_value#

From the iOS and Android tag commander SDK, we have more information on the hidden event, but not sent to Webbtrekk today. Here is a list from the app overlaps event (from an iOS simulator):

TC_IS_FIRST_VISIT	FALSE
TC_CONNEXION	WiFi
TC_MANUFACTURER	Apple
TC_LAST_SESSION_LAST_HIT	1532105526
TC_LAST_SESSION_LAST_HIT_MS	1532105526897
TC_TAGCOMMANDER_VERSION	4.1.3
TC_CURRENT_CALL	1532105528
TC_CURRENT_CALL_MS	1532105528845
TC_NOW	1532105528
TC_NOW_MS	1532105528845
TC_UNIQUEID	6C523153-A48A-466C-A8F8-74F965BD308F
TC_CHARSET	MacRoman
TC_SCREEN	375x812
TC_APPLICATION_BUILD	269
TC_MODEL	iPhone
TC_SYSVERSION	11.4
TC_SYSNAME	iOS
TC_LANGUAGE	en_US
TC_LANGUAGE_CS	en
TC_LANGUAGE_GA	en-US
TC_EMPTY
TC_RAND	1089586647
TC_IP	10.175.27.241
TC_BUNDLE_IDENTIFIER	ch.srf.srfplayer.debug
TC_APPLICATION_NAME	Play SRF
TC_RUNTIME_NAME	ios
TC_DEVICE	x86_64
TC_MODEL_AND_VERSION	Simulator
TC_JAILBROKEN	1
TC_VERSION_FIRST_VISIT_MS	1531903037499
TC_APPLICATION_PREVIOUS_VERSION	0
TC_APPLICATION_VERSION	2.8.2-debug
TC_APPLICATION_STARTS	18
TC_FIRST_VISIT	1528469971
TC_FIRST_VISIT_MS	1528469971000
TC_LAST_VISIT	1532003351847
TC_LAST_VISIT_MS	1532003351847000
TC_CURRENT_VISIT	1532105526
TC_CURRENT_VISIT_MS	1532105526000
TC_CURRENT_SESSION	1532105527
TC_CURRENT_SESSION_MS	1532105527627
TC_NUMBER_VISIT	35
TC_NUMBER_SESSION	36
TC_LAST_SESSION_START_MS	1532003351737
TC_NEW_SESSION	1
TC_CS_START_DIT	101856598
TC_CS_START_IT	101856598
TC_CS_START_DFT	2141
TC_CS_START_FT	318572
TC_USER_SESSION_DURATION_MS	101856598
TC_USAGE_SESSION_DURATION_MS	9517764756
TC_FIRST_EXECUTE	FALSE
TC_CURRENCY_CODE	USD
TC_CURRENCY_SYMBOL	$
TC_USER_AGENT	Mozilla/5.0 (iPhone; CPU iPhone OS 11_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15F79
TC_DELTA_BACKGROUND_TIME	0
TC_FOREGROUNDS	1
TC_SDK_ID	65F05C00-A5B0-4933-888B-6FD212EA5DE6
TC_NORMALIZED_ID	65F05C00-A5B0-4933-888B-6FD212EA5DE6
TC_SESSION_DURATION	2
TC_SESSION_DURATION_MS	9517767601
TC_LAST_CALL	1532105528
TC_LAST_CALL_MS	1532105528764
TC_FOREGROUND_TIME	1949
TC_DELTA_FOREGROUND_TIME	81
TC_BACKGROUND_TIME	0
event_type	hidden
event_value
event_source	SRGAnalytics
event_id	hidden_event
event_name	Installed Apps
app_library_version	3.3.7
navigation_app_site_name	srf-player-ios-v
navigation_environment	preprod
navigation_device	phone

@Taopi :

• Could we send more information in the hidden event to Webtrekk to help Janis? Or it's the limitation described in his mail?
• Could we force to wait the first page view on tag commander to send hidden event? (from my knowledge, the answer is NO).

[Taopi]

Hi

Could we force to wait the first page view on tag commander to send hidden event? (from my knowledge, the answer is NO).

No! AFAIK, that is not possible. Just to be 100% sure I will sent mail to help@commandersact.

Could we send more information in the hidden event to Webtrekk to help Janis? Or it's the limitation described in his mail?

Yes! Webtrekk offers a functionality to add metadata to the event sent to Webtrekk in the http://srgssr01.wt-eu02.net/#Webtrekk_track_id#/wt? beacon. To do so, variables corresponding to the query sting parameters have to created by me. Also the get URL sent with hidden_event must obviously be changed so that it actually contains these newly generated parameters (e.g. http://srgssr01.wt-eu02.net/#Webtrekk_track_id#/wt? ck1000=#new_request#.)

I have done this just now. For the event @pyby has created a screen shot of, I have added much more information to the Request URL.

In the new Version a hidden event send a GET request to Webtrekk, looks like this now:
http://#track_domain#/#track_id#/wt?p=#version#,#page_unique_name#,#client_time#,#referrer#&ct=#event_name#&ck1=#event_source#&ck2=#event_type#&ck3=#event_name#&ck4=#event_value#&ck11=#app_site_name#&ck6=#business_unit#&ck35=#app_library_version#&ck36=#tagcommander.container.tag#&ck41=#event_value_1#&ck42=#event_value_2#&ck43=#event_value_3#&ck44=#event_value_4#&ck45=#event_value_5#&la=#content_language#&ceid=#TC_UNIQUE_ID#&x-wt-ip=#IP#&x-wt-ua=#TC_USER_AGENT#&eor=1
(note that: ck35=#app_library_version#

I will now publish this newly mapped URL to production in V.8.07 on the TagCommander. This means that the users will have these additional data available from now on forward.

I will after that sent a Mail to Janis explaining how he can retrieve this newly add information in Webtrekk. Asking him, if this already solves his needs. I will keep SPA-Mobile in CC

BTW: "unfortunately" I am on break next week. I will check mails on monday 6th

[Taopi]

BTW: please note that I have added &ck2=#event_type#&ck41=#event_value_1#&ck42=#event_value_2#&ck43=#event_value_3#&ck44=#event_value_4#

If you equip these values you can this Webtrekk report to easily count how often a certain event of the group you define for #event_type# has been used.

(see this example I created for Ahmed Goumri)

[Taopi]

I just realized: what Janis probaly wats to analyse with event_name: " Installed Apps" is.... the installed apps :)...

This information on the he installed apps is missing in this table

If you passed the apps a user has installed (used to be a variable in ComScore somewhere) in the parameter event_value, then probably Janis can perform the needed analysis...

[pyby]

In my example, it was datas from the iOS Simulator on my Mac. the event_valueis empty. In nightlies, betas and production versions, we have real values.

I think Janis want to check other hidden events than just the "Installed Apps".

@Taopi if event_value_1, event_value_2, event_value_3 and event_value_4 are officials, can you update the app variables pages? Then we can add it the SRGAnalytics libraries, as "defined" properties.

[Taopi]

I have talked to [email protected]. The suggested solution "send more information in the hidden event to Webtrekk" has been rejected. Here is Janis words (translated by me):

Yes, the hidden event is measured. One problem is, that we need to sent parameters as with comScore again with the hidden event, which not only error-prone but also quite a parameter chaos results. A preferred solution would therefore be to send the hidden event before (sic!, I think he ment to write "after") the pageview. The only question is: should all developers program workarounds or can not implement this in the library?

[Taopi]

I have a suggestion for a fix. What I can do -to be proved in concept- in CommandesAct is to delay time between input and output hit:

I would not do it for the "generic hidden event" (i.e. Tag "Webtrekk - event" in Tag Manager), because if I did all hidden_events would be delayed.

I would duplicate the Tag "Webtrekk - event". Then for this "Webtrekk - event (copy)" I would use a constraint of the following kind (so that only Event executed very early in the session will be triggered with delay).

Of course I would have to put the inverse constraint to the original / not copied "Webtrekk - event". So it is not fired twice.

The downside to this solutions are:

• Double maintenance for changes ("Webtrekk - event" and "Webtrekk - event (copy)"
• still possible wrong attribution of event to page impression (if user opens another page faster that delay)

An alternative solution is, for every application, you sent event_id:"hidden" only after event_id:"page".

Please let me know, if you wish for me to make the "delay" change suggested here, or if you prefer another option

[defagos]

I turned this initially major issue into a minor one. We should check with Janis whether the solution proposed by @Taopi works.