Symbols and HTML entities
Countly Server: 22.03
Countly iOS SDK: 21.11.2
Something very strange and frustrating is happening with all of my event names and metrics submitted into Countly: all metrics data and event names that contain symbols are now showing up as HTML encoded entities in the Countly dashboard.
Upon investigation, I noticed that this information is ALSO encoded as HTML entities in the data submitted into the countly endpoints by the Countly SDK on the device.
For instance, an event is logged like this:
{
"app_key": "087052fde5212020965a8babf8e219f1696fccc5",
"device_id": "096DEA23-D8E5-4B58-A628-4147D896224A",
"timestamp": "1655232675997",
"hour": "14",
"dow": "2",
"tz": "-240",
"sdk_version": "21.11.0",
"sdk_name": "objc-native-ios",
"events": [
{
"dur": 0,
"hour": 14,
"segmentation": {
"app_runCount": "741",
"src_line": "52",
"id_device": ""52FDD2FB-875D-4C58-A01E-EA7464453F58"",
"src_function": ""application(_:willFinishLaunchingWithOptions:)"",
"id_vendor": ""48F63BC8-24CC-405E-8874-1C0C48AE2B2E"",
"app_theme": "".deep"",
"app_version": ""3.1.7"",
"app_reviewed": "2021-11-07 20:49:34 +0000",
"app_premium": "true",
"app_build": ""54"",
"app_environment": "private",
"app_appIcon": "Spectre.AppIcon.dark",
"app_rating": "5",
"app_container": ""app"",
"id_owner": ""096DEA23-D8E5-4B58-A628-4147D896224A"",
"app_colorfulSites": "true",
"src_file": ""AppDelegate.swift"",
"app_masterPasswordCustomer": "true",
"app_allowHandoff": "true"
},
"count": 1,
"key": "app >startup",
"timestamp": 1655232653833,
"dow": 2,
"sum": 0
},
You'll notice the metrics are getting their data HTML encoded, and even the event name is. Other elements in the request are not, such as the device_id at the top. This HTML encoding makes no sense whatsoever in the context of a JSON-encoded request. We ALSO notice that strings are getting an extra layer of " pre- & suffixed to them, further messing up the true data in these values.
It is interesting to compare this event request with the session request that came before it, where custom metrics were set:
{
"app_key": "087052fde5212020965a8babf8e219f1696fccc5",
"device_id": "096DEA23-D8E5-4B58-A628-4147D896224A",
"timestamp": "1655657712164",
"hour": "12",
"dow": "0",
"tz": "-240",
"sdk_version": "21.11.2",
"sdk_name": "objc-native-ios",
"begin_session": "1",
"metrics": {
"app_environment": "private",
"_device": "iPhone13,2",
"_carrier": "Freedom Mobile",
"_os": "iOS",
"app_allowHandoff": "true",
"_app_version": "3.1.7",
"app_version": "3.1.7",
"app_colorfulSites": "true",
"app_theme": ".deep",
"app_runCount": "744",
"app_rating": "5",
"_os_version": "15.5",
"id_vendor": "48F63BC8-24CC-405E-8874-1C0C48AE2B2E",
"app_reviewed": "true",
"_locale": "en_CA",
"id_device": "52FDD2FB-875D-4C58-A01E-EA7464453F58",
"app_premium": "true",
"app_build": "54",
"_device_type": "mobile",
"app_appIcon": "dark",
"id_owner": "096DEA23-D8E5-4B58-A628-4147D896224A",
"_resolution": "1170x2532",
"_density": "@3x",
"app_masterPasswordCustomer": "true",
"_store": "iOS",
"_source_channel": "Direct",
"_browser": "WebKit",
"_browser_version": "[webkit]_605.1.15"
},
"location": "",
"checksum256": "1cf4cba831dc10a3aad45365d0c9515a220cba7823913d679ea474beddfc56af"
}
We do NOT see these extra " elements in this request.
Note that on the device this data is being passed into the iOS Countly SDK as a Dictionary of NSString: NSString, and both the keys and values are clean (ie. non-encoded and non-encumbered with excess "" or " values).
I was under the impression that this was not a problem in the Countly Dashboard before the redesign, but perhaps the redesign has merely surfaced a problem that exists in the Countly Cocoa SDK.
Either way, please look into what is going on here - I need clean and reliable data, obviously.
This is a mess:
Comments
I am unsure if it is related, but I am also no longer seeing event details show up even though the Event Overview reports many events being recorded and the Request Log clearly sees them coming in.
Hi Ihunath,
Thank you for reaching out to us.
We're investigating it closely with our engineering team.
Could you please confirm if you are getting any errors on the browser console?
@... Certainly:
See http://ix.io/40Hd
Hi Ihunath,
Thank you for your patience.
We weren't able to reproduce either of the issues (the main one is about the SDK), and the second one:
Is regarding the server.
To better investigate it, could you please enable logging and then send the SDK side logs?
Regards,
(Prefixing this with a note: I discovered that the additional `"` characters at the leading and trailing end of my values were my fault, this has been fixed now)
As an example, from the Countly cocoa SDK logs on an iPhone device:
Which URL-decodes into the following value for the `events` query item:
[{"dur":0,"hour":11,"segmentation":{"app_container":"app","src_line":"117","src_file":"BaseUsersViewController.swift","items":"8","value":"1","src_function":"collectionView(_:didSelectItemAt:)"},"count":1,"key":"users >user","timestamp":1656863861262,"dow":0,"sum":0},{"dur":0.73727691173553467,"hour":11,"segmentation":{"app_container":"app","src_line":"563","src_file":"BaseUsersViewController.swift","src_function":"attemptBiometrics()","type":"biometric","result":"success"},"count":1,"key":"users #user","timestamp":1656863862000,"dow":0,"sum":0},{"dur":0,"hour":11,"segmentation":{"app_container":"app","biometricLock":"true","defaultType":"long","algorithm":"v3","avatar":"avatar-6","src_line":"225","sites":"138","src_file":"Tracker.swift","src_function":"login(user:)","maskPasswords":"false","loginType":"personal"},"count":1,"key":"user >signed_in","timestamp":1656863862002,"dow":0,"sum":0},{"dur":72.369281053543091,"hour":11,"segmentation":{"name":"Spectre.MainUsersViewController","segment":"iOS"},"count":1,"key":"[CLY]_view","timestamp":1656863789708,"dow":0,"sum":0},{"dur":0,"hour":11,"segmentation":{"function":"viewWillAppear(_:)","segment":"iOS","visit":1,"name":"Spectre.MainSitesViewController","line":"103","file":"BaseViewController.swift"},"count":1,"key":"[CLY]_view","timestamp":1656863862077,"dow":0,"sum":0}]Here, we can for instance see:
On the server side, under `Request Logs`, we find this event shown like this:
{"app_key": "---",
"device_id": "096DEA23-D8E5-4B58-A628-4147D896224A",
"timestamp": "1656863888776",
"hour": "11",
"dow": "0",
"tz": "-240",
"sdk_version": "21.11.2",
"sdk_name": "objc-native-ios",
"events": [
{
"dur": 0,
"hour": 11,
"segmentation": {
"app_container": "app",
"src_line": "117",
"src_file": "BaseUsersViewController.swift",
"items": "8",
"value": "1",
"src_function": "collectionView(_:didSelectItemAt:)"
},
"count": 1,
"key": "users >user",
"timestamp": 1656863861262,
"dow": 0,
"sum": 0
},
{
"dur": 0.7372769117355347,
"hour": 11,
"segmentation": {
"app_container": "app",
"src_line": "563",
"src_file": "BaseUsersViewController.swift",
"src_function": "attemptBiometrics()",
"type": "biometric",
"result": "success"
},
"count": 1,
"key": "users #user",
"timestamp": 1656863862000,
"dow": 0,
"sum": 0
},
{
"dur": 0,
"hour": 11,
"segmentation": {
"app_container": "app",
"biometricLock": "true",
"defaultType": "long",
"algorithm": "v3",
"avatar": "avatar-6",
"src_line": "225",
"sites": "138",
"src_file": "Tracker.swift",
"src_function": "login(user:)",
"maskPasswords": "false",
"loginType": "personal"
},
"count": 1,
"key": "user >signed_in",
"timestamp": 1656863862002,
"dow": 0,
"sum": 0
},
{
"dur": 72.36928105354309,
"hour": 11,
"segmentation": {
"name": "Spectre.MainUsersViewController",
"segment": "iOS"
},
"count": 1,
"key": "[CLY]_view",
"timestamp": 1656863789708,
"dow": 0,
"sum": 0
},
{
"dur": 0,
"hour": 11,
"segmentation": {
"function": "viewWillAppear(_:)",
"segment": "iOS",
"visit": 1,
"name": "Spectre.MainSitesViewController",
"line": "103",
"file": "BaseViewController.swift"
},
"count": 1,
"key": "[CLY]_view",
"timestamp": 1656863862077,
"dow": 0,
"sum": 0
}
],
"checksum256": "---"
}
Where we can see:
Should I expect a follow-up?
Please sign in to leave a comment.