Health app from Apple is an iOS application that measures different kind of activities of its OS user, like movement, sleep, diet, etc.
Demo can be found here
For this project I wanted to use personal data because I was curious if I could get some new insights about myself. Following the steps mentioned in the workflow I managed to display data over four charts:
- Step counts per day.
- Flights climbed per day.
- Distance walked or ran per day.
- Sleep cycle per day.
All the bar charts are displayed on the same x scale, namely time. Time is the only property that all data entries (Step count, flights climbed, etc.) have in common. In this way it is possible to view and compare all data entries per day, per week or per month. More detailed information will be displayed when hovering one the of the bars with your curser. Detailed information includes the exact value meauserd, and how this value differs, in percentages, from the mean. The mean is a dashed line that is drawn for every time period you are viewing.
Now I have explained the general idea of the data visualization. The stepts taken to work out the idea are as follows:
- Export data - Export an XML file from the Health app.
- Import data - Use
d3.xml()to load an XML file and map data within the callback. - Clean data - Filter all
<Record>elements found in the XML file. Select attributes for each<Record>and parse its value. - Transform data - Map cleaned data to a workable JSON object.
- Create axis - Create y scale based on min and max values found in JSON object. Create x scale based on a range of dates (start date and end date).
- Create charts - Draw bar charts based on newly formed JSON object. Value of each entry is displayed on the y axis and the x position of each bar is a moment in time.
- Add transitions - Animate scale and bars when data is filtered or loaded.
- Add events - Event listeners are added to chart elements to add interaction to the visualization.
Within the XML file I only selected the <Record> elements. Then for each <Record> element I filtered out the attributes and parse its value.
I have used the following code to clean the XML and transform it to a JSON object:
data = [].map.call(data.querySelectorAll('Record'), function (record) {
return {
type: parseType(record.getAttribute('type')),
value: parseValue(record.getAttribute('value')),
startDate: parseTime(record.getAttribute('startDate')),
endDate: parseTime(record.getAttribute('endDate')),
creationDate: parseTime(record.getAttribute('creationDate'))
}
});Example of JSON output (viewing one data entry):
{
"type": "stepCount",
"value": 21,
"startDate": "Sun Oct 29 2017 20:57:57 GMT+0100 (CET)",
"endDate": "Sun Oct 29 2017 21:02:57 GMT+0100 (CET)",
"creationDate": "Sun Oct 29 2017 21:05:01 GMT+0100 (CET)",
}There are four types of <Record> elements found within the XML. Step count, flights climbed and distance walked or ran are practically the same. Down below are all types and their XML samples:
<Record type="HKQuantityTypeIdentifierStepCount" sourceName="iPhone van Levi" sourceVersion="9.2" device="<<HKDevice: 0x174484920>, name:iPhone, manufacturer:Apple, model:iPhone, hardware:iPhone8,1, software:9.2>" unit="count" creationDate="2016-01-13 19:57:52 +0200" startDate="2016-01-13 19:08:24 +0200" endDate="2016-01-13 19:09:26 +0200" value="31"/><Record type="HKQuantityTypeIdentifierFlightsClimbed" sourceName="iPhone van Levi" sourceVersion="9.2" device="<<HKDevice: 0x174893150>, name:iPhone, manufacturer:Apple, model:iPhone, hardware:iPhone8,1, software:9.2>" unit="count" creationDate="2016-01-13 19:57:52 +0200" startDate="2016-01-13 19:29:43 +0200" endDate="2016-01-13 19:29:43 +0200" value="2"/><Record type="HKQuantityTypeIdentifierDistanceWalkingRunning" sourceName="iPhone van Levi" sourceVersion="9.2" device="<<HKDevice: 0x17089eaf0>, name:iPhone, manufacturer:Apple, model:iPhone, hardware:iPhone8,1, software:9.2>" unit="km" creationDate="2016-01-13 19:57:52 +0200" startDate="2016-01-13 19:08:24 +0200" endDate="2016-01-13 19:09:26 +0200" value="0.02432"/><Record type="HKCategoryTypeIdentifierSleepAnalysis" sourceName="Clock" sourceVersion="50" device="<<HKDevice: 0x174c8cf30>, name:iPhone, manufacturer:Apple, model:iPhone, hardware:iPhone8,1, software:10.0.2>" creationDate="2016-10-17 07:30:22 +0200" startDate="2016-10-17 00:30:00 +0200" endDate="2016-10-17 07:30:22 +0200" value="HKCategoryValueSleepAnalysisInBed">
<MetadataEntry key="_HKPrivateSleepAlarmUserWakeTime" value="2016-10-18 05:30:00 +0000"/>
<MetadataEntry key="_HKPrivateSleepAlarmUserSetBedtime" value="2016-10-15 22:30:00 +0000"/>
<MetadataEntry key="HKTimeZone" value="Europe/Amsterdam"/>
</Record>When a clean JSON object is created I transformed it to another JSON object where all entries are merged per day. All data types except for the SleepCycle can be merged easily.
To transform all activity data I use the d3.nest() function. Within this project I apply this function as follows:
/*
* Returns data merged, where all entries are merged to a day and stored as array element
*/
self.mergeDataPerDay = function (data, type) {
var dataPerDay = d3.nest()
.key(function (data) {
return data.startDate.toLocaleDateString();
})
.rollup(function (data) {
return d3.sum(data, function (group) {
return group.value;
});
})
.entries(data);
save(dataPerDay, type);
return dataPerDay;
};All entries will be on the same level. Every entry is identified by its date, which is transformed to a locale date string. Using the key function of D3 we can use the locale date string as key for each entry. Then within the rollup function the total sum of all values combined per day is returned. The entries function passes the data to the nest function.
It is a bit more complex to transform the SleepCycle data type. People tend to sleep before the new day begin, meaning sleeping before 00.00h and waking up the next day. So merging data per day requires a certain detection of hours I could fall asleep and hours I could wake up. The code to do that looks as follows (comments explain the code):
/*
* Returns sleep cycle data merged per day
* - Gets spread of hours accepted as minimal hours
* - Gets spread of hours accepted as maximal hours
* - Creates locale date string as key for each entry
* - Merges all entries to object {start: Date, end: Date, slept: number}
* - Checks if starting hours and waking hours of sleep are found within accepted spread of hours
* - Filters all undefined data out of data
* - Returns all entries with found values
* - Save sleepCyclePerDay to originalData
*/
self.createSleepCyclePerDay = function (data, minThreshold, maxThreshold) {
var minSpread = getMinHourSpread(minThreshold);
var maxSpread = getMaxHourSpread(maxThreshold);
var sleepCyclePerDay = d3.nest()
.key(function (data) {
return data.startDate.toLocaleDateString();
})
.rollup(function (data) {
return data.map(function (entry) {
var startTime = entry.startDate.getTime();
var endTime = entry.endDate.getTime();
var diffTime = endTime - startTime;
var startHours = entry.startDate.getHours();
var endHours = entry.endDate.getHours();
if (minSpread.indexOf(startHours) !== -1 && maxSpread.indexOf(endHours) !== -1) {
return {
start: entry.startDate,
end: entry.endDate,
slept: Math.round(diffTime / 36000) / 100 // 2 Decimal rounding: https://stackoverflow.com/questions/11832914/round-to-at-most-2-decimal-places-only-if-necessary
};
}
})
.filter(function (entry) {
return entry !== undefined;
});
})
.entries(data)
.filter(function (entry) {
return entry.value.length > 0;
});
save(sleepCyclePerDay, 'sleepCycle');
return sleepCyclePerDay;
};After all the transforming and cleaning is done, the data is drawn as a bar chart. Drawing a bar chart with a standard Enter, Update, Exit pattern is not that exciting to highlight in this documentation. That is why I [refer][u_general_enter_update_exit_pattern to a simple example of that pattern. What is exciting is the filtering and bar interaction that is added to this project. The following interactions are possible:
- Switch filter type - View data per week or per month.
- Navigate filter type - Navigate to next or previous type of timeframe (week or month).
- Hover the bar - View detailed information about each day by hovering a bar with your cursor.
timeFilter.js Adds two filter type buttons to the DOM. This includes filtering on 'week' and 'month'. When clicking on one of these buttons an event will be emited using Events.js.
Events.emit('timefilter/select', {
type: 'month'
});Using the pub sub pattern the created instances (BarChart and SleepCycle) trigger their subscription to this event and will filter the data accordingly. After filtering the data, the visualization will be redrawn within every instance.
/*
* Handles time filter select event
* Gets start date
* Gets end date
* Gets originalData based on type of chart
* Filters data based on new startdate, enddate and given originalData
* Sets new data which is filtered
* Sets new start date
* Sets new end date
* Sets max value of filtered data
* Emits filter done event when everything is set
*/
function handleTimeFilterSelect(data) {
var endDate = t.getEndDate();
var newStartDate = t.getStartDate();
var originalData = u.originalData[self.params.type];
var filteredData = u.filterDataOnDate(newStartDate, endDate, originalData);
self.params.data = filteredData;
self.params.startDate = newStartDate;
self.params.endDate = endDate;
self.params.maxValue = u.getMaxValue(filteredData);
Events.emit('barchart/filter/time/done');
}/*
* Handles time filter select event
* Gets start date
* Gets end date
* Gets originalData based on type of chart
* Filters data based on new startdate, enddate and given originalData
* Sets new data which is filtered
* Sets new start date
* Sets new end date
* Emits filter done event when everything is set
*/
function handleTimeFilterSelect(data) {
var endDate = t.getEndDate();
var newStartDate = t.getStartDate();
var originalData = u.originalData[self.params.type];
self.params.data = u.filterDataOnDate(newStartDate, endDate, originalData);
self.params.startDate = newStartDate;
self.params.endDate = endDate;
Events.emit('sleepcycle/filter/time/done');
}timeFilter.js Adds two filter navigation buttons to the DOM. This includes next and previous type of time frame ('week' or 'month'). When clicking on one of these buttons an event will be emited using Events.js.
Events.emit('timefilter/nav');Using the pub sub pattern the created instances (BarChart and SleepCycle) trigger their subscription to this event and will filter the data accordingly. After filtering the data, the visualization will be redrawn within every instance.
/*
* Handles time filter select event
* Gets start date
* Gets end date
* Gets originalData based on type of chart
* Filters data based on new startdate, enddate and given originalData
* Sets new data which is filtered
* Sets new start date
* Sets new end date
* Sets max value of filtered data
* Emits filter done event when everything is set
*/
function handleTimeFilterSelect(data) {
var endDate = t.getEndDate();
var newStartDate = t.getStartDate();
var originalData = u.originalData[self.params.type];
var filteredData = u.filterDataOnDate(newStartDate, endDate, originalData);
self.params.data = filteredData;
self.params.startDate = newStartDate;
self.params.endDate = endDate;
self.params.maxValue = u.getMaxValue(filteredData);
Events.emit('barchart/filter/time/done');
}/*
* Handles time filter nav event
* Gets start date
* Gets end date
* Gets originalData based on type of chart
* Filters data based on new startdate, enddate and given originalData
* Sets new data which is filtered
* Sets new start date
* Sets new end date
* Emits filter done event when everything is set
*/
function handleTimeFilterNav() {
var newEndDate = t.getEndDate();
var newStartDate = t.getStartDate();
var originalData = u.originalData[self.params.type];
self.params.data = u.filterDataOnDate(newStartDate, newEndDate, originalData);
self.params.startDate = newStartDate;
self.params.endDate = newEndDate;
Events.emit('sleepcycle/filter/time/nav');
}mouseover Events listeners are added to every bar in each chart. Within the handler function bound to the event listener an event is emitted using Events.js. The event looks as follows:
Events.emit('bar/on/mouseover', {
point: data
});The variable data is the data passed from the bar element which is hovered. This data object has a date as property. This date is used to call other bars that have same date. Then in their turn these bars will display their detailed information. Every instance (BarChart and SleepCycle) is subscribed to the bar/on/mouseover event.
/*
* Marks .bar element when is hovered over
* Also marks bar when a corresponding bar in another chart is hovered over.
* Adds text label above the bar
*/
function markBar(data) {
var bar = self.svg.select('[data-date="' + data.point.key + '"]')
.classed('active', true);
self.svg.append('text')
.attr('x', xTextPosition.bind(data.point))
.attr('y', yTextPosition.bind(data.point))
.attr('text-anchor', 'middle')
.attr('class', 'bar-label')
.text(getTextLabel.bind(data.point));
}The bar is marked by adding a active classname. Then a <text> element is added to hold detailed information about the bar. This detailed information is the exact value of bar and the percentage the value differs from the mean of the whole chart (e.g. '+5.1% (32)')
/*
* Returns text label content
* Uses queryDataByKey() to ensure it gets the value of this instance
*/
function getTextLabel() {
var data = queryDataByKey(this.key);
if (!data) {
return;
}
var value = u.round(data.value, 2);
var mean = u.round(getMean(), 2);
var diff = u.round((value / mean) * 100 - 100, 0);
var text = '+' + diff + '%' + ' (' + value + ')';
if (diff < 0) {
text = diff + '%' + ' (' + value + ')';
}
return text;
}/*
* Adds text element above bar element display detailed information
*/
function showDataOnBar(data) {
var bar = self.svg.select('[data-date="' + data.point.key + '"]')
.classed('active', true);
self.svg.append('text')
.attr('x', xTextPosition.bind(data.point))
.attr('y', yTextPosition.bind(data.point))
.attr('text-anchor', 'middle')
.attr('class', 'bar-label')
.text(getTextLabel.bind(data.point));
}The bar is marked by adding a active classname. Then a <text> element is added to hold detailed information about the bar. This detailed information is the time slept and the percentage the slept time differs from the slept mean of the whole chart. (e.g. '-7.5% (6.7h)')
/*
* Returns text content
* - Gets mean of time slept
* - Creates percent differnce of mean value
* - if is positive percent return '+' + text otherwise default number which is negative
*/
function getTextLabel() {
var data = queryDataByKey(this.key);
if (!data) {
return;
}
var mean = sleptMean();
var percent = u.round((data.value[0].slept / mean) * 100 - 100, 1);
var text = '+' + percent + '% (' + data.value[0].slept + ')';
if (percent < 0) {
text = percent + '% (' + data.value[0].slept + ')';
}
return text;
}This project has a couple of dependencies listed below. These dependencies are mandatory to get the code running without bugs. There are other includes in this project, but are not required to get the code working.
- MomentJS(v2.19.1) - Parse, validate, manipulate, and display dates and times in JavaScript.
- D3(v4) - D3.js is a JavaScript library for manipulating documents based on data.
- Events.js - Adds pub-sub pattern possibilities to your code.
- Utils.js - Adds a bundle of functions to do general data mutations and get information.
- BarChart.js - Creates a bar chart with given parameters.
- SleepCycle.js - Creates a sleep cycle bar chart with given parameters.
select()- select an element from the document.attr()- get or set an attribute.scaleTime()- create a linear scale for time.range()- set the output range.domain()- set the input domain.nice()- extend the domain to nice round numbers.selectAll()- select multiple elements from the document.transition()- schedule a transition on the root document element.duration()- specify per-element duration in milliseconds.call()- call a function with this selection.ticks()- generate representative values from a numeric interval.tickFormat()- set the tick format explicitly.timeFormat()- alias for locale.format on the default locale.timeDay- the day interval.axisBottom()- create a new bottom-oriented axis generator.data()- join elements to data.enter()- get the enter selection (data missing elements).exit()- get the exit selection (elements missing data).text()- get or set the text content.remove()- remove elements from the document.mean()- compute the arithmetic mean of an array of numbers.sum()- compute the sum of an array of numbers..classed()- get, add or remove CSS classes.scaleLinear()- create a quantitative linear scale.rangeRound()- set the output range and enable rounding.on()- add or remove event listeners.min()- compute the minimum value in an array.max()- compute the maximum value in an array.xml()- get an XML file.nest()- create a new nest generator.key()- add a level to the nest hierarchy.rollup()- specify a rollup function for leaf values.entries()- generate the nest, returning an array of key-values tuples.
format()- Format string based on format string.clone()- Clone a moment instance.subtract()- Subtract amount and type of time.add()- Add amount and type of time.toDate()- Convert moment to JS Date object.
getHours()- The getHours() method returns the hour for the specified date, according to local time.getMinutes()- The getMinutes() method returns the minutes in the specified date according to local time.setHours()- The setHours() method sets the hours for a specified date according to local time, and returns the number of milliseconds since 1 January 1970 00:00:00 UTC until the time represented by the updated Date instance.setMinutes()- The setMinutes() method sets the minutes for a specified date according to local time.setDate()- The setDate() method sets the day of the Date object relative to the beginning of the currently set month.https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/getDate()- The getDate() method returns the day of the month for the specified date according to local time.toLocaleDateString()- The toLocaleDateString() method returns a string with a language sensitive representation of the date portion of this date.Object.assign()- The Object.assign() method is used to copy the values of all enumerable own properties from one or more source objects to a target object. It will return the target object.bind()- The bind() method creates a new function that, when called, has its this keyword set to the provided value, with a given sequence of arguments preceding any provided when the new function is called.find()- The find() method returns the value of the first element in the array that satisfies the provided testing function.getElementById()- Returns a reference to the element by its ID; the ID is a string which can be used to uniquely identify the element, found in the HTML id attribute.createElement()- In an HTML document, the Document.createElement() method creates the HTML element specified by tagName, or an HTMLUnknownElement.setAttribute()- Sets the value of an attribute on the specified element.addEventListener()- The EventTarget.addEventListener() method adds the specified EventListener-compatible object to the list of event listeners for the specified event type on the EventTarget on which it is called.appendChild()- The Node.appendChild() method adds a node to the end of the list of children of a specified parent node.querySelectorAll()- Returns a NodeList representing a list of elements with the current element as root that matches the specified group of selectors.contains()- The Node.contains() method returns a Boolean value indicating whether a node is a descendant of a given node or not.remove()- The ChildNode.remove() method removes the object from the tree it belongs to.add()- The add() method appends a new element with a specified value to the end of a Set object.querySelector()- Returns the first Element within the document that matches the specified selector, or group of selectors, or null if no matches are found.split()- The split() method splits a String object into an array of strings by separating the string into substrings, using a specified separator string to determine where to make each split.filter()- The filter() method creates a new array with all elements that pass the test implemented by the provided function.map()- The map() method creates a new array with the results of calling a provided function on every element in the calling array.indexOf()- The indexOf() method returns the first index at which a given element can be found in the array, or -1 if it is not present.slice()- The slice() method returns a shallow copy of a portion of an array into a new array object selected from begin to end (end not included).isNaN()- The isNaN() function determines whether a value is NaN or not.
- Health Data from my iPhone - Released under the GNU General Public License, version 3.