Skip to content
Publicly Available Published by Oldenbourg Wissenschaftsverlag August 7, 2018

Sensing Home: Designing an Open Tool That Lets People Collect and Interpret Simple Sensor Data from Their Homes

  • Arne Berger

    Arne was born in the 70ies in the Eastern Bloc. He is an interaction design researcher and post doctoral researcher at Chemnitz University of Technology, Germany, where he explores novel modes of co-creation of smart connected technology in the context of the home.

    EMAIL logo
    , Albrecht Kurze

    Albrecht has a PhD in computer science. His research interests are networking aspects in all flavors and especially the Internet of Things. He develops not only the technology for it but is also interested in the innovation potential and the created implications.

    , Sören Totzauer

    Sören studied informatics. He is interested in bringing together social dynamics and technological possibilities. His current research interests focuses on Ideation Processes and Tools to foster creativity.

    , Michael Storz

    Michael is a human-computer interaction researcher. In his research he studies and develops interactive devices for groups. His current interest is in the participative R&D for groups of older adults.

    , Kevin Lefeuvre

    Kevin is a product designer with a master degree from Bauhaus-University Weimar. He is a tinkerer by profession and a user advocate by heart. At Miteinander he is exploring co-design methods and building co-design tools for designing smart connected fictions, scenarios, and devices.

    , Andreas Bischof

    Andreas studied sociology and cultural sciences. He is interested in the technical mediation of interaction and the technization of social situations. His current projects concern technology for age and ageing, and participative R&D with older adults.

    and Mira Freiermuth

    Mira is a sociologist. The methods of qualitative social research provide her with tools to accompany, understand and reflect on human-computer interaction. For her, the participation of people affected by technology in the process of development is most important; technology from people for people.

From the journal i-com

Abstract

The Internet of Things in the home is a design space with huge potential. With sensors getting smaller and cheaper, smart sensor equipped objects will become an integral, preinstalled part of the future home. With this article we will reflect on Sensing Home, a design tool to explore sensors in the home together with people. Sensing Home allows people to integrate sensors and connectivity into mundane domestic products in order to make them smart. As such, it can be used by people to experience and explore sensors in the home and daily life. They may explore possible use cases, appropriate sensor technology, and learn about this technology through use. At the same time people may also be empowered to understand the issues and implications of sensors in the home. We present the design rationale of Sensing Home, five usage examples of how Sensing Home allowed people to explore sensor technology, and the deployment of Sensing Home together with a self-developed group discussion method to empower people to understand the benefits and pitfalls of sensors in their home. The article ends with a brief reflection whether Sensing Home is a probe or a toolkit.

1 Introduction

More and more sensor equipped Internet of Things (IoT) devices in different forms and functions are deployed within the home. These smart objects can take many forms. There are, for example, dedicated devices for heating control, ambient light, or securing the front door. Moreover, simple sensors are also being integrated in already existing devices, in order to make them smart. The concept of the smart fridge is relatively old, but now on sale. Likewise, smart TVs contain sensors and are connected to the internet. Sensors might even augment the most mundane objects, like post boxes, flowerpots, or windows. In the future, sensors will become even smaller and cheaper and might be everywhere. Energy-self-sufficient connected sensors with only 2 cm³ volume for less than 10€ are already under development [16]. The vision of tiny sensors resembling Smart Dust of only a few cubic millimeters [14] is not far away. As such, sensors will become an integral, preinstalled part of the future home.

The IoT is a technology space with huge potential. Yet, it is also a particularly challenging topic for design. Because it combines tangible materiality, like people, places, and things with the intangible materiality of data, services, and networks. Assessing the rather uninspired use cases of most current smart home products, even expert designers arguably miss the mark when they create products and experiences within this design space. Likewise, lay designers equally struggle to imagine worthwhile uses of these smart home products, much less envision meaningful novel design scenarios. Here, the question arises how to allow end users to explore and interpret the opportunities of sensor technology in their homes.

By the same token, intelligence services, big data analysis companies, and criminals alike already exhibit a huge interest in collecting and interpreting as much data as possible about people. The power and potential of accumulating and interpreting, for example, social media data is relatively easy to grasp. The issues arising from these data breaches are tangible and imminent. In contrast to social media data, cameras, or microphones, IoT sensor data may seem much more intangible or abstract. IoT sensor data may even look mundane: What can a humidity sensor within a fridge reveal, after all? However, all the data of these sensors may have stories to tell. What can simple IoT sensors reveal about daily activities of people in their domestic environments? Various concerns about data security, data privacy, and power and control arise from implementing seemingly inconspicuous sensors in everyday objects. Here, the question arises how people can be empowered to understand not only the potential but also the pitfalls of sensor technology in their homes.

To investigate, we have developed Sensing Home. It represents the easy integration of sensors and connectivity in mundane domestic objects in order to “make them smart”. Sensing Home is thus one mean to meet two ends. It is a versatile design tool for people to experience and explore sensors in their home and daily life. They may explore and ideate possible use cases, appropriate sensor technology and learn about this technology through use. At the same time users can also be empowered to understand the issues and implications of sensors in the home in relation to data security, data privacy, and power and control.

With this article we will first discuss related work on design tools, probes, and toolkits for the IoT in the home, the concept of situated data, and similar studies within the context of the home. In the next section we will present the design rationale behind and design process of designing Sensing Home. The next section of this article will briefly present five usage examples of Sensing Home aimed at exploration of sensor technology together with people. The next section will present findings from deploying Sensing Home together with the group discussion method Guess The Data we developed, where we will briefly reflect on aspects of empowerment. The article closes with a brief reflection on lessons learned.

2 Related Work

In this section we will introduce the concepts of sensemaking and situated data. After that we will review related work in the context of sensor data in the home.

2.1 From Citizen Science to the Smart Home

Enabling and empowering lay-people to participatorily sense, record, document, and interpret environmental conditions based on low-cost IoT technology has gained significant interest in HCI design research. However, this research direction has not yet been extended to the domestic context. Currently, most examples are concerned with empowering citizens in measuring and understanding air pollution or radiation exposure in cities. For example, Smart Citizen Kit [6] enables users to “collect data and to measure, understand, and compare sensory qualities” of their city. It consists of sensors to measure air composition, temperature, humidity, light, and sound, as well as data-processing, data-transfer, and battery power. Users can place it within the city to explore issues like sound pollution or air quality. Data from all user deployed Smart Citizen Kits is collected and displayed together with the respective geolocation on a website. As such, users can connect and reflect on common issues. Another toolkit for the smart city is the Air Quality Egg. It also contains sensors to measure the air quality and to display collected data on a website [1]. These sensor toolkits for the smart city tend to focus on critical issues by combining the data from a large number of sensors to a given context like air quality. Yet, they also provide the freedom to explore several more issues in combining both data from the included sensors and from those sensors employed by other users.

A direct transfer of such “Citizen Science” toolkits to explore the meaning of IoT data in the context of the home is difficult. Because little is known about how people can and want to use and understand sensor data that depicts their living situation. The relationship between smart home data and their users only recently sparked interest in the HCI design research community. Smart home sensor probes have been deployed in participants’ homes to explore usability [5] and visualization [13] of smart home data. Comparably less interest has been paid to how participants, researchers, and algorithms actually can make sense of smart home data.

2.2 Sensemaking of IoT Data in the Home

Sensemaking is a central human activity. It happens, when data is loaded with meaning. This is a concept from ethnomethodology and comprises the methods with which humans describe their activities to constitute sense, e. g. data work or articulation work [20]. As such, it reconstructs processes of structuring and organizing communication in everyday worlds.

In the smart home, sensemaking can happen in various ways: Either as automated data work focusing on raw data, or as human-centered data work employing situatedness. Automated data work in form of machine learning is used to learn which sensors produce which data. By employing machine learning it is possible to predict data sources reliably [15]. While machine-based interpretation can discern the sources of many data-based patterns, it misses the context-sensitive situatedness. Automated data work is currently seen as unable to predict and explain context-sensitive situations the way human interpretation can, because of humans using their situated knowledge [12], [8], [23]. Yet, the everyday situated knowledge of those who design such algorithms and subsequent visualizations does indirectly influence end-users’ sensemaking of data. For example, sophisticated data presentation can hinder reasoning about privacy [17], [18].

Fischer et al. have shown that simple sensor data from the home is beneficial for energy advisors to optimize the energy consumption in the home [8]. The authors have shown that this simple sensor data can be used to successfully speculate about further meanings. For example, when residents leave or arrive at their home. Yet, making sense of this simple sensor data requires additional knowledge about the situatedness of the sensor data. In this study, energy advisors have been responsible for interpreting the data, before discussing it together with participants. This additional layer of sensemaking, seems at least in parts to influence participants’ own interpretations of the sensor data and introduces a certain power relation by design. In another example of human sensemaking through data exploration Tolmie et al. [23] have studied legibility of sensor data from sensor probes in people’s homes. In a two month study in three households they have gathered sensor data from four areas within a home. The data was then interpreted by participants in collaboration with researchers only some weeks later. The authors found that participants’ reasoning about the origin of the sensor data can be categorized as: place, time, people, practices, events, routine, exceptions and moral order. The authors conclude that this reasoning between household members is necessary for actual sensemaking. The Citizens Maker [17] is another sensor probe to collectively explore and share data between users. While this study is not focused on domestic data, it suggests that privacy concerns around personal sensor data can be alleviated through data abstraction (e. g. visualization). Similarly, if “raw data” is heavily preprocessed and transformed into “useful insights”, the data may be more helpful for a “primary task” but hinders sensemaking [18]. Contrarily, more data transparency can lead to more doubts about sharing data [24].

It is interesting that these studies did not give their participants full live access to the sensor data that is collected from their homes. In all the presented studies, the researchers had both exclusive access to the data and the preprocessing of this data. We argue, that the power relation between researchers and participants was heavily shifted towards the researchers who had the right and possibility to confront participants with preprocessed data. This is striking, because sensemaking of data has to start where, how, by and about whom data is collected. We deem it practically and ethically necessary to give participants full data sovereignty. As such, our research takes a participatory design approach.

3 Designing Sensing Home

Sensing home can be considered a design tool that incorporates properties of probes and toolkits. It is a probe inasmuch as we intent it to allow people to “share their own meanings” with us as designers. It is also a toolkit, because we use it to allow people to “construct and try interactive systems” [19]. The core of our concept is Sensing Home, which consists of several sensor devices to collect and browse sensor data. In addition, as a probe pack it also contains tools to document, report, and support data work. Taken together, these sensor devices can be deployed in the field for various applications. For example, Sensing Home can be used for open exploration, teaching in interdisciplinary settings, or collective data interpretation [2]. In the following section we will report on how we designed Sensing Home.

3.1 From Maker Based Device to Batch Deployment

We started designing Sensing Home with a maker-based approach. Our concept was developed based on the name, shape, and meaning of the “Kinder Surprise Egg” candy. In its original form this is a chocolate candy for kids containing a surprise like a toy or a collectible. We use this notion as a symbol for a small and cheap everyday object with a twist in it. Following this rationale, we developed the concept of the Kindred Surprise. Here, the surprise is not necessarily “kinder” than other seemingly harmless sensor devices. In addition, we wanted the single sensor devices of Sensing Home to be small and reasonably cheap. To achieve this, for our first working version we equipped the inner plastic capsule of a Kinder Surprise with off-the-shelf Maker components instead of toys: a battery, a MCU with wireless connection and some simple sensors. These components represent the basic functional blocks of the things in the IoT in an inconspicuous hull, resembling the principle of the new IoT paradigms (small, cheap, everywhere) at its core. While first field tests have been promising, this Maker approach did not scale well for our intended batch deployment. In addition, we wanted even smaller devices than our DIY prototypes, and we wanted the flexibility of including more sensors. In the development process we selected the TI SensorTag [22] as a suitable base for our further development.

Figure 1 
            First renditions of kindred surprise.
Figure 1

First renditions of kindred surprise.

Figure 2 
            Maker capsule.
Figure 2

Maker capsule.

3.2 Sensing Home

The technical base for the final version of Sensing Home is a modified TI SensorTag. The TI SensorTag has a very compact size, energy efficient BLE communication, and a large number of different sensors (luxmeter, hygrometer, infrared thermometer, barometer, accelerometer, gyroscope, magnetometer, simple key sensors and a reed switch). In addition, the price of about 30 EUR is very attractive. In contrast to most smart home products, the SensorTag allows complete control over the software and the data flows.

Figure 3 
            TI SensorTag.
Figure 3

TI SensorTag.

We modified the firmware of the SensorTags making them more useable for field studies, e. g., power optimizations for longer operations (up to two weeks), extended sensor intervals and on-device data preprocessing. Our sensor devices are small (45 × 35 × 8 mm) and lightweight enough (15 g) to attach them nearly everywhere. We modified and 3D-printed custom housings to offer a variety of mounting options. It is possible to attach a sensor e. g. via glue pad, rubber bands, or cable strips to an object, to hang it somewhere or to just put it down. Sensing Home also contains a tablet computer for providing visualizations of sensor data. All parts of Sensing Home are deployed in a box and are ready to use.

3.3 Towards Ethics by Design and Ethics through Design

We believe it is important not only to highlight possible benefits but also potential risks and implications of research. Therefore, ethical, legal, and social implications (ELSI) should be an integral part in design and research activities. This includes the necessity for us as researchers to behave ELSI compliant. However, many computer science faculties, including our own, still do not employ an ethics board, which leaves reflection on ethics to us researchers.

One aspect of ELSI is to raise literacy, fluency and by this the awareness for implications which are raised by technology. With Sensing Home we expand this notion to the usage and appropriation of ubiquitous IoT products. In this regard, we consider Sensing Home as a citizen science toolkit for the smart home. Sensing Home has the potential to empower users and to give them a feeling and understanding of what data is collected, processed and analyzed, including which information and knowledge about them and their life can be extracted out of these data. It can be used for finding new and innovative usages of sensors in the smart home as well the critical reflection on these usages. In the context of the smart home, the term privacy has many meanings [7]: control, boundary management, contextual integrity, concern and protective measure. Even if the sensor probe is not concerned with privacy per se, its open nature bears a significant potential to irritate privacy. Security is another significant issue. Every technical system may be hacked, and an IoT enabled research device has the potential for people to exploit its capabilities for malevolent means.

Figure 4 
            Screenshots of sensor data graphs.
Figure 4

Screenshots of sensor data graphs.

To answer this, Sensing Home intentionally leaves out obviously intrusive sensing capabilities such as a camera or a microphone. The SensorTag does not contain a camera and we disabled the microphone in the firmware. We designed our whole system to handle the data under full control of our own, on our own hardware, without relying on third party cloud services. For every deployment and study, we engaged with our participants to explain them the possibilities and potential dangers and disturbances of using Sensing Home. We also instructed participants to use the sensors only on informed consents of others in their home.

3.4 Data Processing and Visualization

Sensing Home consists of several sensor devices that are connected to the Internet via a Raspberry Pi 3 as an edge gateway. The edge gateway runs Linux and Node-Red, a graphical IoT wiring framework based on node.js. We implemented advanced modules for connecting the SensorTags while enabling new functionality and better energy efficiency. The sensor data is transmitted to a server of our research group. We use InfluxDB for data storage on our own server.

In addition, Sensing Home contains a tablet computer that is preconfigured and gives participants full access to the data, both live and historic, from their homes. Data is visualized as simple line graphs in Grafana [10] for all sensors of all sensor devices. This includes temperature, light level, humidity, barometric pressure and movement through accelerometer values. These line graphs are state of the art for most IoT probes and toolkits and are also much in line with those that are used in other studies [8], [17], [23]. Furthermore, these line graphs contain only little pre-processing and are quite near to the “raw data”, which we hope would minimize bias from pre-interpretation. We configured the sampling interval for sensor readings depending on the nature of the sensors. Both to keep battery consumption low and to allow for potentially interesting activities in the home to be recorded. For slowly changing measures (ambient and object temperature, barometric pressure, and relative humidity) the sampling interval is every 10 s, for faster transient measures (light) every 2 s, and externally triggered events (accelero-, gyro- and magnetometer) every 0.1 s for 10 s once triggered by motion.

3.5 Deployment

For deploying Sensing Home in people’s homes, we assembled “sensor packs” around three SensorTags each. We modified and 3D-printed custom housings for the SensorTags to offer a wide variety of mounting options. It is possible to attach a sensor e. g. via glue pad, rubber bands or cable strips to an object, to hang it somewhere or to just put it down. Mounting material is also part of the “sensor pack”. It also comes with a pre-configured WiFi hotspot for instant use without further configuration and to transfer data to a server of our research group for processing. The “sensor pack” also contains a tablet computer for providing live visualizations of sensor data, a booklet and an instant camera to document and perform data work. All parts of Sensing Home are deployed in a box and are ready to use. Having a dozen of probe packs with three SensorTags each requires some kind of (automatic) device and account management which we also implemented.

Figure 5 
            “Sensor pack”.
Figure 5

“Sensor pack”.

In the following two sections of this article we will briefly present examples of the various deployments of Sensing Home for “Exploring Sensor Data in the Home” (see box for section 4) and will continue with more in-depth findings from one deployment of Sensing Home in combination with the method Guess The Data, which is called “Collaboratively Interpreting Sensor Data“ (see section 5).

4 Exploring Sensor Data in the Home

As the goal of Sensing Home is to both inspire and empower, we conducted various participatory field trials for exploring sensors in the home together with participants from diverse user groups. We report on these field trials elsewhere in depth [2] but will illustrate quite briefly the variety of explorations and appropriations of Sensing Home within the home. We do so through the lens of five tangible implementations from the field. Section 5 will then present one study of sensing home that focusses on aspects of empowerment.

The first field trial aimed to explore potentially interesting applications of sensors in the home. Here, computer science students, staff of a computer science department as well as elderly volunteers explored possible scenarios of using Sensing Home within their homes. The second field trial was an interdisciplinary teaching project. Here, computer science and social science students collaborated in order to design and subsequently conduct user studies in different social settings. The third field trial was about sensemaking and empowering people in understanding the potential and pitfalls of sensors in their home. Here, we deployed Sensing Home within nine homes and explored the data together with the inhabitants in order to make sense of sensor data collected in their homes. These three participatory field trials show the versatility of Sensing Home as a generic co-design-tool for exploratory and participatory research on the smart home. It allows for an empirically grounded exploration of IoT in the home and allows for participatory design work with participants of diverging technical literacy and fluency. Participants with some technical expertise came up with highly creative applications for sensors in the home. Participants with only a little technical expertise were significantly empowered in understanding the gains and risks of sensor data on the home. Students in turn were able to independently design and conduct empirical studies in the context of the home.

While all field trials employed the very same Sensing Home design tool, the variety of applications shows the versatility of Sensing Home for participatory research on the smart home. The following examples are – in our eyes – especially noteworthy in regards to how participants explored the capabilities and possibilities of smart sensors in their homes. These examples are paradigmatic as they either inspired us as researchers or allowed participants to construct and try interactive systems, and sometimes both simultaneously. For this presentation we draw inspiration from Bowers [4] notion of annotations. Surely, one can argue, that the – often mundane – placement of sensors in places, on things, or animals may not warrant this attention. Still, annotating the outcomes of participants’ deployments allows us to juxtapose their resemblances and differences, while relating to the mainly unchanged shape and materiality of Sensing Home itself.

By the same token, the generic nature of Sensing Home and our own slightly ambiguous framing of Sensing Home – as both a probe to “share their own meanings” while simultaneously allowing participants to “construct and try interactive systems” [19] – gives us the opportunity to reflect on some of the outcomes of the participatory field trials. We do so, in order to explore the intersections of probes and toolkits through the outcomes of the various deployments.

4.1 Is There Enough Water in the Aquarium?

Figure 6 
            Is there enough water in the aquarium?
Figure 6

Is there enough water in the aquarium?

Mode.

Free exploration.

Description.

This is an example of how Sensing Home inspired one participant to think outside the box to generate an innovative solution through free exploration. The participant wanted to detect the filling level of the aquarium. Sensing Home did not provide a sensor dedicated and intended to directly measure what he was interested in. He tried out how the provided sensors might also work and mounted one on the top lid of an aquarium. The participant then systematically tried and documented the different sensor functions at varying filling levels until finding a satisfying solution. This way he found the intensity of light reflected on the surface of the water as a usable indicator of the filling level.

Benefits.

Here, Sensing Home worked very well in means of supporting creativity and satisfying human curiosity by exploring the possibilities of usage. It enabled this participant with arguably more knowledge and higher skills to ideate, prototype, and evaluate an interesting and complex scenario. This particular scenario is also a good example for the usefulness of Sensing Home’s functional versatility. Even if only one sensor function was used in the end, it was helpful to be able to compare all different sensor functions and to choose the one that fits best.

4.2 What Is on TV Today?

Figure 7 
            What is on TV today?
Figure 7

What is on TV today?

Mode.

Free exploration.

Description.

One participant used Lego to build a small stand to allow for an optimal position of a sensor in front of the TV. There he used Sensing Home to detect when TV is being watched. Given the fact that the TV is the most prominent technology in our living rooms, this use case may have been obvious. Yet, in order to achieve the participant’s goal, sensor technology has to be able to perform adequately, even for proof of concept. Here, in particular, the light sensor had to refresh sufficiently fast. As such, this example also illustrates how participants tested the technical limits of Sensing Home: The participant even went so far to expand this setup to determine whether certain TV shows would cause characteristic light patterns. So, he was interested not only to detect if someone watches TV but to eventually recognize what is being watched. He then compared the data sets of an episode with a replay of the same episode for similarity. We were impressed and inspired by the idea and also the results. Thus, we decided to use a position opposite the TV set as one of three predefined data collection points in later studies as watching TV is a very common domestic activity.

Benefits.

This is perhaps the most complex usage example we encountered, as the participant even verified whether the pre-set sampling interval of the light sensor was fast enough to capture similar data for a second view of the same content. This example did again show the versatility of Sensing Home and the potential as probe for inspiring both the participants and us as researchers.

4.3 Fridge Friction and Kitchen Action?

Figure 8 
            Fridge friction and kitchen action?
Figure 8

Fridge friction and kitchen action?

Mode.

Initially free exploration and later on predefined data collection points to compare data sets.

Description.

For this example of Sensing Home a sensor has been attached to the outside of the fridge door. While this is an arguably less innovative example, it showed us a certain relevance, because it illustrates the transferability between different modes of utilizing Sensing Home. In this example participants started in a free exploration mode. Here, one of the participants experimented with the sensor in the kitchen, especially in and on the fridge to observe, for example, if the fridge door was opened. Later on, we also chose this fixed position as a blueprint for a future study. While it is obvious to employ the motion sensor, the placement of the sensor inadvertently allowed to monitor various other activities in the kitchen at large: light been switched on or off as an indicator of presence, temperature and humidity as indicators of typical kitchen activities. Some of the activities might even further differentiate between various kitchen activities, like cooking vs. roasting vs. baking through correlation of temperature and humidity. By reflecting the initial usage and further adaptation, we realized that both placement and the availability of all the other sensors would allow to compare data accrued from different homes. Therefore, we employed the data collected this way in a workshop format where participants would then be able to compare the data accrued in their homes with data accrued in homes of neighbors in order to learn about the capabilities and dangers of smart sensor data.

Benefits.

Here we see advantage in the probe’s versatility because more than one sensor function are meaningful in such a scenario. Sensing Home also allowed for a change of mode while keeping the technological base unchanged, from a mode of free exploration and inspiration to data collection at predefined data points for comparability and reflection.

4.4 What Is the Cat Doing?

Figure 9 
            What is the cat doing?
Figure 9

What is the cat doing?

Mode.

Free exploration.

Description.

Giving participants the opportunity to explore Sensing Home with an open-ended narrative, they used Sensing Home for their own interests. Participants in several participatory field trails used the toolkit to monitor domestic animals: A cat, a dog, and even a horse were tracked and their movements subsequently analyzed by mounting sensors to their collar or mane or into their stable. This is also an interesting example for the use of environmental measurement not only to indirectly measure activities but also to measure directly on an actor. This not only led to interesting use cases but showed that the Sensing Home provokes deeper engagement with IoT in social contexts.

Benefits.

The sensors of Sensing Home are relatively small, battery driven, and can be used wirelessly. These circumstances allowed participants to conduct their own research projects beyond the limits of fixed mounts inside the home. In particular participants from non-technical backgrounds were curious and capable to use Sensing Home as a technological basis for comparatively complex projects.

4.5 Smart Tomato, Healthy Tomato?

Figure 10 
            Smart tomato, healthy tomato.
Figure 10

Smart tomato, healthy tomato.

Mode.

Free exploration and extension.

Description.

This smart tomato is one example of a monitoring condition where participants tried to find an ideal location for a tomato plant in their apartment correlating light and humidity values in order to monitor the well-being of the plant. Participants not only used the available input sensors, but also added a new sensor to the Raspberry Pi to measure the soil moisture through electric resistance. The participants also designed other outputs and incorporated other types of use, like notifications to remind users to water the tomato. As such, this example is of particular interest, because participants deliberately expanded the capabilities of Sensing Home. What was planned as a probe was then expanded to be a toolkit.

Benefit.

While Sensing Home worked ad-hoc in this example, technically experienced and interested participants could go beyond the scope of the delivered functions with relative ease. Participants could do that, due to the openness of the hardware and software.

4.6 Comparing These Usage Examples

In presenting these usage examples we have outlined how Sensing Home was used by our participants in various participatory field trails. This inspired us researchers for future studies with and future deployments of Sensing Home, while simultaneously allowing participants to construct and try interactive systems. In what immediately follows, we will briefly reflect and compare these usage examples to explicate the overlaps of Sensing Home both as a probe and as a toolkit. Sensing Home was used with different purposes and interests as well as with different audiences in mind. In comparing these examples of how participants used Sensing Home several benefits become apparent.

4.6.1 Changing Purposes

In the TV, the Fridge, and Cat example Sensing Home has been employed as a probe to inspire us researchers. Participants explored sensors in regards to their own interests. They were encouraged to put the probe into places they like to explore through sensors. Interestingly, with both the Fridge and the Cat the decision where to put a sensor was inspirational. With attaching sensors to their pets participants challenged the concept of the home by extending it to the places their pet would wander. By attaching sensors to the fridge, participants unknowingly pointed us to both look at possible surveillance scenarios in the smart kitchen and also beyond the movement sensor that has been initially selected by the participant. While this was inspirational on the level of monitoring through sensors it was a leap in magnitude of the extent of possible surveillance scenarios. This and the TV example both primed us to consider fixed positions within the home in order to make collected sensor data comparable in order to critically reflect privacy implications together with participants. By the same token, Sensing Home has also been a toolkit in both the TV and Tomato example. While this is most apparent with Tomato, where participants extended Sensing Home beyond its initial scope in hardware, also the TV example shows qualities of a toolkit, as the potential access to the sensor data enabled our participant to measure and compare data from various sessions.

4.6.2 Shifting Audiences

Looking at the outcomes from the participatory field trails, all examples are inspiring for designers and developers to learn about possible future smart sensor products. Yet, taking TV and Fridge as examples, both have been inspiring for the participants as well: with TV our participant was able and willing to adjust data collection and comparison towards different outcomes – from knowing if something is watched to discerning what is being watched. With Fridge on the other hand, a seemingly uninspiring case of a door motion was born out of the participants curiosity when the fridge door is opened and then developed to a full-fledged blueprint of kitchen surveillance. Especially the Aquarium and the TV scenario show some research directions that could be worth following up. The Aquarium showed a very interesting appropriation of the light sensor to measure water in the aquarium, while the TV shows a very surprising possible surveillance scenario. Indeed, the TV example did challenge both usability and performance of the sensors. At least in theory, both show innovative usage scenarios for sensors in the home.

4.6.3 Empowering Participants

The TV example was both the most complex and most technically complicated usage example we encountered. Here the participant even verified whether the pre-set sampling interval of the light sensor was fast enough to capture similar data for a second view of the same content. In contrast to this, the Fridge example was both less complex and less technically advanced. Yet, the stationary position of the sensors allowed for a sophisticated scenario, as it enabled participants to gather insights into individual experiences but also allowed them to collectively make sense of relatively simple sensor data that reveals the potential and dangers of making sense of this data by applying situated knowledge. In subsequent workshops we conducted with participants from different households, they eventually realized the surveillance potential of gathering data in the home through their own practice. The active usage and data work of these stationary sensors led to forms of surveillance of other family members. With Cat even participants from non-technical backgrounds were able to use Sensing Home as a technological basis for comparatively complex empirical projects, such as monitoring their pets. Finally Tomato shows that the open nature of Sensing Home is beneficial for customization of such a toolkit / probe, as it allowed participants with more technical skills to add an additional sensor. The TV example also enabled participants with more knowledge and higher skills to ideate, prototype and evaluate interesting and complex scenarios. It worked very well in means of supporting creativity and satisfying human curiosity by exploring the possibilities of usage which goes both beyond the scope of a probe and a toolkit.

5 Collaboratively Interpreting Sensor Data

In section 4 we presented field trials where we deployed sensing home so that participants can explore sensor data in their home. In the following section we will describe one particular deployment of Sensing Home aiming to empower participants. It enabled participants to do their own data work and simultaneously was setup to allow for a subsequent, collaborative group discussion. We refer to the later as “Guess the Data”. This combination of Sensing Home and Guess the Data is special, because it did not focus on free exploration, as in the samples provided in section 4. Instead, it is aimed at empowering participants through use and collaborative interpretation. We will first explain how Sensing Home was set up to enable participants self-performed data work in their home on the generated data. We then describe the subsequent collaborative group discussion format Guess The Data. This section ends with a description of preliminary findings and their discussion.

5.1 Study Design: Setup of Sensing Home & Guess the Data

We deployed Sensing Home in the households of three different groups of participants in two large-sized German cities. We sampled those groups with regard to participants’ age and spatially and socially similar housing situations. All nine participating households used the Sensing Home pack for 10 to 14 days. Thereby we collected about 100 days of accumulated data. Before the study started, we briefed participants on the following: how the “sensor packs” work, which data and sensor data is collected, where data is stored, and how data is processed. Also, that anonymized data sections might be used and processed, that participation is voluntary and that they can end the study whenever they wish without repercussions. A summary of this was given to participants as an informed consent form. We also encouraged the participants to use the sensor packs with the informed consent of those in their homes.

5.2 Sensing Home: Enabling Self-Performed Sensor Exploration

At the start of the study, one of our researchers set up the “sensor packs” in the homes. All participants got a short introduction how to use the sensor probe pack, how to do the data browsing and how to document their work. It was also explained how to use the tablet in order to browse the live sensor data visualization. We gave the participants our contact details and encouraged them to contact us any time they have questions or problems. We ended the home visits with a short pre-structured interview, asking for literacy and fluency with (smart) technology and Internet usage. We visited the homes again for technical troubleshooting as needed. In deviation to other deployments, we selected three positions for the sensors in participants’ households: In the hall on the inside of the main entrance door, in the kitchen on the outside of the fridge door, in the living room opposite to the TV set.

In contrast to Tolmie et al. [23] and Fischer et al. [8] we focused to enable the participants to individually browse and interpret the data already during the deployment. Participants had constant access to real-time data as well as all previously collected data. Participants could monitor all sensor values or select individual sensors and change the temporal granularity of the visualization (from minutes to several days). The complete sensor data was accessible throughout the whole deployment period. This setup enabled participants to continuously conduct their own data work (see section 2). Finally, participants were encouraged to document their data work in order to do self-performed data work. We did so, because we expected that they would read the data, speculate on the cause, and annotate the data [8]. For this to work out, we created different documentation formats. We provided a pre-structured working booklet similar to a scenario diary. Participants were also able to make annotated screenshots of graphs. We encouraged participants to note and send us data sections they found interesting. We did not expect annotated data sets with a quality and granularity as done by trained researchers as in [15]. Rather, we wanted the participants to do their individual data work regularly to develop an own understanding of the data, to become able to reflect on the data from their homes and how their activities are visible in the data. After the data collection phase of the study, the sensor packs were removed from the homes and access to the data ended for all participants. After that we started to prepare Guess The Data.

5.3 Guess the Data: Conducting Collaborative Data Work

Guess the Data is a data driven group discussion format between participants and researchers. It is self-developed and engages participants from different households in reflecting the sensor data and their interpretation within the context of their homes. Guess the Data is based on the graphs of data collected by our participants. In preparation of a Guess the Data session, we browsed the data in a group of researchers and looked for interesting patterns in the data. We used our experience on working with sensor data, as well our everyday knowledge of common activities to interpret the data. Thereby it was inevitable to include knowledge about the context and the situatedness of the data, gained from previous contacts and home visits. We selected 10 to 12 data sections per discussion session. We especially looked for data and graphs that seemed promising as an interesting data episode later on in the discussion. For the collaborative group discussion, we printed data sections in the graph format that participants were already familiar with. We anonymized the graphs by removing all markings that directly hinted at the creatorship of the data. We chose a large paper format to present the prints as some kind of shared working sheets for group discussion.

Figure 11 
            Print out of sensor data sheets.
Figure 11

Print out of sensor data sheets.

About a week after data collection we invited participants from various households for the group discussion. We started the session with relatively open questions on the overall experiences of using the “sensor pack”. While presenting the printed graphs one after another we encouraged the participants to articulate on what they see in it and which everyday experiences they can identify. We expected the participants to speculate on the data, to guess the data. We fostered the evolving discussion with immanent questions when necessary. The performed group discussions took an average of 90 minutes and were recorded and transliterated afterwards. In what follows we will present and subsequently discuss preliminary findings. We aim to fully analyze the collected data by following the analytic principles of grounded theory [21] in a two-step process with open coding and axial coding. The vignettes in the following examples have been translated from German to English by the authors.

Figure 12 
            Guess the Data – Participants in discussion initiated by the collective on the printed data graphs.
Figure 12

Guess the Data – Participants in discussion initiated by the collective on the printed data graphs.

5.4 Making Sense of Sensor Data in the Home

The Guess the Data methodology helped us to understand the process of sensemaking of sensor data in the home. In what follows, we will have a first look at how people make sense of sensor data from their homes, both individually and collaboratively. Throughout both deployment of Sensing Home and the group discussion Guess the Data we could observe a basic triadic mechanism of visualization, realization, and reflection in engagement with sensor data. By gathering data and providing visualizations to participants, topics became aware that have not been prominently visible before. Participants experience this moment of visualization as a surprise, which leads to a discursive reflection. That in turn has the potential for behavior changes, for asymmetrical surveillance scenarios to emerge, and for understanding of surveillance mechanisms.

5.4.1 Visualization and Realization: Making Daily Routines and Deviations Visible

A: OK, this could be the evening, when I arrived at home after visiting my sister, and then i seriously baked a cake, because he [her boyfriend] got on my nerves for weeks: ‘I absolutely want to have a cake. ’Thus I really baked a cake. That could be the Sunday.’

In interpreting data sets, participants started to reason about whose home is shown, which day of the week it could be, and if it was just a ‘normal’ day without any special event. Daily routines are usually not conscious, they “run” in the background and do not need to be scrutinized. As such, the visualization of data helps participants to gain further insights into their daily routines, but more importantly deviations from it. It also leads to further discussions about activities, which have not been seen in the data set.

B: And then here in the evening you switch it off. It looks like you are always going to bed at eleven.

C: Yes, we have to get up early.

In this example participants realize that sensor data from the light sensor can be indexical for their sleep rhythm. Here, the visualization of data can also be used to confirm assumptions about other residents’ behavior.

D: This is a consequent biorhythm.

E: Wait till you see mine. This will probably be funny.

This data might not have the high power of expression about biorhythm as claimed. However, it shows that participants understood that simple data may reveal everyday patterns, especially if combined with contextual knowledge about their lives.

5.4.2 Reflection: Sensor Data as a Feedback Loop

F: I came home from work and the light in the hallway was turned on and my boyfriend sat in his room. […] It is very often like that and now he acknowledged it too […] And then I turned it off. Like I do it always. But he gave it up now.

In this example we see how the visualization of the light sensor turns out to proof a problematic everyday behavior of the male resident. His partner used the graph to show him that he forgot to turn off the light in the corridor. While she was aware that this happens often, he was surprised and realized the implication only after he saw the visualization. This reflection then led to behavior change. However, as soon as such a reflection is based on an asymmetrical information access, sensor usage can also turn into a surveillance instrument. During the group discussion many participants realized that access to seemingly unconscious sensor data can be used to interpret patterns in the home and to infer about people’s behavior. In other words, participants understood that simple sensors in the home can be used for surveillance purposes. Most participants could imagine further scenarios where this might be useful for various purposes. For example, this may become problematic when small sensors are deployed in the home, and not all inhabitants are aware. In this case, someone – another inhabitant, or the manufacturer of a smart device – could have exclusive access to additional data within the home and may use this asymmetrical access to sensor data to their advantage.

6 Lessons Learned and Future Work

With this article we presented both the design rationale of Sensing Home and a variety of its deployments. We have also shown the potential for exploring sensors in the home together with people and ways to empower them to understand issues and opportunities of sensor data from their homes.

In regards to the capability of Sensing Home to explore sensor usage in the home, Sensing Home has a lot of potential to be more user friendly. In its current form, sensor data is shown as graphs on a tablet computer. Future work may need to explore ways of preprocessing the sensor data for easier interpretation. This may for example take the form of text output on the smart phones of participants or speech output through speech assistants in the home. There are also opportunities for Sensing Home to feature different shapes for the sensors and a greater variety of ways to affix sensors to things. We have started to explore this idea with the initial concept of Kindred Surprise.

User friendliness has also some room for improvement when using Sensing Home to empower people in regards to data privacy and issues of power and control. We have identified some issues with the way data can be annotated. Printouts of sensor data, as we have used them for Guess the Data are the most straight forward way to support easy annotation of graphs. If a printer would be part of Sensing Home, participants would be able to directly write on top of collected data. Such a direct access to annotating data work would also allow preprocessed data to take a stand point. For example, if a certain predefined threshold of a sensor reading is reached, the respective graph could be printed out automatically in order for people to take a stance.

And finally, there is more design research necessary to spell out whether Sensing Home is a probe or a toolkit: Since their first proposal by Gaver et al. [9], “Cultural probes” have been adapted very differently in HCI [3], [11]. The main objective of a probe lends itself particularly well to the development of IoT products in the context of domestic environments. The potential of networked sensors and actuators in homes has not yet been thoroughly explored. So far, social practices, constellations, needs and critical aspects such as data sensitivity and privacy have become too rarely methodologically and systematically part of the development and research of IoT in the smart home. Similar to Gaver’s suggestion, we wanted to give Sensing Home users the opportunity to share their own meanings with us as designers and researchers. As such, we lean towards a comparatively original understanding of the concept of a probe. Here, a probe is used for inspiration and needs analysis at an early stage of development. In extending the concept, however, it was also a goal to make sense of sensor data together with users. Sensing Home can also be considered a toolkit, in as much as “toolkits allow people to construct and try interactive systems” [19]. In this regard, Sensing Home can arguably be considered both a probe and a toolkit to explore the design space of gathering and interpreting sensor data in the home. Certainly, there is more design research necessary to spell out the similarities and differences of probes and toolkits in more detail. For now, this article is aimed at shedding some light on the empowering aspects of Sensing Home for users.

About the authors

Arne Berger

Arne was born in the 70ies in the Eastern Bloc. He is an interaction design researcher and post doctoral researcher at Chemnitz University of Technology, Germany, where he explores novel modes of co-creation of smart connected technology in the context of the home.

Albrecht Kurze

Albrecht has a PhD in computer science. His research interests are networking aspects in all flavors and especially the Internet of Things. He develops not only the technology for it but is also interested in the innovation potential and the created implications.

Sören Totzauer

Sören studied informatics. He is interested in bringing together social dynamics and technological possibilities. His current research interests focuses on Ideation Processes and Tools to foster creativity.

Michael Storz

Michael is a human-computer interaction researcher. In his research he studies and develops interactive devices for groups. His current interest is in the participative R&D for groups of older adults.

Kevin Lefeuvre

Kevin is a product designer with a master degree from Bauhaus-University Weimar. He is a tinkerer by profession and a user advocate by heart. At Miteinander he is exploring co-design methods and building co-design tools for designing smart connected fictions, scenarios, and devices.

Andreas Bischof

Andreas studied sociology and cultural sciences. He is interested in the technical mediation of interaction and the technization of social situations. His current projects concern technology for age and ageing, and participative R&D with older adults.

Mira Freiermuth

Mira is a sociologist. The methods of qualitative social research provide her with tools to accompany, understand and reflect on human-computer interaction. For her, the participation of people affected by technology in the process of development is most important; technology from people for people.

References

[1] Airqualityegg. https://airqualityegg.wickeddevice.com/, accessed January 6, 2018.Search in Google Scholar

[2] Berger, Arne, Kevin Lefeuvre, Albrecht Kurze, Michael Storz, Sören Totzauer, and Andreas Bischof. 2018. Sensing Home: Participatory Exploration of Smart Sensors in the Home. In Social Internet of Things, ed. Alessandro Soro, Margot Brereton, and Paul Roe. Internet of Things: Technology, Communications and Computing. Springer.10.1007/978-3-319-94659-7_7Search in Google Scholar

[3] Boehner, Kirsten, Janet Vertesi, Phoebe Sengers, and Paul Dourish. 2007. How HCI Interprets the Probes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’07. 1077–1086. New York, NY, USA: ACM. doi:10.1145/1240624.1240789.Search in Google Scholar

[4] Bowers, John. 2012. The logic of annotated portfolios: communicating the value of ‘research through design’. In Proceedings of the Designing Interactive Systems Conference, 68–77. ACM.10.1145/2317956.2317968Search in Google Scholar

[5] Castelli, Nico, Corinna Ogonowski, Timo Jakobi, Martin Stein, Gunnar Stevens, and Volker Wulf. 2017. What Happened in my Home?: An End-User Development Approach for smart home Data Visualization. In, 853–866. ACM Press. doi:10.1145/3025453.3025485.Search in Google Scholar

[6] Diez, Tomas, and Alex Posada. 2013. The Fab and the Smart City: The Use of Machines and Technology for the City Production by Its Citizens. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, TEI ’13, 447–454. New York, NY, USA: ACM. doi:10.1145/2460625.2460725.Search in Google Scholar

[7] Fischer, Joel E., Andy Crabtree, James A. Colley, Tom Rodden, and Enrico Costanza. 2017. Data Work: How Energy Advisors and Clients Make IoT Data Accountable. Computer Supported Cooperative Work (CSCW) 26: 597–626. doi:10.1007/s10606-017-9293-x.Search in Google Scholar PubMed PubMed Central

[8] Fischer, Joel E., Andy Crabtree, Tom Rodden, James A. Colley, Enrico Costanza, Michael O. Jewell, and Sarvapali D. Ramchurn. 2016. “Just Whack It on Until It Gets Hot”: Working with IoT Data in the Home. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, 5933–5944. New York, NY, USA: ACM. doi:10.1145/2858036.2858518.Search in Google Scholar

[9] Gaver, Bill, Tony Dunne, and Elena Pacenti. 1999. Design: Cultural Probes. Interactions 6: 21–29. doi:10.1145/291224.291235.Search in Google Scholar

[10] Grafana 2018, https://grafana.com, accessed June 1st, 2018.Search in Google Scholar

[11] Graham, Connor, Mark Rouncefield, Martin Gibbs, Frank Vetere, and Keith Cheverst. 2007. How Probes Work. In OZCHI ’07 Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces, 29–37. New York: ACM. doi:10.1145/1324892.1324899.Search in Google Scholar

[12] Haraway, Donna. 1988. Situated Knowledge: The science Question in Feminism and the Privilege of Partial Perspective. Feminist Studies 14: 579–599. doi:10.2307/3178066.Search in Google Scholar

[13] Jakobi, Timo, Corinna Ogonowski, Nico Castelli, Gunnar Stevens, and Volker Wulf. 2017. The Catch(Es) with smart home: Experiences of a Living Lab Field Study. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, 1620–1633. New York, NY, USA: ACM. doi:10.1145/3025453.3025799.Search in Google Scholar

[14] Kahn, Joseph M., Randy H. Katz, and Kristofer S. J. Pister. 1999. Next century challenges: mobile networking for “Smart Dust”. In Proceedings of the 5th annual ACM/IEEE international conference on Mobile computing and networking, 271–278. ACM. doi:10.1145/313451.313558.Search in Google Scholar

[15] Laput, Gierad, Yang Zhang, and Chris Harrison. 2017. Synthetic Sensors: Towards General-Purpose Sensing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, 3986–3999. New York, NY, USA: ACM. doi:10.1145/3025453.3025773.Search in Google Scholar

[16] Lemme, Helmuth. 2017. Energieautarke Sensoren: Einbruchalarm mit Solarstrom. Elektronik: 26–27.Search in Google Scholar

[17] Puussaar, Aare, Adrian K. Clear, and Peter Wright. 2017. Enhancing Personal Informatics Through Social Sensemaking. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, 6936–6942. ACM Press. doi:10.1145/3025453.3025804.Search in Google Scholar

[18] Rader, Emilee, and Janine Slaker. 2017. The Importance of Visibility for Folk Theories of Sensor Data. In Thirteenth Symposium on Usable Privacy and Security ({SOUPS} 2017), 257–270. Santa Clara, CA, USA: {USENIX} Association.Search in Google Scholar

[19] Sanders, Elizabeth B.-N., and Pieter Jan Stappers. 2014. Probes, toolkits and prototypes: three approaches to making in codesigning. CoDesign 10: 5–14. doi:10.1080/15710882.2014.888183.Search in Google Scholar

[20] Schmidt, Kjeld. 1992. Taking CSCW Seriously: Supporting Articulation Work. In Cooperative Work and Coordinative Practices, 45–71. Springer.10.1007/978-1-84800-068-1_3Search in Google Scholar

[21] Strauss, Anselm, and Juliet Corbin. 1998. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 2nd ed. Sage Publications, Inc.Search in Google Scholar

[22] Texas Instruments. Simplelink SensorTag. http://www.ti.com/ww/en/wireless_connectivity/sensortag/gettingStarted.html, accessed August 16, 2017.Search in Google Scholar

[23] Tolmie, Peter, Andy Crabtree, Tom Rodden, James Colley, and Ewa Luger. 2016. “This Has to Be the Cats”: Personal Data Legibility in Networked Sensing Systems. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, CSCW ’16, 491–502. New York, NY, USA: ACM. doi:10.1145/2818048.2819992.Search in Google Scholar

[24] Van Kleek, Max, Ilaria Liccardi, Reuben Binns, Jun Zhao, Daniel J. Weitzner, and Nigel Shadbolt. 2017. Better the Devil You Know: Exposing the Data Sharing Practices of Smartphone Apps. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, 5208–5220. ACM Press. doi:10.1145/3025453.3025556.Search in Google Scholar

Published Online: 2018-08-07
Published in Print: 2018-08-28

© 2018 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 28.3.2024 from https://www.degruyter.com/document/doi/10.1515/icom-2018-0013/html
Scroll to top button