Automating Behavior? An Experimental Living Lab Study on the Effect of Smart Home Systems and Traffic Light Feedback on Heating Energy Consumption

This work was carried out in collaboration between all authors. Author JB performed the statistical analysis and wrote extensive parts of the manuscript. Author MH conducted the interviews and performed the qualitative analysis. Author PS managed the data collection in the households and analysed their energy demand. Author HB managed the literature searches and designed the study. All authors read and approved the final manuscript. ABSTRACT In the face of growing popularity of eco-feedback innovations, recent studies draw attention to the relevance of the human factor for a more effective design of eco-feedback. This paper explores these challenges more deeply by employing a mixed methods approach. We provide in-situ insights from a Living Lab experiment on the effect of smart home systems and traffic light feedback on heating energy consumption in private households. Our results from an interrupted time series analysis of logged data on indoor room temperature, CO 2 concentration and consumption of natural gas show that the interventions do not affect heating as expected, neither for automating behaviour via high-tech smart home systems nor via low-tech traffic light feedback. Smart home systems do not promise a significant reduction of heating energy consumption and a traffic light feedback on indoor air quality does not lead to a reaction of indoor CO 2 concentrations, but may reduce heating energy consumption. Qualitative interviews on heating practices of participants suggests that comfort temperatures, lack of competences and inert heating systems do override expected effects of the feedback interventions. We propose that high-tech smart home systems should carefully consider the handling competences of users. Low-tech feedback products on the other hand should by design stronger address user experience factors like comfort temperatures.


INTRODUCTION
Eco-feedback innovations fostering behavioural change could provide an important and easy accessible source of reducing resource use and greenhouse gas (GHG)-emissions caused by heating [1,2].Studies have shown that the ventilation behaviour may be more relevant for the energy consumption than room temperature and year of construction of the building [3].However, recent investigations have also shown that interventions with the intention of lowering energy consumption in flats/houses often failed to achieve their aim.For example, Buchanan et al. [4,5] demonstrated in their analyses of inhome-displays for feedback on energy consumption that there is limited evidence that providing data on the energy flow alone is sufficient to reduce energy consumption.They propose that triggering user engagement directly should be a part of smart metering systems to reduce energy consumption in households.However, it is shown that smart thermostats also cannot reach their potential in saving energy and result sometimes in increasing energy consumption due to incorrect use [6].The authors hypothesize that the incorrect use will increase with the complexity of smart thermostats.
In this regard, a more user centred research design that considers the human factor more deeply is proposed to create a better understanding of energy related to heating.User centred research is increasingly conducted within LivingLab research addressing problems of user acceptance, incorrect usage and other unintended side effects of human technology interaction.Liedtke et al. [7,8] consider Sustainable Living Lab (SLL) research to be an effective way to create and implement new ecoinnovations in households due to stronger focus on users' behaviour and habits or practices.The core of SLL is the development of sustainable product and service innovations and their testing in experimental designs to provide insights into the usability and adoption of sustainable innovations for industry, public, and academic stakeholders.According to Liedtke et al. [8], three phases of research and development are conducted in SLL: insight research, prototyping and field tests (see Fig. 1).Insight research focuses on the analysis of the status quo of user behaviour.The second phase, prototyping, includes stakeholders in research, especially the end-user and business partners, to develop sustainable product and service innovations.Testing, the final phase, includes an extended investigation of the latest innovations.
Living Lab research is multidisciplinary and thus strongly supports multi method approaches.A mixed method design encompasses qualitative research (e.g. based on interviews, user observation) and quantitative research (e.g. on basis of survey data, technologically supported data logging or life cycle data) for a better understanding of user behaviour.The focus in testing lies on field experiments.Field experiments in Sustainable Living Lab research make use of sensory technologies and gather behavioural information from logged data of, e.g., indoor room temperature or CO 2 concentrations.Participants of field tests are for instance interviewed on their behaviour, practices and intervention experiences in order to gain and provide a better and deeper understanding of experimental results.This paper focuses on an experimental design within the testing phase, due to extensive prior insight research.Findings from comprehensive insight research on heating behaviour do propose that well designed feedback technologies need to address sociopsychological traits of users properly in order to foster behavioural change effectively.Baedeker et al. [10] highlight the influence of social norms and the importance of hospitality concerns for users when it comes to indoor temperatures.In this regard, Buhl et al. [11] highlight the importance of comfort temperatures for users' well being and therefore for their acceptability for changes in heating behaviours.With respect to promising interventions for the sake of more energy efficient heating practices, Bettin and Buhl [12] find that feedback interventions motivating for direct user interaction promise more effective incentives for behavioural change in terms of indoor room temperature than installations that require the user to remain passive like smart home systems.However, their conclusions are based on extensive surveys and are not confirmed in real life situations, yet.Jensen et al. [1] highlight the potential of diffusing feedback devices and behavioural change among leading lifestyles for energy conservation on a larger scale.Buhl and Acosta [13] estimate non trivial rebound effects of behavioural change via turning down the thermostat and propose to consider sociopsychological dimensions of users in order to better understand and mitigate those unintended effects.Deducting from these insights, we hypothesise that innovative feedback designs are promising interventions into behavioural practices in the heat domain depending on how complex the eco-feedback innovation addresses its users.
In light of this, we have conducted experimental research on two low-invest technologies with different complexity that has been developed to improve heating and airing behaviour: a smartmetering system and a so-called 'CO 2 meter'.The tests were based on the above-discussed research on eco-feedback and shall add a more practice-oriented approach to the growing literature on the effects of eco-feedback.We analyse and compare the effects of a more automated, high-level technological driven smart home system, promising a more energy efficient heating in private households, to a more informative, low-level technological feedback driven CO 2 meters, helping to improved the management of indoor air quality and consequently, affecting indoor temperature management indirectly.In this regard, we hypothesise that 1) smart home systems lead to a more energy efficient indoor room temperature management, i.e. lower average indoor room temperatures; and 2) CO 2 meters lead to a better management of indoor air quality, i.e. lower average indoor CO 2 concentrations.The experimental, quantitative study provides test results on the effects of feedback devices on indoor room temperature, CO 2 concentration and gas consumption.
How can heating and ventilation behaviour be conceptualised theoretically, taking a users' perspective?Here we draw on theories of social practices [14] in our qualitative part of the study [cf. 8].This approach has recently gained attention in research on sustainable consumption [14] and also heating behaviour [15,16].Gram-Hansen [15] showed variance in heating energy consumption of families living in similar buildings due to their heating practices.According to Shove et al. [14] social practices as a routine kind of action consist of meanings, materials and competences.For our qualitative part of the experimental study we draw on materials (e.g.heating systems), meanings (e.g.values, norms, attitudes) and competences (for dealing with feedback devices).The insight research already hints at the major relevance of behavioural aspects of heating such as comfort temperature and doings around keeping a warm and comfortable home.Based on that, we discuss 3) to which extend differences in materials, meanings and competences resonate in heating practices and thus interact with the effects of feedback devices on heating practices.The qualitative study on heating practices discusses why and how feedback devices affect heating practices (and vice versa).The combination of quantitative and qualitative analysis is a main advantage of this study.In that way we cannot only observe the importance of materials, meanings and competences for behavioural change, but also their direct implication for environmental data like temperature and air quality as well as the energy consumption.The qualitative study on heating practices provides a deeper understanding of the quantitative results.

METHODS AND MATERIALS
We follow a mixed methods design as recommended in Living Lab research.In the following sections we shortly introduce qualitative methods conducted and, more extensively, the methodological design of the experiment.
A quantitative design was realised through a quasi-experiment without control group but with pre-tests for a small sample.In such case, the interrupted time series analysis (ITSA) offers a quasi-experimental design with high internal validity.ITSA is commonly used to evaluate large-scale interventions (e.g. policy interventions) in which researchers often face a small sample size of n=1 -the country, state, city or units of similar scale.We adapt ITSA for product and service design evaluation, which faces similar small sample restrictions due to extensive prototyping efforts.For instance, in user centred Living Lab research it is often the case that prototypes of products and services are tested within single real life test-beds (i.e.Living Labs).Consequently, Living Lab research is often constricted to effective sample sizes of n=1 or unbalanced experiments resulting in multiple single tests along differing time frames.It is usually not the case that multiple prototypes are implemented in multiple Living Labs resulting in large samples treated at the same time.As we conduct Living Lab research and as such combine qualitative and quantitative insights to evaluate user integrated product and service innovations, we opted for a single group ITSA where every single household is treated as a single group.By design, a single-group ITSA has no comparable control group; rather, the preintervention trend projected into the treatment period serves as the counterfactual.
The regression model follows the following form is the outcome variable measured at each equally-spaced time-point , is the time since the start of the study, is a dummy (indicator) variable representing the intervention (preintervention periods 0, otherwise 1), and is an interaction term.In the case of a single group study, represents the intercept, or starting level of the outcome variable.is the slope of the outcome variable until the introduction of the intervention.is the change in the level of the outcome immediately following the introduction of the intervention (compared to the counterfactual).
is the difference between pre-and postintervention slopes and is an error term.Thus, we look for significant -values in to indicate an immediate treatment effect, and in to indicate a treatment effect over time, or the trend effect [17].
However, the estimated level and slope effects do not tell the average treatment effect (ATE), but may be employed to estimate the ATE accordingly.Wagner et al. [18] suggest expressing the average treatment effect as the ratio of the predicted and the counterfactual value.The counterfactual in time series analysis without control group is the predicted baseline for the whole experimental phase, thus assuming, no intervention took place.

= +
(2) Where is a vector for the all the time points for the whole duration of the experiment along .
The difference between the outcome of the treatment phase and the outcome of the counterfactual is an estimate of the absolute average treatment effect: The ratio between Equation (3) and (2) is the relative average treatment effect.As the essence of this investigation is the reduction of energy consumption for heating, the consumption of natural gas in the homes was captured for each period.These values have to be corrected due to changing weather conditions over time and hence different outdoor temperatures.According to the German directive VDI 3807 and Grinewitschus et al. [22, p. 1096] as well as Lovric and Grinewitschus [23], heating degree days have been utilised as follows: Where are the heating degree days with heating limit temperature of 15°C, ! is the mean indoor air temperature (set to 20°C), " is the mean outdoor temperature on heating days in °C and % is the number of heating days.
In the next step, the heating degree days of the specific period have to be divided by the long term average heating degree days of one year.In this case the mean value of the city Duesseldorf, close to Wuppertal, has been used: Where & ' is the percentage of heating degree days of specific period over long term heating degree days in % .The following equation forecasts the energy consumption for heating for the whole year: Where 5 678 is the energy consumption for heating for the whole year in ;<= " , and 5 ' is the energy consumption for heating for the specific period in &>ℎ.We calculated the characteristic value of energy consumption by: @ 8 = 9 ABC D EC (7) Where @ 8 is the characteristic value of the energy consumption in ;<= F + " and G 98 is the area of indoor heated space in H . Transcribed qualitative interview data was analysed using the method of thematic coding [24].A pre-defined guideline containing the relevant thematic categories is used to mark interview sections in order to draw conclusions about a case.Here, it is important to remember that we were interested in the kind of material, meanings and competences around a warm and comfortable home that resonate from heating practices through interaction with effects from feedback devices.These questions were addressed by 1) comparing what participants named as their usual comfort temperatures with actually measured values and searching for respective doings in the interviews that might explain differences; 2) analysing the interview data using a guideline containing the categories of materials and meanings in order to carve out what users describe as relevant in their heating and ventilation routines; 3) analysing the interview data using the guideline categories of 'previous experiences with smart home or feedback devices' and 'experiences with the intervention' in order to gain more insights into how participants changed routines or not due to competences.
Eight households in Wuppertal (Germany) participated in a Living Lab study in late autumn 2015.They all had in common that they were using natural gas as the main fuel for their heating systems.Despite that the households were a rather heterogeneous group as they and their dwellings were of different size, urban region and age.

Data Logging and Interventions
The households participated in the field experiments for two periods, the baseline period to obtain control values and the test period to obtain treatment values.Each period lasted two weeks minimum.Moreover, data logging continued after removing treatments so that the experimental period lasted overall from 16th November 2015 till 18th of January 2016.
Two different interventions were used: the 'CO 2 meter' and a smart home system.The CO 2 meter uses a traffic light system to indicate air quality (CO 2 concentration) and provides additional information for temperature, CO 2 concentration or humidity on a small display.It is hence a lowlevel technology with a high level of behaviour dependency as the inhabitant is only directed by the appliance to act, but the system does not act by itself.This device primarily helps to improve the indoor air quality as the user easily recognise bad air quality by coloured lights (<1000 ppm green, >1000 ppm and < 1500 ppm orange, >1500 ppm red).Additionally the CO 2 meter can help to regulate the ventilation duration as one can see when the CO 2 level reached a healthy level again.Here lies the potential for energy savings, as it is important that ventilation is long enough to refresh the indoor air but should not be as long as to reduce the heat of the building itself significantly.The smart home system connects thermostats and windows in a system that allows users to control their radiators in a web-interface in addition to the normal regulation of the digital thermostat.In that Interface time profiles and automatic actions (turn off radiator if windows are open) for radiators (combined as rooms when there are several radiators in one room) can be implemented.The smart home system focuses on automating heating and does not provide feedback in order to motivate specific behaviour.Here, a high level of technology is combined with a medium level of behaviour dependency.The potential for saving energy lies in the regular periods of a day where lower temperatures are sufficient, e.g. office and sleeping hours.The researchers installed the smart home systems and the web-interface was explained to the participants but the participants had to set up their time profiles on their own after the researcher left.Participants always had the possibility to contact researchers if they have problems, however no one asked for additional help for their smart home systems.In that way the competences for such technologies affect the outcome.
Four households were equipped with smart home systems and four households were equipped with CO 2 meters (hid5 excepted due to malfunctioning devices).The temperature, humidity and CO 2 concentrations were logged in living rooms every minute using RaspberryPi based logging systems.They were placed on open shelves, distant to windows and outer walls.Households had to have a wireless internet connection and needed to grant access to their network, which could be an obstacle for some.The data logger itself had no display or other feedback that could affect the participants.The data has been transferred to a SQL database and has been processed as comma separated values for the inferential analysis.Twice a day the database sent error reports via e-mail if some devices were malfunctioning.The fuel consumption (in all cases natural gas) was captured at the beginning of the baseline period and at the beginning and end of the test period.Thus the overall gas consumption for both periods were captured.

Qualitative Research
The participants were invited to a telephone interview in the course of the study.The interview comprised (1) standardised survey items on the households' situation and building characteristics, and (2) a qualitative semistructured part on heating and ventilation routines in interaction with the heating system and among the household members.Seven of the eight participating households also completed this interview part.
Standardised interview data was used to compile case descriptions (e.g.type of building and heating system installed) and put measurements of temperature, humidity and CO 2 levels in relation to building characteristics; this information is reflected in Table 1, where the household situation, sex of interview partners, building characteristics, heating system including self-reported amount of gas consumption for the last year (2014) as well as the type of intervention are reported.For the qualitative interview data an interview guideline was developed containing open-ended questions.The semi-structured interview guideline first asked for how participants usually proceed to have a warm and comfortable home.This question was used to evoke narrations about the usual doings at home.Participants were furthermore asked to name their usual comfort temperature at home, which will be interesting to compare to actual measurement results during the experimental design.Interviews were audiorecorded and transcribed word-by-word for analysis.

RESULTS AND DISCUSSION
We present the results of quantitative experimental testing and the qualitative interviews separately.We give a summary of the main findings at the end of each part and give the integrated interpretation of the results of the Living Lab study in the conclusions.

Quantitative Experimental Results
We follow rigorous inferential statistical testing and only present the average treatment effects for significant level and trend effects for the pre as well as post intervention studies in the single households (with I<0.1 at least).We depict the level and trend effects visually and describe the direction of effects.Additionally, we describe removed treatment effects if applicable.

Indoor room temperature
When it comes to changes in temperature, we find significant level and trend effects for 4 out of 8 households.The relative average treatment effects range from -5.9 % for hid25, to +2.1 % for hid29, +2.8 % for hid28 and up to +4,8 % for hid2.Fig. 2 shows the respective scatter plots and locally weighted regression fits of the logged temperature for the pre and post intervention period (and removed treatment period if applicable).In three of four households, we find positive trend effects, suggesting that the negative immediate level effects are cancelled out by behavioural relapse patterns with increasing time.The also significant removed treatment trend in hid29 underlines this finding.Only hid25 shows significant negative treatment effects.As hid25 reacted to the CO 2 meter, the smart home system shows only positive significant treatment effects in the remaining three households.Thus we cannot conclude that the smart home system enhances a more energy efficient indoor room temperature management.When it comes to CO 2 concentrations, only 2 out of 8 households show significant level and trend effects pre and post intervention.One household shows a negative average treatment effect of -18 % (hid28) and one household shows a positive average treatment effect of +13% (hid29).The latter shows a significant negative level effect for the removed treatment measures (see Fig. 3).As both households have been equipped with a smart home system, the CO 2 meter does not perform any significant effect in any household.

Energy consumption
So far, we used indoor room temperature as a proxy for energy consumption.Next, we provide the results on changes in energy consumption by providing the resulting change in the consumption of natural gas for every household (corrected for weather effects).In contrast to the data logging of CO 2 concentration and indoor room temperature that was captured every minute, the gas consumption was only captured before and after the reference and intervention period.Without the high resolution of quasi real time data logging and hence without proper inferential testing, we are limited in drawing conclusions of the intervention on the energy consumption.
Four households show relevant changes in energy consumption for heating after implementing the interventional feedback devices.Three of these four households show a reduction of energy demand, while all of them got a CO 2 meter.The characteristic energy consumption decreases by 12% (hid5), 13% (hid27) and 51% (hid25).The unexpected high reduction of 51% can be explained with an unplanned absence of the participants for several days, which is discussed in the interview section.The energy demand of hid28 more than doubled after receiving a smart home system (see Fig. 4).This can be explained with the help of the temperature graph in Fig. 2. In the base period one can observe a clear day-night cycle with higher temperatures at day times and lower temperatures at night.This cycle is somehow blocked by the smart home system, which leads to a rather constant high temperature without a decrease at night.The reasons will be explained in the interview section.

Fig. 3. Single-group interrupted time series analysis of CO 2 change for households with significant trend and level effects
Note: ITSA with higher-order autoregressive models and Newey-West standard errors

Summary of main quantitative findings
First and foremost, the field-testing reveals significant effects of feedback devices on heating and airing behaviour as expected.However, the results reveal directions of effects that are different than hypothesised.Our test shows that the CO 2 meter as a low-level technological intervention may exhibit significant effects of indoor room temperature, but not for CO 2 concentrations.Thus, we do not give evidence in favour of hypothesis 2).The CO 2 meter did not intervene into heating practices in favour of lower CO 2 concentrations in living rooms.
The smart home system as a high-level technological intervention may exhibit both, significant changes in indoor room temperature and CO 2 concentrations.However, we refuse to accept hypothesis 1) as well, as we only identify positive treatment effects when it comes to indoor room temperature.The smart home systems did not intervene into practices in favour of lower indoor room temperatures.Interestingly enough, the smart home system does exhibit significant changes in CO 2 concentrations, but in both ways, leaving the effect of smart home systems on indoor air quality inconclusive again.
As we identify positive post intervention trends and removed treatment effects, the level effects might be confounding with potential reactivity to the test setting.The installation procedure and knowing to be part of an experiment considering heating and airing behaviour might have had an effect on the immediate level effects of respondents, despite the fact that households did not face any other treatment than the installed devices.In this sense the study is rather exploratory than confirmatory.
The measured changes in consumption of natural gas for heating do mirror the unanticipated findings from the time series analysis of logged data of indoor room temperature.Given our findings, indoor room temperature is a robust proxy for measuring energetic changes.Most relevantly, the smart home system is associated to severe increase in energy consumption (over 100% for hid28) and the CO 2 meter is associated to a relevant decrease in energy consumption.The results give rise to the question whether CO 2 meters may intervene into heating practices in favour of more energy and resource efficient indoor room temperatures, i.e. lower average indoor room temperatures and smaller temperature spikes during ventilation.This may be the case because more regular but shorter ventilation is associated with a more energy efficient indoor temperature management.

Qualitative Interviews on Heating Practices
Complementing the experimental testing and based on its results, we selected three cases that seemed most interesting for analysing heating practices more thoroughly in a qualitative study.These selections were made due to unusual circumstances in houses and/or specific practices that residents described in terms of their materials, meanings and competences.These are two cases in which an underfloor heating system is installed and for which both interview partners described specific practices that were necessary in using the system to their comfort (hid1 and hid25).The case hid27 was selected due to the rather poor standard of the building envelope and problems with the older heating system described by the interview partner.Hid25 was also chosen in order to analyse effects of the CO 2 meter on heating practices.The following analysis of heating practices is therefore focused on these three cases.Results from interview data could be used to resolve contradictions that appeared when looking at quantitative data alone.

Comfort Temperatures
As part of the interview participants were asked to name their usual comfort temperatures at home 3 .It showed that some participants gave reasons for different temperature levels and often differentiated between rooms, stating that bedrooms were not or minimally heated, while higher room temperatures were maintained in the other rooms (mostly the living room).Furthermore, some of the male interview partners expressed gender-based differences that their female partners would usually prefer higher room temperatures, leading to a kind of conflict about heating routines.
Despite these limitations, Table 2 summarises the comfort levels named by interview partners.
Measurements show some relevant deviations between named preferences and actual temperature levels (e.g.hid1, hid2, hid22, hid25).In most cases the actual room temperature was higher, pointing to a gap which allows for different interpretations: there might be a gap between what participants assume to be their usual comfort levels, it could reflect an interview effect to state what is generally considered a societal "standard" of room temperature (as e.g.reflected in the media) or some kind of problem in set up or control of the heating system.A closer look at heating routines might resolve some of these aspects.

Materials
For hid1 and hid25 it is first obvious that the underfloor heating system appears to have a high impact on heating routines.Both interview 3 For this part the question in the interview guideline was: "which temperature do you usually prefer as your comfort temperature?"Following the qualitative paradigm of research and the principle of 'openness' the question was sometimes adjusted ad hoc according to how the interview proceeded beforehand and no definition of "comfort temperature" was applied to let room for subjective interpretations of participants.
partners described this system as rather inflexible to make short-term adjustments because they experienced long reaction times when the outdoor temperature changes or inhabitants are absent for some time.The inflexibility is reflected in the following statement by hid25, but was also expressed by hid1 in a similar way: This is because an underfloor heating cannot be regulated at short-notice.You have to do that in advance, you comply roughly with the weather forecast and after that you regulate the heating….But this must have a flow of one and half to two days ahead.
This know-how was acquired through trial and error and people seemed reluctant to change the now functioning routines, e.g. they feared that turning down the heating too much for warmer periods might result in a cold home for several days when the outdoor temperature falls again.In hid1 it was also stated that the family is uncomfortable turning the heating down when outside temperatures are unusually high for a winter but it is uncertain when they might fall again -interestingly the male interview partner associated this with interaction with his wife.An underfloor heating therefore is also a good example of interlinkages between material and competences in heating practices.Participants described that a good portion of practical knowhow was required to regulate the heating.It can be expected that due to this inflexible heating system rather constant temperature levels should be visible.Measurements actually can be said to show just this.

[...] For example now in
An Exception is hid25 with a drop of room temperature during the intervention phase for about one week that could not be explained without looking at doings that the interview partner describes here: Yes, then it comes/ well it works ALWAYS.When I/ as I said, if I Know, now days I have looked, I haven't been HERE for one week, it had turned off/ I had it on 15 degrees in every room and now I have seen: Oops, that was too COLD, and it is going to be colder and then I have tuned it on 18 degrees and then I am going to have a room temperature of 20 degrees.
While the temperature was set to 15°C, actual measurement shows that it did not drop below 20°C during that time.This again points towards difficulties of steering the heating system.
For the two households with underfloor heating a rather high difference between what was named as the comfort temperature and the actually measured mean temperature appeared.In any case the indoor room temperature significantly differs from the named comfort temperature (see Table 2: named comfort temp.: 20°C; measured mean temp.: 25.6°C).
Hid27 also shows the impact of the building envelope (the house was built around 1900) on everyday doings.The interview partner described technical problems with the rather old singlestorey heating system, which were only recently fixed, poorly insulated windows, which cause a feeling of cold draught, combined with living on the ground floor without an insulated ceiling of the cellar.Single-storey heating systems mostly offer a time-based steering of the heating (hid2, 27, 28).Here, heating practices show that often inhabitants manually change the programme due to individual comfort.

Well, about one degree. Sometimes we correct it about one degree. You can save the regulation if it should change
temperature permanently -we don't do that but we regulate it separate per day about a half to one degree, if we need to.
The automated time-based steering is furthermore manually regulated because in hid27 the interview partner's girlfriend works alternating shifts and therefore sometimes is at home during the day.In this case the heating is manually controlled to heat also during daytime, when it is usually turned down.These manual interventions also seem to be well reflected in the temperature time series where the temperature curve stays above the mean level for several days and does not show the otherwise usual falls towards 18°C (see Fig. 5).
These conditions caused by materials apparently lead to a very conscious handling of the heating system and thus awareness of practices.Looking at named comfort temperatures and actual measurements, hid27 shows that both values match fairly well.Taken together, the interview quotations and the measurement results both point towards a higher awareness and reflexive heating practices (what is seen as personal comfort, is reflected as known restraints from the building envelope and living conditions and the actually measured temperature levels fit together very well), which seem to be caused by the rather poor standard of the building envelope and, as the interview partner also stated, by costs sensitivity.This points towards the element of meanings as an important factor in heating practices.

Meanings
Meanings associated to heating routines were rated qualitatively regarding how important the dimensions costs, comfort and environmental concern are for the interview partners.Results Interview partners associated their underfloor heating system (hid1, hid25) with ambiguous meanings, i.e. it provides a very comfortable kind of warmth but is also inflexible and participants wished for some kind of automation reacting to outdoor temperatures.Therefore, comfort is highly rated in both cases; costs seem to play a minor role especially for hid25.

Table 2. Comparison between named comfort temperatures and mean temperatures (including standard deviations (SD)) measured in the living
In hid27 costs are foregrounded and together with being aware of problems associated with the building envelope the interview partner is willing to accept some loss of comfort as reflected in this statement: Well in this case it's about the costs, because we know that we live in an old building, we know that we live on the ground floor and that is not going to change for us, I have lived my whole live on the ground floor, we have a cat and it has to walk in and out.And we have agreed/ or because we know we have to accept the constraints.[...] Hence we are aware of what we have and that we have to deal with constraints, it was very important to us to come down from that €110.
For hid1 the highest meaning connected to environment was found as this household stated to be willing to invest in future heating technologies (e.g.combined heat and power unit) for reasons of sustainability even if it would not pay off economically.The interview partner reflected on resource use of heating with natural gas and put it into perspective to his own personal interest in sustainable agriculture/food.

Competences
In this section we concentrate on experiences of users in interaction with materials specifically concerning the interventions and how they incorporated them into heating and ventilation practices.To start with, none of the households reported here had relevant prior experiences with smart home-systems or other feedback devices.
We zoom in into results on the CO 2 meter in two households (hid25 and hid27) in order to exemplify some of the implications.The interview partner in hid25 stated: […] and because of that I am very/(coughs) very happy about this meter, this CO  These kinds of long ventilation periods can partly be shown in the corresponding CO 2 graph (see Fig. 6) In the pre-intervention phase some peaks of CO 2 concentration above 1000 ppm can The corresponding temperature graph also shows drops of temperature in these phases from the usual ca.24°C to around 20°C.Especially in the early intervention phase, peaks are much lower and ventilation periods also appear to be shorter.Drops of room temperature also appear to be less frequent, although still present.
In the case of hid27 especially in the early intervention phase ventilation was much more frequent and CO 2 concentration is much less.This behaviour is reflected in the following statement: This aspect is very well reflected in Fig. 2 where in the intervention the typical drops of the preintervention phase are suddenly absent, leading to a generally higher and very constant indoor temperature This clearly is an effect of interaction with the new device which was not yet incorporated into usual routines and led the participant forget about the heating was still on.Furthermore, the interview partner described how he had turned down the heating remotely a couple of time, but as the temperature curve shows, apparently to a higher level compared to how the nocturnal drop was set up before the intervention.

Summary of main qualitative findings
What can be learned from results regarding the third research assumption on materials, meanings and competences intervening in the effect of feedback and smart home interventions?First, we could identify differences between stated comfort temperatures and actually measured indoor temperatures for most participants.Measurement showed higher room temperatures as what was stated as by participants as their comfort temperatures.

Fig. 6. Single-group interrupted time series analysis of CO 2 change for hid25
Note: ITSA with higher-order autoregressive models and Newey-West standard errors This might indicate that interview partners stated what is considered a "normal" room temperature as social norm (meaning of comfort), while actual comfort levels are higher.In the case of hid27, where the stated comfort temperature matched measurement very well, it also turned out that inhabitants are very conscious of their heating routines.This is apparently due to a rather poor standard of the building envelope and heating system (materials) as well as -linked to thishigher cost sensitivity (meaning of costs).Second, materials showed to influence heating practices (e.g.underfloor heating as very inflexible; building envelope/insulation). Importantly though, not the materials themselves are responsible for heating routines but how they interact with meanings and competences (e.g.practical and implicit know-how of underfloor heating; being aware of flaws of living on the ground floor of an old building but accepting them for other reasons; foregrounding costs or comfort).Third, environmental concerns were sometimes explicitly addressed and in one case the interviewee was willing to invest in new heating systems even though they might not pay off economically.However giving up comfort levels is not an option for most interviewed households.When, fourth, it comes to competences, the interviews revealed that the CO 2 meter as low-tech intervention was anticipated by participants and was also described as a fun thing in handling and in interaction between household members.The high-tech smart home system in one case (hid28) showed to be sensitive to user interaction.Here, a missing set up of a heating profile resulted in a higher and very constant temperature level.Together these aspects seemed to influence how aware people are of maintaining a certain room temperature and thus energy consumption.
Finally, some contradictions of a solely quantitative analysis could also be resolved (i.e. the one week drop of room temperature could be explained to result from simple absence in hid25).For future research, we hypothesise that meanings, most and foremost comfort as well as user competences highly affect the impact of feedback and smart home devices on heating energy consumption.

CONCLUSION
The results from rigorous experimental testing do not support our hypothesis that a CO 2 meter changes the room's CO 2 concentrations significantly.The same accounts for the smart home intervention.We find only positive significant changes in terms of indoor room temperature (increasing temperature).Thus our results do not support the hypothesis that a smart home system leads to more energy efficient management of indoor room temperature.The identified changes in energy consumption in terms of natural gas do underline this finding.Those overall results are in line with previous research on this matter.
At the same time, the CO 2 meters do affect indoor room temperature while the smart home system affects CO 2 concentrations significantly in both directions -positively as well as negatively.Again, the changes in consumption of natural gas for heating do underline the finding, that a CO 2 meter may as well affect energy consumption (lowering indoor room temperature and consumption of natural gas, i.e. negatively).Therefore, we do support the findings from previous studies, that eco-feedback affects behaviour significantly, but remain rather inconclusive when it comes to specifying the direction of the effects, positive or negative.
Our qualitative interviews revealed that ambiguous effects might result from various reasons stemming from materials, meanings and competences, i.e. the social practices, of users.We identified two main reasons for this with respect to heating practices.First, the smart home system calls for enhancing the competences of users in order to set up the systems properly and to let it help users to manage their energy consumption more energy efficiently.Second, we found that comfort may trump energy and resource efficiency concerns leading to rather inert behavioural changes (if at all).Positive trend effects after intervening in heating practices may support the qualitative findings and hint at inert comfort temperatures that users desire to meet, rather than energy efficiency concerns.In this respect, our results support previous findings on the issues and problems of changing heating practices in favour of more sustainable behavioural pathways with technological and economic implications that both have political implications.
Technologywise our research has shown that high-level technological interventions like smart home systems need to address problems linked to human-technology interaction much more directly and efficiently.A more deliberate user integrated design of high-level technology driven feedback may overcome shortcomings in terms of technology handling as well as the probable lack of competences of users.In addition, highas well as low-level technology interventions on heating behaviour need to directly address meanings of users that address user experience factors like comfort concerns, rather than energy or resource efficiency, which represent outcomes, in order to support more sustainable pathways of behavioural change by design.For future political measures addressing energy efficiency -specifically but not solely -for housing this implies to not only subsidise technological development, as this may be unlikely to bring potential savings to their full effect or may even cause more consumption through externalities or rebounds produced by misapplication.On the other side the role of behaviour may require more attention from the side of research and development and from the side of policy aiming to reduce energy consumption.
Economic interpretations can be drawn for households/consumers/homeowners and for technology producers on the micro-level and for innovation policy on the level of economic policy.For the former group our research illustrates the potential of achieving savings with relatively little investment when willingness for behavioural change exists.Conversely it is shown that technology investments into smart home applications are not per se apt at realising a savings potential.This is specifically beneficial for low-income households as our findings indicate that for groups where energy poverty can be an issue, savings can be realised without larger scale technological investments.With regard to producers of smart home systems and other energy management equipment in buildings or households our findings imply to take application and behavioural aspects as potentially interfering variables into concern.I.e.such technologies should be developed taking these factors into concern ideally in interactive co-creation settings such as LivingLabs.This way more ergonomic, and thus more effective solutions can be expected.Consequently innovation policy should strive to foster and support interactive R&D methods, so as to avoid misspending on non-effective technologies.
With regard to future research using an experimental design with relatively high effort per testing revealed important insights through time series and in-depth interviews.This mixed methods research and user centred research needs to work with small samples in order to offer thorough, qualitative insights in user behaviour for more effective product and service design, including the benefits of co-creating prototypes.However, further research should also realise test settings that allow conducting research in relation to a control group and larger samples in order to provide more representative conclusions.Additionally, larger groups would minimise the effects of corrupted data sets due to malfunctioned devices (hid5) or unplanned absences (hid25) on the overall results.As well, the ITSA could then be modelled more complexly by introducing additional covariates which control for and elicit confounding, seasonal or external effects, such as day and night-shifts or more individual specific variables like work-leisure rhythms.Another important addition would be research on the rebound effects of using energy innovations in households.

Fig. 4 .
Fig. 4. Change of energy consumption per square metre and year for households with CO 2 meters (left) and smart home systems (right) longer periods of values around 500 ppm, pointing to the long ventilation cycles.

Single-group interrupted time series analysis of temperature change for households with significant trend and level effects
Note: ITSA with higher-order autoregressive models and Newey-West standard errors

Table 3 . Qualitative evaluation of categories of meanings in households (from + less important to ++++++ more important)
This manuscript was presented at the 4th European Conference on Behaviour and Energy Efficiency -BEHAVE 2016 in Session E1.Behaviour change interventions".., Lockton, D. (Hrsg.).Living Labs -Design and Assessment of Sustainable Living.Springer, Cham Switzerland.2016;343-371. O