Sense Smart Home Energy Tracking

Home Energy Monitoring with AI and Machine Learning

Foreword by Ian Thompson, Editor

I’m the type of person who doesn’t hesitate to reach into my pocket when it’s time to buy a round at the bar. My business ideas? I share them freely, even when I probably shouldn’t. Some might call me generous. But one thing I can’t stomach is wastage.

I often find myself in a cat-and-mouse chase with my kids, switching off lights they’ve left on. It’s not that the cost is astronomical — certainly less than covering for that friend who always manages to vanish when it’s their turn to buy a round. Rather, it’s the principle of the thing. I can’t stand the idea of padding the pockets of big corporations with more money than necessary. I feel the same way about council rates when the services they’re supposed to cover are nowhere to be seen.

This mindset permeates my work. When I design buildings, I’m always zoning in on sustainability and cost-efficiency aspects that most architects and designers bypass. Designing to some passive house standards is a good start, but it’s not enough if the homeowner is going to install an inefficient spa, fridge or water heater. Such poor consumer choices can negate all the thermal efficiency savings.

That’s why I’m an advocate for simple energy monitoring to keep tabs on energy usage and spot inefficiencies. But when I ask clients about their energy usage, the answer is nearly always a noncommittal, “it’s good”. In reality, they have no clue.

That’s what this video interview is about. We’re going to delve into how machine learning can assist us in monitoring our energy use, alerting us when something isn’t working optimally, and guiding us to make better decisions.

However, one unsettling truth prevails: a common trait among us is that we often prefer living in blissful ignorance until an undesirable event forces us to act. Why do we ignore the health of our buildings until it’s too late? What’s the psychology behind that?

Join us as we unpack these questions and more, using energy monitoring technology to make our homes more energy-efficient and sustainable.

Once again, over to the one and only Matt Ferrell.

Unleashing the Power of Sense: Energy Monitoring with Machine Learning

Video Transcript

Dependence on electricity has become a way of life – laptops, mobile phones, air conditioners, stoves, televisions, electric vehicles. You get the idea…our daily lives are using an ever-increasing amount of electricity. But we tend to not think about how much energy each of these things uses, at least until we get our electric bill.

Out of sight, out of mind. But what if you could tap into the power of machine learning to help you monitor your energy use, to find out where it’s coming from and where it’s going, to potentially let you know if one of these devices is failing and in need of being replaced? Would you be willing to put that artificial intelligence into your home?

I’m Matt Farrell, welcome to Diydad [Music]. When you get a car that requires charging, whether it’s a plug-in hybrid or a full-blown EV, it always seems to send you down a path of wanting to know exactly where your energy use is coming from and how much you’re spending. When I got my first plug-in hybrid car about six years ago, I started down this path for myself.

And it only got more intense when I added solar and a Tesla Model 3 to my home about two years ago. I installed a Sense whole home energy monitor, which not only helps me track how much energy my solar panels are generating but also how much energy I’m using and provides a picture of what devices are using that energy.

There are a bunch of home energy monitors on the market that you can use, but Sense caught my attention because of the machine learning aspect of the product. I had a chance to sit down and talk with Mike Phillips, the CEO of Sense, about how the sensor system works and how it came about.

“What is it that got you to start the company? Like what was your motivation behind it?”

“You know, it started from this broad notion of people want to be able to save money by saving energy in the house, but they just don’t have visibility into where it’s going. So that’s what we set up. There’s other systems like this out there, but the thing that drew me to Sense was the simplicity of the system. Because most these systems I was looking at were like, you have to put all these connections on every single line on your electrical box. But here was Sense: just put two clamps around your mains and the machine learning will take care of the rest.”

“Machine learning seemed to be the secret sauce of Sense. Was that your basic approach that kind of pulled this all together?”

“Yeah, I mean, so I also started by putting on these systems in my house that had the clamps on every single circuit and it just was too cumbersome. And even then, knowing power by circuit wasn’t really what I wanted. I wanted to know what the different devices were doing. So we started wondering, well, if we measure the power in a detailed enough way, can we figure it out just from the power signals?

I mean, we’re making use of the fact that the different things in your home use power slightly differently. So if we measure well enough, can you tell the difference? Sure enough, it’s not a new idea, it’s called load disaggregation, and there’d been tons of research done on this. But, you know, what we found was it just was much harder than most people thought it was. Things kind of worked in the lab but not in the real world, which is kind of how speech recognition thirty years ago, you know, if you remember what that was like…systems that would work in a lab and not the real world…and this is just as hard.”

“And we realized, okay, first of all, we gotta get really ambitious about the signals that we collect, so we have very high resolution data. And then, just a machine learning problem on top is a tough, tough problem, but we’ve been deep into it.”

Continue Reading

“I’m not an expert, so my understanding is kind of a base level. But I know there’s basically two approaches you can take. There’s the – I think you even described it on your website – is there’s the supervised and the unsupervised paths towards teaching something. Supervised being showing a machine, like, here’s a picture of a dog, a picture of a cat, and then you show them a whole bunch of pictures and you tell them when they got it right or wrong. Versus unsupervised, where they’re figuring it out kind of for themselves. Is that approach for unsupervised a little bit more challenging approach but with more upside? I’m curious like why one approach over the other.”

“Yes, a great point. And look, it’s the case that supervised machine learning is much easier. So if you can get a nice lingual data that tells you this is a refrigerator and this is a toaster and whatnot, that right great. The problem is we can’t do that here. Like in speech recognition or image processing, like speech recognition you can play audio to people, they can listen to it and say oh the person said this or you can show people pictures of cats and dogs, you know that was a cat another dog.

Here, you just for the most part you can’t do that because um, you show people electrical signals that they don’t even know, and you know a lot of our users want to help and it’s great to get their help, but they, they don’t know when the contents and St. pump of their furnaces running, they’re just they’re just not the so-called ground truth that we can do. So we, we had to go down the unsupervised learning approach.

“As the end user, like when it detects a device in my home and it says ‘we think it’s a washing machine’ and then it says ‘but there’s it could be one of these other things,’ I’m supposed to pick which one it is. Is that the training of the system or is there more to it than, as there’s something that you guys are doing that Sense to really kind of Shepherd it along?”

“Yeah, I mean there’s a lot that goes on behind the scenes and and that one particularly, if we find something that we believe to be unique and we don’t know what it is, we can say ‘we found something, it looks like maybe a motor’ and then it could, you know, 30% of people call this a blender, 20% of people call it a coffee grinder, you go ‘Oh it must be my coffee grinder,’ you type in ‘coffee grinder,’

it’s good it says that nerd in your app and then that feeds back to us. So that, that is super helpful. But some of the other thing behind the scenes that comes before that is we can’t even show you something in the app until we believe it to be unique, right?

Because it right, we just did this with your we found your hallway light, like a 60 watt incandescent in your hallway and said ‘great we founded it,’ but if you also happen to have a 60 watt light in your in your bathroom, we would get them confused. So we have this really hard part of what we do, which is a uniqueness test. So say, hey, have we found something and we’re modeling correctly and do we believe it’s unique? We have to do all that first before we say, ‘Oh we found something, help us know what it is.'”

“Why do I sometimes see duplicates that show up, like it will detect my refrigerator and then six months later it detects my refrigerator? Yeah, I’ve had that happen on occasion. I think I have interesting as to why that happens on occasion, but it’s a – I was curious if you had any insight as to why they might happen.”

“By the way, one thing I didn’t mention on the machine learning side that I think is important to point out, though the in addition it having to be unsupervised, in addition to having this uniqueness test that we have to do and why new things in real time, you got to realize that signals are all on top of each other. So it’s kind of like doing speech recognition with 30 people all talking at the same time, not just one. More layer of why this is hard.”

“Now back to your question about why do you sometimes get duplicates, you know? I mentioned that we we have this unique FAFSA test to decide is this unique thing? Well, sometimes we get that wrong, right? And sometimes we think that that what we see now is a different device than we saw before. This is actually quite active internally right now. We’re working on that. We notice a problem so you should see better performance around that here coming up fruit soon.”

“They’re actually kind of leads right into one of the things I was really excited to talk to you about, which was a few months ago I read and one of your blog posts on your website about the challenges of detecting EVs. As an EV owner, I am super interested in knowing how much energy is going into my car, how much it’s costing me. I was curious if you could kind of like expand on on what you just mentioned as well as that, as to the challenges around actually identifying devices, especially things complicated like an EV.”

“Yeah, so first of all, we are super excited about EVs also, both in terms of to be able to detect them and also be able to start to do things like load shifting. We can talk about that later but it starts with be able to detect them. And look everyone thinks and we do too. Like they should be super obvious. There’s these great big things in your home.

We actually made life a bit harder for ourselves but for a good reason and that is we wanted to be able to show you things real-time in the app, right. There’s a big benefit in the app to be able to turn on your microwave, seen it turn on right now and go up. That’s my microwave saying the coffee grinder and so on. So we put a lot of work into making the app real time and a lot of what we do is focus on how do we make that real-time.

The reason that makes things difficult now for EVs is when they turn on they actually don’t turn on. You know in a big clear way right away. They’re very complicated devices inside, actually inside the car is where the charger is and they start by they do a first little test to make sure everything’s connected and then they ramp up in a little slower way and then they they ramp up to some intermediate things.

So there’s a this kind of complicated dynamic of how th sethings ramp up and down when they start and even harder for us is, as the battery starts to get full they start to gradually ramp down their charge rates. So I’m sorry for all the excuses here but what it means is trying to show, what your Eevee is doing in real time is a tough tough problem for us. We’re giving up and we have increasingly be able to do these things but still not perfectly but it’s a major focus of of our team to better and better do EVs over time.

“Where is it happening? Where as the model actually live? Is it on the device in my home or is it actually being processed out on a server?”

“Yeah, great question. And at least that one has an easy answer. A lot of your questions that have tough answers, that one’s an easy one. What we call the ‘runtime modeling’, so the thing that all day long is saying ‘did my toaster just turn on?’ – that’s happening in the device because we don’t want to have to wait to, and we don’t want to send it up data up to the server for doing that.

But the thing that says ‘do you have a toaster?’ and ‘what does your toaster look like?’ Because it turns out to do this well enough, we have to not just have a generic model of toasters, which we do, but we also have to know kind of specifically what your toaster looks like in order to give that real time, ‘We see right now your toasters turning on’, we need a specific model for your toaster in your house.

“All right, that model learning happens server-side. So what that means is we send enough data up to both drive the application but also to then learn what your toaster looks like. And then we push those models back to the device to say ‘oh hey keep looking for that toaster.'”

“One of the things like I mentioned the beginning is my motivation for it was to understand how, where my energy was going. I’m curious if what you see, the average Sense user is saving on energy based on what they’re doing. If that makes sense…”

“Look, we think even in energy savings there’s a lot that we can do over time automatically to really point out where your potential savings are. And that was our first notion that we would just identify for you where the savings opportunity and automatically tell you. But the first thing we found was that most of the savings are not well defined categories like, you know, like utilities think it’s around lightbulbs and refrigerators.

Well that’s kind of done. I mean they’re all pretty good now. And meanwhile there’s this big chunk that we just call ‘energy hogs’, right? Which is random stuff that if you knew about it, you go ‘oh I don’t need to do that, I can fix that thing.’ And it’s a really broad range. It’s like it ranges from you’re leaving the roof coils on that melt your ice dams and you leave them all in all summer. Or you, you didn’t realize that dehumidifier uses a third of your power. Or there’s something wrong with the setting of your heat pump system or you know, the list just goes on and on.

And because it’s so broad, we realize ‘well we can’t automate all that, so let’s start by giving consumers just visibility.’ And then hopefully they’ll be like you, in actively use the app and the functionality go track these things down. So that’s been kind of focus just based on the conditions so broad. And what we’re finding is that if you’re really active doing that, about you can get 15% or so savings in most films. But only about half the people do that, right? So they, it’s not across everyone, it’s like half that.

“I like your your examples because that actually happened to me. I had a dehumidifier that was on a smart system in my garage and something had happened to the smart outlet and it wasn’t doing what was supposed to be doing. The dehumidifier, I was running 24 hours a day. And I noticed in Sense that there was this huge uptic in energy use and it helped me track down exactly what it happened.”

“Yeah, you know, it really is that, that if you have no visibility – look, if water was running in your basement you’d know, right? There’s, yeah it’d be collecting, you see? If if electricity is leaking and they don’t leak in the same way there’s a puddle on the floor in the basement but it leaks in these other ways, you have no, no way to know. You just have no visibility. So so just giving people this kind of direct real-time visibility, ‘Burt, oh what’s this?’ they get savings.”

“I don’t think you’ll necessarily be able to answer this, but it’s because it’s probably future plans. But have you guys considered deeper integrations into other devices? We’re talk about load balancing, so let’s say my solar system is over producing electricity, it can automatically turn on my wall charger, the wall connector, to start my car charging. Have you guys considered something like that?”

“Yeah, absolutely. And we only have a few things that are alive today that don’t go as far as what you just said, but we’ve already done integrations with ‘if this than that’, we’ve done integrations with TP-Link plugs, with Wemo plugs.

And so we’re already using those where you can, we use that to get both a direct measurement and what’s going on the application. And we are able to, in the app you can control it. So you, if you see in the app, in the Sense app that you left your your garage lights on, and if it’s on the Wemo plug, you can click there turn off. But the next step is starting to automate these things very much is where where we’re heading.

“So so in fact your case is exactly, so my house I have solar panels, I have EVs, and I have a very early kind of rigged up version of Sense where it’s actually controlling my EVs to only charge when solar is overproducing. If not, it goes and finds the marginal carbon intensity. And we haven’t to productize that yet just due to priorities. So but that’s, that kind of thing is very much what we’re excited about, we’re experimenting with in absolutely as the future.”

“It’s one of the reasons I was excited about the Sense when I got it, was it felt like something that will definitely grow and evolve long term. So I felt like I was investing a little bit in the future of the product when I bought it.

And you know, a lot of these things that we want to be able to do, like is talking about, we couldn’t really do until we start to get data. And you couldn’t really start to get data until we got these things out there. So there is this virtuous loop now where we got, you know, ton of data flowing and kind of engaging consumers like themselves. And and just more to be done. And now we’re kind of limited by how fast we can get the stuff out.”

“So what, where do you see, like not just Sense but in the broader picture of this type of system, where do you see this going in five or ten years? Where’s the end game for you on this?”

“Well, we really do view that kind of the, the your home itself needs to become smarter in a way that it hasn’t before. Yeah, most people when they talk about smart home, they’re talking about you know automated lights or their entertainment system automating, which are all fine, I’m not trying to knock those.

But that’s to me just fundamentally different than the your home itself or the core systems in your home becoming smart for efficiency, for reliability, and safety, you know things like that. I think is what’s coming and has a real, real benefit to consumers from saving money, from a making their homes more reliable and more healthy.

So you know, I think that’s where the opportunity is. It’s kind of what happened in cars. I mean, you know, cars 50 years ago where independent mechanical systems and you know, carburetors and points that you had to go and tune up every couple weekends. And since then they became incredibly instrumented and automated resulting in huge efficiency gains and huge reliability gains. In the same should be true of homes.

I mean homes are harder, right? Because they are built out of independent systems versus built by one, you know, as a system itself. But doesn’t mean that the same principles don’t apply, that by making this thing smart we can make these these big gains around efficiency, reliability and convenience.

“Whenever I’d make smart home videos, one of the things I’m always telling people is it’s more than making your lights flash green and red by talking to your voice assistant. Smart homes have so much more value to our everyday lives than just that.

“And look, I think one that’s been under, under explored even by us also is these human things like health of the occupants, right? Things like air quality, you know, like do you have a radon fan and is it running at the right rate? Do you have a ventilation system and is it running appropriately? These things will start to be, there’s no reason your home can’t know things like the indoor air quality and you know, just as needed to make sure you’re healthy. So and those are big, big benefits to people obviously.

“So one of the other things that we’re, we’re excited about and working on that’s gonna result in even more doing what we can do today is is just getting more details about what’s actually in your home. Because right, right now what we’re doing on the machine-learning side, on the unsupervised side is, we get these data signals a little bit of hints from the consumer but then we have to figure out everything about what’s going on the home, right?

If we start to work with like home builders for example where they already know what dishwasher is there what furnace is there what hot water heater is there. In knowing ahead of time, what systems are there how they’re supposed to be working. And now our job is to make sure that they’re working as expected, you know. There’s a lot more value that we can provide versus our current kind of handle. Every single house out there case so to be more curated because you know exactly what’s going on ahead of time. 

That’s right.  So that kind of curated we think it also becomes really important for things like performance monitoring over time, right. If you have a an air conditioner, is it maybe it was super efficient when you got it installed. Is it still efficient three years later? Well if we already know, we’ve been watching it the whole time. We can say yeah it’s it’s worth fine or no it looks like it’s been degrading. Maybe the refrigerants leaked out. Maybe you should have a look at that well. 

Thank you so much for taking the time to talk to me today great to meet you, I’d really like to thank Mike Phillips for taking the time to talk to me. It’s companies like Sense that are helping to not only raise awareness of where our energies is going but building the foundation of AI systems that can eventually help us know when devices are breaking down and to help keep our homes safe and healthy.

With knowledge comes power. Get it?  Bad dad joke I know,

Supercharging a Net Zero Home – Ultimate Smart Home Tour

Introducing Sense: Home energy insights in real time