Picture from loxone smart home

iPhone led us to the era of the smartphone, Tesla Model S is leading us to the era of smart car. What will be the next "smart" thing? And more importantly, how will it become truth?

According to a report titled The Internet of Things: Mapping the Value beyond the Hype by the McKinsey Global Institute, the economic impact of the Internet of Things could be $3.9 to $11.1 trillion per year by 2025: up to the 2 percent of the global economy. The smart house is one of the IoT consumer commodities that we are very likely to buy in the next decade.

In this article, the first principles are firstly adopted to start the analysis from the ground truth. Then a convergence perspective helps to identify the core difficulty to achieve truly smart house. And in the end, a possible solution using pattern recognition and deep learning is given.

A First Principles Perspective

Let's start with the first principles perspective to boil the smart house down to essential truths. Why do we need smart houses? What does current smart house look like? What do we want the smart house to be? Can current smart house satisfy our needs?

For current smart house solutions, most companies are focusing on smart devices in the house. For example, Phillips Hue light is a typical smart device in the house: after installing the hub and bulbs, we can easily control the states of light bulbs by interacting with the APP on phone by touching screen or speaking to Siri/Alexa. We can also set "certain scene" for "smart" control. For example, we can set the lights automatically on when I near home and set the lights automatically off at 12:00 in the midnight.

Yes, these functions are useful and delightful. They give us more ways (or say more switches?) to control devices in the home. Comparing with traditional switches which only have two states: on / off, new "smart" devices in the home give us more switches: slide button for light brightness, time button for setting on / offline time, etc. But they are not literately "smart": smart houses do not need switches because they read your heart.

Let's get one step further to figure out how smart house should be — a house without switches.

For me, I want my smart house automatically turn off lights when I go to bed, turn on coffee machine when I get up, open door when I reach doorstep, turn on heater when I feel code, turn on humidifier when I feel fry, play music according to my mood when I take a bath, tell me the step-by-step recipe when I want to cook, buy food for me when the refrigerator is empty, etc.

I also want it to know my preference: play more Jazz music, buy Starbucks instant coffee instead of coffee beans, tell me to put more chili because I like spicy dishes, wake me up a little bit later on Wednesday because my class on that day starts at 5:00 PM, etc.

What's more, a smart house may even help me to live a better life by changing my lifestyles. For example, I used to sleep late at 3:00 AM in the morning because of drinking too much coffee in the afternoon. If the smart house alerts me when I'm pouring coffee out from the coffee pot, I may live a healthier lifestyle getting a higher sleep quality.

These imaginaries are fancy, but are they possible to become truth? Let's figure it out from a convergence perspective.

A Convergence Perspective

As the figure above shows, the three principle elements of smart houses are:

• Sensor: Sensors are used to collect data as well as control devices in the home.
• Data: Data is collected by sensors and upload to the cloud for further analysis.
• Computing: Computing ability includes models for scene analysis and the ability to give instructions to sensors for controlling devices in the house.

The convergences of each two elements are what they can do with multiple elements:

• Sensor + Computing => IoT. The combination of sensor and computing is the IoT we are familiar with.
• Sensor + Data => WoT. WoT (Web of Things) adopts the idea of the Web as an application-layer for the IoT. With WoT, data from sensors can intuitively upload to the world wide web.
• Data + Computing => Machine learning. How to use to data collected and make good indication and prediction is the hard part to get an integrated smart house solution we want.

In general, the convergence of the three elements above is enough for a truly smart house. But just like the red question mark in the figure, we still don't know where to buy truly smart houses.

How can the Smart House Become a Reality?

What's the core we lose to get the truly smart home? Why companies like Phillips keeps doing smart devices like hue lights instead of integrated smart home solution? Why all current "smart" devices still have switches for us to control? Why are they not smart enough to control themselves?

I think the core is pattern recognition with deep learning. In recent years, pattern recognition is closely akin to machine learning. And most of the Pattern recognition systems are in many cases trained from labeled "training" data (supervised learning), which means using already labeled data to train a model and hope the model can be utilized to process unseen inputs to get the right category. Deep learning is a trend in machine learning emphasizing end-to-end learning. With inputs and outputs given to the model, the model can train and fine-tune the parameters to get good recognize models.

Specifically, we may build a model for smart houses to be truly smart with pattern recognition & deep learning by the following steps:

1. Get input data: Massive data can be collected by the sensors in smart house, like date, time, temperature, humidity, weather, wind strength, brightness, the time user going to bed, the time user wake up, the time user drinking coffee, the time user cooking, the time user refill the refrigerator, etc.
2. Get output data: Use current time as the standard to get data (e.g. every half hour, collect a set of data). Collect the current states of house, including: each light is on or off, coffee machine is on or off, heater is on or off, humidifier is on or off, music player is on or off, which music is playing, what food are in refrigerator now,
3. Train the model: Select relative data in the dataset to train models. For example, choosing the date, time, temperature, humidity, weather, wind strength as input, and the on or off state of the heater as output to train a heater control model. Similarly, more models like playing music model, refill refrigerator model, etc. can be trained. These models are highly customized because they are trained using data from the owner.
4. Generalization: Training process takes time, several days or even several weeks, to customize. In this process, a generally used model can be loaded first for usage.

More technically, LSTM (Long Short-Term Memory) / RNN (Recurrent Neural Network) can be used as the deep learning neural network layer structure because they regard the data as time sequence and take the time correlation into consideration. This fits with they way we collecting data every half minute or half hour, which is intuitively time correlated.

Hope the next smart thing is the truly smart house. :)