Apple’s first smart display needs to make the smart home just work

The smart home has an interface problem. Searching for an app on your phone, fighting with a confused voice assistant, and fiddling with a “smart” display to turn off a light is enough to send anyone running back to the wall switch. But now, Apple is taking a crack at solving the frustrations of smart home control — and it has the opportunity to actually make the smart home work.

Next year, the company will reportedly launch an “AI wall tablet for home control,” and it’s said to be developing more devices for the home (including cameras, a tabletop robot, and maybe even a TV). Among other features, such as video calling, this new wall tablet will reportedly be a hub for Apple’s home automation platform, Apple Home, providing a communal household interface for controlling smart devices like lights, locks, security systems, and cameras.

It’s about damn time Apple took the smart home seriously, having let Apple Home / HomeKit largely languish in the decade since its launch. All signs point to a renewed interest here, kick-started by the company’s involvement in Matter (a new smart home connectivity standard it helped develop) and spurred by a need to find its next big thing.

Apple’s supposed first new home product jumps right into one of the trickiest categories: the smart display. Originally designed as a way to “show” you what the voice assistant in your smart speaker was doing, today’s smart displays have become a jack-of-all-trades and master of none. It turns out that stuffing a smart home control panel, video calling device, camera surveillance system, digital photo frame, calendar, alarm clock, and even a kitchen TV into an underpowered, underprogrammed touchscreen stuck on a voice-activated speaker has not been a huge success.

Can Apple do any better? Last week, reports from Mark Gurman at Bloomberg gave us our first real details about the long-rumored Apple Home device, the HomePad, as Gurman calls it. And it sounds like a fairly standard, albeit small, smart display.

According to Gurman, the product will have a square, six-inch tablet form factor that can be wall-mounted (like Amazon’s Echo Hub) or docked to a speaker (like Google’s Pixel Tablet). It will have a built-in security camera (like the Nest Hub Max) and sensors that adjust the screen as you approach to show a more detailed interface (like the Echo Show 8). And, of course, it’ll play music, act as a video intercom, stream footage from security cameras, and control smart home devices (like every smart display on the market).

The problem is, with all their many capabilities, today’s smart displays aren’t any better than smart speakers for controlling your home. The original smart displays — the Echo Show and Nest Hub — were designed as voice-first devices with laggy touch control added as an afterthought.

While high-end smart home tablets that run pro-install automation systems like Crestron and Savant are more capable, they require hours of programming by a professional. The same goes for control dashboards from powerful DIY platforms like Home Assistant, which need more time and expertise than most homeowners have. Recent consumer-level tablets like the Echo Hub and Pixel Tablet have gone some way toward addressing this interface problem, but they are still too complicated to set up and program for the average user.

Apple needs to bring its signature simplicity to this space: it needs to make it all just work. To be successful, Apple needs to offer an intuitive user interface that effectively combines voice and touch in a way no other smart display has to date. This won’t be easy, but developing new user interfaces that push existing device categories further is a proven path for the company (see iOS, tvOS, watchOS). 

While it’s late to the game, Apple has a few things going for it. The company’s HomeKit framework and Apple Home smart home platform leverage local control, meaning that, unlike competitors Amazon and Google, it doesn’t rely on a cloud connection. Matter is reducing some of the barriers to entry of Apple Home: cost and availability of compatible products. So, on day one, an Apple Home smart display could connect to and control smart devices like lights, locks, cameras, and appliances already in people’s homes.  

According to Gurman, the HomePad will run a new homeOS codenamed Pebble. This will be based on Apple TV’s tvOS and have a pared-down touch interface that incorporates elements of watchOS and the iPhone’s StandBy mode. Gurman says Apple intends for most users to interact with the device using voice.

If Apple can figure out an effective interface that combines voice and touch, it can succeed here. Imagine a flow such as saying, “Siri, turn on the lights” as you approach the device. The screen then shifts to show relevant controls based on contextual inputs, giving you a way to easily adjust the assistant’s action with one tap. That would be a huge leap forward from where we are today, but the possibilities are much greater.

Here’s where I’m both worried and hopeful. Apple’s biggest problem is Siri. While capable of basic tasks, it’s significantly behind the competition, especially with smart home voice control. For this to work, Siri needs to get a lot smarter. Gurman reports that the HomePad’s hardware is designed around App Intents, “a system that lets AI precisely control applications and tasks,” and will “bring Siri and Apple Intelligence to life in a way that hasn’t happened before.” 

But it will need a lot more intelligence to take the voice assistant from simple command and control interactions — “Siri, adjust the thermostat to 68 degrees” — to understanding and interpreting context. For example, saying, “Siri, I’m cold,” and the device knowing to adjust the heat in the room you’re in. That’s a much better user interface — if Apple can pull it off.

While Apple said some form of a smarter Siri is coming, it looks like we’ll be waiting a while for a truly capable assistant to run our smart home. Gurman reports Apple’s more capable, conversational LLM-powered Siri won’t be here until at least 2026. While both Amazon and Google have also said they are working on similar capabilities for their voice assistants, neither appears close to launch.

It seems meshing a digital assistant’s existing technology with its new LLM-powered future is a complicated endeavor. These companies face the hurdles of a decade of technical baggage in the code, millions of devices in customers’ homes, and voice assistants that people already rely on for basic controls. We’re still waiting for Amazon’s “new” Alexa to roll out because, apparently, it can’t reliably control lights. And, in all its Apple Intelligence hype, Apple hasn’t once mentioned that its smarter Siri might also control your Apple Home.

Apple has a history of a tortoise and hare approach to emerging categories, taking its time, “borrowing” ideas from successful products, and then adding its layer of innovation on top to seal the deal. In this case, Apple needs more than that. It has to nail the interface and the voice assistant. That’s a tall task, but the market is waiting for someone to make it all just work.