Hitting the books: AI can help us design the greener, cleaner homes of tomorrow

In his new book, SuperSight: What Augmented Reality Means for Our Lives, Our Work, and the Way We Imagine the Future, author David Rose delves into the current state of augmented reality and discusses how technology is already transforming countless industries – from food service to medicine to education to construction and architecture – and what it could achieve in the near future. In the excerpt below, Rose takes a closer look at two companies that are leveraging computer vision and generative hostile networks to transform existing properties into 21st-century electrified smart homes.

Supersight Front Cover

BenBella Books

Reprinted with permission from SuperSight: what augmented reality means for our lives, our work and the way we envision the future by David Rose, published by BenBella Books.


We should all use solar panels. Period of time. The average cost for a renewable energy system has fallen by about 70% over the past ten years, from $5.86/watt to $1.50/watt, so it’s a financial no-brainer. You can’t finance an installation for free and save a hundred dollars a month in the first month, and even more if you live in the sun-drenched south.

So why aren’t we? It’s complicated! Mathematics, logistics, taxation and aesthetics all play a role. Many homeowners fear it will make their homes shiny and reflective, such as the Tin Man from The Wizard of Oz. The process of figuring out the number of panels in what size you need requires learning to “talk about solar energy” in unfamiliar units like kilowatt hours. And change always involves risks, whether they are real or merely perceived.

The pro-climate mission of Boston-based company Energy Sage is to get people to electrify their homes. That means solar panels on your roof, an electric car, a home battery system, automatic blinds and a smart thermostat that pre-cools or pre-heats while you drive home. And they’re working with us at Continuum to make potential customers more familiar with the idea by showing them what an electrified version of their home could look like. Using publicly available Google Home satellite imagery, we’re sizing solar panels, placing them digitally on customers’ rooftops, and showing them what their path would look like from both the street and their neighbors’ fences. We’ll take those images and match them with data from Project Sunroof, a Google project that helps you calculate your roof’s solar-saving potential. Once you’ve seen the beautiful photos of your electrified home and realize how much you’ll be saving over the years — and have the visual and financial data to hand — it’s an easy decision to go ahead and make that change. bring.

Other home improvement projects will benefit from a similar approach envisioned by SuperSight. Let’s take a look at landscaping: another complicated, potentially expensive project with its own disorienting language, risks and a desperate need for pre-project visualization.

I met landscape designer Julie Moir-Messervy at an MIT pitching competition and was immediately intrigued by her mission: to give homeowners the confidence and tools they need to turn their barren yard into a collection of outdoor spaces. Her company, HomeOutside, helps people see new possibilities for their backyards using AI and computer vision. Once they’ve visualized their garden in an engaging way, the company makes it easy for them to realize that vision by hiring the landscape installer, providing materials, and even helping to spread the payments over time.

Landscaping isn’t just good for real estate values; greenscapes filter pollutants in the air that cause asthma, help people recover faster from illness, lower summer temperatures and even reduce crime. Good native landscaping powers a dynamic system that helps bees and birds, which in turn pollinate trees and reseed plants. Shade trees in the Southwest can reduce the need for air conditioning, and hedges in the Northeast reduce winter winds and heating bills. More trees mean more carbon capture — a ton over the lifetime of each tree — because they literally suck the bad stuff we produce out of the air while reducing runoff and erosion.

But “most people don’t do anything in their yard because they don’t know where to start,” Julie told me. “They don’t know what plants to select and how to arrange them, or they don’t know how to install a landscape design and take care of it over time.” I was so inspired to work on the problem that I accepted a position on its board and got to work.

HomeOutside trains a generative adversarial network (GAN) to automatically compose beautiful and sustainable landscape designs based on the thousands of designs (think of these as recipes) the agency has developed for clients over the past twenty years. The company uses Google Earth Engine and photogrammetry to start with a 3D representation of each address (currently only in the US). The GAN architecture then uses one network (the Generator) to create a new design, and another network (the Discriminator) to review or score the work. These two networks continue their iterative game, generating and scoring, until the discriminator judges that the landscape has a good composition: shade trees, natural pollinators, grass to play, pavements/decks and furniture for gathering areas, plant diversity, and so on.

Companies that sell plants, furniture, lighting and hardscapes are clearly interested in this kind of “imagination engine” technology, as it bridges the conceptual gap between the current state of one’s garden and what could be — and motivates so many more people to explore the dream real. Not only is it great for homeowners and outdoor retailers, it’s great for the environment too. But what the company’s eco-minded investors find most exciting about this project is its ability to change the landscape of entire neighborhoods on a massive scale. What if we could create a new national park in millions of backyards stitching together places for birds and bees? Each hectare of forest absorbs about 2.5 tons of carbon per year. What if we turned neighborhoods into key carbon capture zones?

I helped Julie and her team develop HomeOutside’s grand plan to proactively redesign 70 million front yards, then partnered with Home Depot, Lowe’s, Wayfair, IKEA and garden centers to give their customers a 3D redesign of their yard. to email. Customers simply go outside, open their phone and, using spatial world anchors in the app, walk through an immersive animated landscape on top of their current garden. A time-lapse view from sunrise to sunset shows why the edible garden was placed where it is. The winter visualization explains the choice of new pine trees between their garden and those of the neighbours. Spring flowers bloom with a cacophony of colors.

Will people be alarmed at the idea of ​​an algorithm proactively redesigning their yard, with new shade trees and naturally pollinating shrubs? It’s not like your front yard is private now, thanks to Google Street View. And if you’re selling your home, you may decide not to hire landscapers and choose to post images of the makeover version of HomeOutside instead to maximize your appeal.

Once this visioning technology is commonplace, many different fields will start using it. For example, Home Depot recently invested in a startup called Hover, which, after digitizing your home in 3D, will visualize and price new paint, siding and roofing materials. SuperSight will soon be showing the real paint crew on their ladders as they finish the last few brushstrokes, giving you that delightful experience of a job just completed. Volkswagen can put a new Passat in your driveway, complete with the kayaks and mountain bikes it knows you’ll love. And the company trying to sell you home and auto insurance? They project a disaster scenario: solar panels fall off, the shade tree is struck by lightning, and your new Passat is ravaged by a hail storm. Better buy the insurance before repainting.

How do we deal with this kind of immersive design? With our SuperSight glasses on, shall we point out and place trees, or paint flowers from a palette of choices, like a 3D version of Photoshop? Shall we select each plant from a vast menu of options for infinite control and customization, or shall we just tell the system what we like so it learns our preferences and then propose a single solution we’ll love? I believe in the golden mean: that we will largely prefer seeing and choosing from several “expertly curated” options, just as we do today when working with an architect, interior designer, or wedding planner.

Experts are usually so good at what they do that it’s often a mistake to over-specify certain details. For example, you shouldn’t tell an architect that you want a window right here, or an interior designer that you want this particular chair in a particular color in this corner. Instead, express your opinion at a higher level of abstraction (“I want the space to feel more connected to the environment”) or by describing a required feature (“We want a vegetable garden”), and let them do the detailed work .

The same expert-led interaction model will dominate our relationships with SuperSight AIs. For landscaping, we can ask for a more formal French garden with rectilinear layouts and exotic colorful plants, or a curvaceous organic design that prioritizes privacy from our neighbors. We can indicate a preference for an open space for play, or for a completed schedule with more space for a productive garden. And as we express these interests at a higher level, our 3D landscape design is dynamically recalculated to suit our preferences. With SuperSight glasses on, we can test our hunches faster by seeing reconfigurations instantly and in context, on top of our real home.

The jury is still out on whether HomeOutside can use this technology to convince millions of homeowners to invest heavily in a sustainable landscape. However, the testing is promising; customers are delighted to see their yards redesigned and redecorated. Over the next five years, HomeOutside plans to use Google Earth and Street View imagery in a generative AI tool to automatically redesign tens of millions of landscapes, with sustainable plants, shade trees, natural pollinators and bird-friendly berries. If successful, that means a million homeowners will plant at least 3 million new shade trees, such as oak and beech, each of which will capture 48 pounds of carbon per year as they age. That’s 14 billion tons of carbon captured over the lifetime of those trees.

As one of the HomeOutside consultants summed up, “You are building the equivalent of a new national park – our national park! Visualization tools like HomeOutside can convince homeowners to reshape the American landscape.”

That’s the ultimate potential power of SuperSight: helping people envision a future that benefits them and the planet.

All products recommended by Engadget have been selected by our editorial team, independent of our parent company. Some of our stories contain affiliate links. If you buy something through one of these links, we may earn an affiliate commission.

Give a Comment