California is in the midst of a hugely transformative, experiment: an effort to redesign the future of electricity generation, distribution and use to meet a surging demand. The complex system that powers the world’s fifth-largest economy is at a turning point.

Utility executives, policymakers and government regulators are peering into a future where California has shed fossil fuels and is fully buzzing with electricity. Before the state completes its shift to a modern, safe, sustainable energy grid, it has to decide precisely how that should be accomplished.

Among the key issues:

• The grid is aging, large pieces of it having been installed to serve a state with a few million people, not today’s 40 million. Its hundreds of thousands of miles of high-voltage transmission lines throughout the state, serving virtually every home and business, are at full capacity, operating well past their designed limits. Some of the system’s vulnerabilities—even its lethality—have been laid bare by wildfires, when power was interrupted by flying tree limbs and communities were devastated by blazes sparked by broken equipment.
• The 100-year-old system in which power flows one way, from mega-utilities to their millions of customers, is coming apart. Power now runs into and out of the grid from multiple sources, all the time. Millions of customers have turned their backs on utilities in the past decade. Many homeowners already run their lives with power generated on their own roofs. Scores of small towns and counties have cut the cord and now operate their own mini-utilities, buying power directly from wholesalers. More than 100 “micro-grids” have sprouted across the state, producing and sharing their own energy. And these trends could grow as utility companies blamed for wildfires face increasing hostility from consumers.
• Storage is the Holy Grail. By law, the state must obtain all of its power by 2045 from clean sources, including sun and wind, which are cheaper than ever but unpredictable and difficult to fully harness. For example, more solar power may be generated in the middle of the day than the grid demands, and California often pays neighboring states to siphon off the unneeded bounty. That abundance presents a technology problem: how to store excess energy until needed.
• The grid could become Big Brother. The “internet of things”—devices large and small that connect us to each other and the grid—offer convenience and ease of control over our lives but also require the user to relinquish some degree of privacy. Consumers are monitored, analyzed and observed—even spied on—by smart devices attached to the walls of our homes like barnacles: thermostats, doorbells, nanny-cams, even speakers that can eavesdrop on conversations long after the devices were switched off.

“Things are changing so fast, this is the time to be intellectually flexible and adaptive,” said Arun Majumdar, co-director of the Precourt Institute for Energy at Stanford University. “We are still in the paradigm and the architecture of … the grid we have had for 120 years. Now we have to modernize.”

That requires a better understanding of what humans need from the grid, and what the grid requires from us. Enter demonstration projects like a five-year experiment at UC Irvine, sort of a real-time Truman Show in which the homeowners were the subjects. Their households received smart appliances, LED lighting, water heaters, insulation, air conditioning, solar panels and batteries, even electric cars and charging stations.

The trade-off for residents was their every decision noted from a remote monitoring station: which lights were flicked on, and when; which families used air conditioning or hot water more than others; which wall sockets residents used. The university, Southern California Edison and the federal Department of Energy sponsored the project, whose goal was to better understand, a street level, how a new generation of appliances and a fleet of electric cars could affect the grid.

Gene Tsudik, a UC Irvine professor and one of the participants in the study, is a computer scientist specializing in privacy and security. His professional antennae were fired up when teams of installers left behind clicking, ticking, blinking monitors that provided his family’s interface with the grid.

“I was very well aware that even simple devices that transmit wirelessly can triangulate the movement of people in the house,” Tsudik said. “We are surrounded by smart devices all over the place. Of course, they violate your privacy.”

Ultimately, Tsudik said, the experiment did nothing to change his family’s energy consumption. But it left him with the certainty that when his home is connected to the grid, the smart devices had better be secure from cyber-hackers as well as eavesdroppers.

“It’s a question of when, rather than if, they can be hacked,” he said. “What you can do well, hackers can do better.”

The hive mind of those blocks in Irvine spent years sending out data, ending in 2015. One takeaway for Scott Samuelsen, director of UCI’s Advanced Power and Energy Program, which ran the project, is that while the adoption of smart home devices is growing fast, regulation and consumer protections that should accompany them are not keeping pace. “The market is out of control with respect to regulation (of devices),” Samuelsen said. “We are in a free-for-all.”

Some aspects of grid modernization—consumer technology and utility upgrades, for example—are indeed under way. But other elements—policies, regulations, new business models—could require another decade to resolve. And who will pay for the upgrades? The cost of wholesale retrofitting is unclear but could run into the trillions of dollars, borne largely by utility companies—who may well pass some of that expense to their customers.

And what will a modernized grid look like? Some policymakers envision a centrally managed Western grid serving everyone from the Rocky Mountains to the Pacific, not ruled over by California alone. Others say simply that the grid of the future will be a good listener, a more responsive partner and a rapid problem solver.

More and more micro-grids will form, ranging from a family with solar panels to hospitals, malls and small counties taking care of their own electricity needs. The state has been encouraging this: A new law begins the process of setting rates and rules, and the California Energy Commission awarded more than $50 million in grants to accelerate the creation of smaller, more resilient power grids.

Regulators have ordered power companies to make their equipment safer, particularly to withstand—and not to cause—wildfires. Much of the equipment we can now see will either be buried safely underground or armored heavily to protect it from the elements.

Wind and solar power plants will rely on energy storage to make clean energy available when it’s needed, not just when it’s produced. One intriguing notion is called “vehicle-to-grid,” or V2G: electric cars storing excess power, which their owners sell back to the grid when the vehicles sit idle. Those same owners could also use the cars’ untapped energy to replenish energy storage they may have at their homes.

The state is encouraging the quest for storage solutions with an incentive plan extended last year to 2025. It added $800 million to a program that aids homeowners in adding batteries to their solar setups. And concerns about the environmental hazards of disposing of conventional, chemical batteries at the end of their lifespan have prompted development of non-chemical storage.

The new grid will also be populated with devices attentive to the nuances of electricity. Microprocessors will snap open and closed instantly, correcting imbalances in the ebb and flow of power. To that end, California utilities are already employing artificial intelligence in their operations.

Supervisors at San Diego Gas & Electric, for example, are able to monitor and shut off circuits to as few as 20 customers at a time when trouble looms, closely pinpointing the threat of weather damage or fire while also averting widespread outages. This makes power cuts ahead of dangerous winds or fire more palatable to residents. A smart grid, with all of its in-the-moment information about supply and demand, will allow utilities to better understand when consumers want power and even charge them accordingly.

Utility customers in California have begun to shift to a time-of-use system in which they pay more at times of high demand, typically in the evening or during extreme weather. Those rate adjustments are possible with interactive devices letting both utilities and customers know what power costs at any moment.

But full adoption of smart meters has run head-on into resistance, and not the electrical kind. While nearly half of American households are tethered to them, the devices are not uniformly welcomed.

California’s largest utility, Pacific Gas and Electric Company, was rolling out the first wave of its multibillion-dollar smart-meter program when customers in Bakersfield bridled. Homeowners claimed the devices were an intrusive tool of the state and the low-energy radiation they emitted posed a health hazard, among other grievances.

As complaints began to spread, PG&E began an opt-out program allowing homeowners to remove the new wireless meters and reinstall the old analog ones. That customer revolt has been called the “Bakersfield Effect.”

The state’s mandate for clean power has generated its own backlash, with renewable energy cast as a grid destroyer, given its feast-or-famine power delivery. To Anne Hoskins, chief policy officer at Sunrun, a San Francisco-based residential solar panel and storage installer, that’s an out-of-date argument.

“We have batteries now,” she said. “There are incentives for building storage. We are not a problem; we really are an opportunity.”

The rise of smaller, local alternatives to the big utilities is a trend known as community choice aggregation. The framework for these groups was established in 2002, motivated partly by locals’ wish to decide for themselves where they would purchase power and what they were willing to pay for it, as well as a desire for more clean energy. The community choice movement was sparked along the Northern California coast and in counties in the Bay Area and has spread across the state. Small aggregators now have 2.5 million customer accounts, including tens of thousands in San Jose.

These local power hubs must report their activities to the state Energy Commission and the Public Utilities Commission, in much the way legacy utilities do. The companies are not currently subject to all of the same rules that govern the big utilities across a web of complex issues, including compliance with California’s clean-energy goals. But the state is in the process of formalizing additional regulations.

Barbara Hale, assistant general manager at San Francisco’s municipal power utility, is also on the board of directors of the California Community Choice Aggregation Association. She said the groups are good grid citizens. But she understands that disrupting a century-old model can be unsettling.

“Part of the regulators’ role is to be cautious and thoughtful,” she said. “It’s difficult to be in that that role and manage change, particularly technological change.”

Ed Randolph, deputy executive director for energy and climate at the Public Utilities Commission, agreed that the state must pivot. But he says policies must be carefully considered, and not adopted at breakneck speed.

“There are people who want to just completely change the paradigm of how the grid is operated. ... I’m not sure that can happen,” Randolph said. “Our approach is, ‘Let’s do something on a smaller scale and understand how it impacts the grid.’ … The legislative and regulatory process is set up in a way that makes sure we don’t get too far ahead.”

California’s grid operator and its cadre of electrical engineers share the concern that runaway innovation could outstrip oversight and create precisely what the grid, even the modern grid, can’t abide: imbalance. “Changes are happening rapidly and we’re trying to keep up with that,” said Mark Rothleder, a vice president at the California Independent System Operator, which manages the state’s grid.

“There will be new players, new resources, new opportunities. We have to be open to innovation and we have to ensure our technology that manages the grid can enable those resources. We have to pay attention.”

This is the third in a series of articles to explore California’s need to modernize the electricity network that powers the state, published in partnership with the Sacramento Bee. This is an abridged version of a feature from CALmatters, a nonprofit, nonpartisan media venture explaining California policies and politics.