Creating electricity to power our industries, schools, hospitals, and modern lifestyles consumes 40% of all primary energy in the U.S. At Carnegie Science, we are studying what paths the electricity system could take to become net zero in carbon emissions in the future.
It would be incredible to have a clean 100% renewable wind and solar based electricity system. However, there are real challenges in meeting energy demand at all hours because the sun does not always shine and the wind does not always blow. These hurdles can be overcome with smart choices in energy storage and by wise planning based on studying the variability of wind and solar resources.
At Carnegie Science, we have built a computer model of a simplified energy system to study net zero emissions systems. Any energy system our model designs must be able to supply electricity to meet the desired consumption of the U.S. for every hour of every day in the future. To begin to understand what is required, we use historical hourly electricity demand as one of the model inputs.
One of my colleagues, David Farnham (@farnham_h2o), and I are working on preparing these historical electricity demand data for our model. The U.S. Energy Information Administration (EIA) graciously collects hourly information from the utilities across the U.S. and publishes that data for analysis and use by the public.
However, we are all at the mercy of the reporting practices of each utility. If utilities report outrageous numbers, the EIA publishes outrageous numbers. And, when these numbers are used in an energy model, they can lead to wild results.
David and I have been developing algorithms to identify these anomalous values. After identifying anomalies, we replace them with a best estimate of what the true value probably was. A great example of some strange values can be seen in the below graphic, which shows the hourly electricity demand for the PacifiCorp West service territory over 10 December days in 2016.
Even without any background knowledge of what electricity demand should look like, the problem region jumps out immediately. The demand increases by a factor of 7 for 24 hours compared to the surrounding data. There is also a sudden one hour drop in demand which we also flag as anomalous. Our brain is phenomenal at pattern recognition and at identifying regions which do not conform with their surroundings.
Imagine designing an energy system which had to provide electricity for those 24 anomalous hours. You would build a system 7 time larger than what is needed for the rest of the year. Utility rate payers would be up in arms.
We could visually check all 56 reporting regions in the U.S. for all four years of hourly data: 56 regions * 4 years * 8760 hours per year = 1,962,240 data points! Instead, we devise algorithms to scan the data for us.
A good algorithm is reusable. We are putting in extra effort now to design the best algorithms possible for the task with an aim of reusability. In 6 months, when there is a new 6 month chunk of data, we will simply run our code to clean it up and share the results with colleagues. David and I plan to publish our techniques and make the clean data available for everyone.
In two weeks, I am going to be sharing our techniques at an upcoming Open Energy Modeling workshop at the National Renewable Energy Laboratory. I hope that the intense effort we put into this work leads to a data product that other research teams can also use for their modeling.