The Weather Company, an IBM Business, talks about the analytics tools they use to help energy companies predict electricity consumption and traders understand expected changes in the weather. They also talk about how load forecasting is rapidly becoming all about handling Big Data, and how it’s going way beyond the spreadsheet systems of yesterday.
I caught up with Ed Cuoco, director, Data Science and Analytics at The Weather Company.
First, what’s the current status of advanced new data analytics tools and data scientists to improve forecasts for electricity consumption?
Cuoco said: “The sophistication and refinement in demand forecasting has increased by leaps and bounds over the last 5 years. The near-real time forward demand curve has become omnipresent within the demand, generation and merchant worlds. Today, the value of forward analytics lies in improving marginal accuracy and timeliness as well as including more diverse and volatile data sources (weather, renewable generation, etc) and applying the insight into more complex business problems where increased speed, accuracy and precision become ever more important.
“Indeed, forecasts have moved beyond capital planning, pricing auctions and energy to become critical across the energy infrastructure, from portfolio management and risk mitigation through demand management and even near-real time operational efficiency.”
So, why is there so much interest in data scientists?
He added that load data and related data sets are seen as core business assets; extracting the value of this data has become a critical task across energy-related businesses. This increased use also increases the criticality of contextual data quality (i.e., is load data that was good enough for thinking about yearly capital spend also good enough for near-real time trading?).
The role of the data scientist lies in the nexus of these two tasks; unlocking more value from diverse data sets (load data, customer data, operational data) while also helping utilities, ISOs, traders and other stake holders understand and overcome limitations in the data themselves to continue to drive more refined understanding and action.
Next, we need to know what sort of analytics tools does The Weather Company work on/with, and what can those do?
Cuoco said: “We use most of the common analytical tools in the market, but what differentiates us is our underlying data platform and its ability to handle huge amounts of weather and load data. Every day The Weather Company maps 62 vertical miles of the atmosphere, all around the globe, to deliver 35+ billion forecasts along with 100+ terabytes of data.
“Four years ago, The Weather Company went through a transformation to a new cloud-based, cloud-agnostic, data-driven infrastructure. That “SUN” platform is an efficient Data as a Service (DaaS) platform that reliably handles these incredible workloads, leveraging 249 different open source tools with proprietary capabilities.
“Most of the platform was written in Scala, and a few of the technologies it leverages include Cassandra, Spark, Riak, and Redis. This platform allows us to turn Big Data into better decision making and provide demand forecasts to energy traders and utilities.
Finally, how is load forecasting now becoming all about handling Big Data?
The robustness, availability and quality of data sets varies tremendously from firm to firm and geography to geography, so there’s a lack of standardization and maturity. In some cases, load data is flowing in near real time at the meter level, in others, only at the grid level, and some in between.
With the implication that all the analytics are the same in principal, the real value to the industry is unlocking the value in the data sets as they exist now, not simply saying “when everything is robust, pristine and perfect, then you’ll see the value.”
What questions can be answered and how confident can one be in that answer is the crux of the big data question in this space. At the same time, load forecasting plays a central role across the energy and utility space, helping drive everything — from real-time demand response to optimizing the use of existing grid infrastructure and preventing catastrophic failures.
These predictions look minutes ahead for traders and days ahead for an ISO or utility, and the business needs to switch seamlessly between geographic and temporal filters. As the questions become more complex, interval meter data is critical but insufficient. Complex weather data, sensor data and customer behavior data all play a role in truly forecasting load and then applying that forecast to finance, operations and customer service.
And, how is it going beyond the spreadsheet systems of yesterday?
Cuoco concluded: “As analytics begin driving ongoing operations as well as strategic planning, the need to handle volume, speed and complexity dwarfs what even the most complex set of spreadsheets can accomplish while also pushing the limits on the underlying products used to create them.
“Further, when this increased business relevance is combined with the more robust, complex, and diverse data, the constant need to revisit and update even the best models becomes critical; analysis is no longer sufficient and, even now, we’re seeing the combination of advanced statistics and machine learning evolve.
“Soon, companies will need solutions that truly learn, interact and reason. Spreadsheets can’t keep up with the load or the complexity and, when pushed to their limits, increase the chances of critical failures in key analytical processes.”
Here’s my last piece for Control Engineering Asia on what should we expect in 2016?
There are certain factors such as the adoption of IoT, advancement in M2M, the increasing demand of robots in the industrial sector, etc., that can result positively on the industrial control and factory automation market.
The global industrial automation services market is likely to exceed $50 billion by 2019. The industrial controls system market is likely to reach $81 billion in 2021!
For industrial controls/valves, the Asia-Pacific market for industrial valves is projected to grow at 6.88 percent CAGR between 2014 and 2019. The industrial control system (ICS) security market is likely to grow at a CAGR of 7.6 percent from 2014 to 2019.
How about PLC/DCS and SCADA? One observer said that the PLC market in Europe is likely to grow at a CAGR of 7.44 percent from 2014-2019. The Asia-Pacific market for PLCs is likely to hit $3.6 billion in 2016. Also, the APAC DCS (decentralized automated control systems) market is set to grow at 4.36 percent CAGR over 2014-2019.
The Southeast Asia supervisory control and data acquisition (SCADA) market is likely to double in five years from $148.7 million in 2012 to reach an estimated $274.8 million in 2018. For PLM, the market is likely to reach $75.87 billion by 2022. The CAD market in aerospace and defense sector in North America is set to grow at a CAGR of 4.2 percent over 2014-2019.
IoT to dominate?
Nearly everyone’s favorite subject happens to be the IoT. The industrial IoT market is likely to grow at 26.56 percent CAGR to 2019. The M2M connections market will likely be worth $35.16 billion by 2020. Even the IoT in retail market is said to be worth $35.64 billion by 2020.
The installed base of wireless M2M devices in the oil & gas industry alone is set to grow at CAGR of 20.1 percent from 0.5 million units at end of 2014 to 1.25 million units by 2019.
How about sensors? Asia Pacific is set to reach $5.23 billion by 2020 at CAGR of 4.09 percent. In water/wastewater, the water recycle market and water re-use market growth are expected to be at 22 percent and 19 percent CAGR to 2019.
Hope some or all of these targets are met in 2016.