A virtuous cycle for a data-driven energy economy in Europe

Based on Article 24 the Directive 2019/944 on common rules for the internal market for electricity [1], the European Commission’s EU Smart Grids Task Force Expert Group 1 [2] has been working for years now to draft a Consistent European approach to facilitate interoperability of energy services and data access across the Union. We have been having lots of broad and intensive stakeholder discussions, coined Implementing Regulations and have quite a clear roadmap forward now.

I don’t want to go into detail on legal acts now, but applied right, they can be a big step forward and serve as a basis for the development of a Common European Energy Data Space and truly interoperable solutions that work and compete in a Common European market.

Now, interoperability and free access to consumption data are fine, but what do we do with that? Who needs this? We see actors that are in need of that data already today to provide customers with services they demand. Examples for such actors are flexibility service providers, automated energy consulting services, 24/7 CoO matching, energy community operators, home automation and energy management systems, but also – for example – wholesale energy traders that have contracted solar farms and wind parks in different Member States.

Also, think of initiatives that help you calculating your personal or company carbon footprint – they need that data for an honest calculation. When it gets to smart EV charging solutions, it must be possible to charge smartly not only at home where you bought your car – say – in the Netherlands, but also when you’re on skiing holiday in – say – Austria.

We see a tremendous variety of so many absolutely great ideas and solutions out there – yet too many of these actors are at the moment acting on a national nutshell with their tools and solutions.

The main reason for this is that they need to cope with national settings – Member State by Member State – themselves, each on their own. They have to gather onboarding prerequisites, find out how to get access to data in each country, who to ask, what it costs, when and in which granularity and reading quality time series are actually becoming available.

We see also a lot of hurdles for non-domestic data-driven services – meaning if you, as an Austrian entity would like get access to data in another MSs, you would have to face a diversity of issues. We hope that this is going to improve with the amendment of eIDAS regulation [3] and its usability for semi-public and private domains.

Another issue that we see is the integration capability of data-sharing infrastructures. They need to allow for effective user flows. In other words, it must be as easy as checking out with online payment services to share your energy data. Complications on that end currently lead to high drop-out rates.

At the same time, from a more general perspective, we know that 50% to 80% of the costs of data projects go into data integration. That means, many of these companies are investing a lot of their time and money into getting access to data and making it processable – solving actually the same problems each on their own. This needs to improve so that these players can take data access for granted in order to being able to focus on adding value and on their core business.

Now, I would like to point you to a publication of Lukas Ilves and David Osimo from the Lisbon Council for Economic Development and Social Renewal – A Roadmap for a Fair Data Economy [4]. Besides a series of other very valuable thoughts with regards to data regulation and development of data-based economies, they sketch a Virtuous Cycle for a Fair Data Economy, demonstrating the triangular symbiosis between 3 high-level fields:
• Customer demand
• Data portability and
• New data-driven services

When data is available, new value-added services can be created, and consumers use those services. This creates demand for better data availability and improved and extended services. Conversely, when data is not easily accessible, few data driven services are created and little demand is there for such data. Ultimately, consumers are not aware of such services and do not generate sufficient demand.

So, we need to make it easy for data-driven services to integrate their solutions. It must be easy for them to ask customers for consent and to lead their users through the process. We need to have an eye on data-sharing infrastructure to keep it safe and reliable to strengthen trust at the user side. And we must get to a point where all of these 3 facets under consideration are be tackled from a European perspective rather than a national one. Customers must be happy to share their data with service providers from other MSs, data-sharing infrastructures must be able to treat non-domestic data re-users the same way they deal with domestic ones. And data-driven services must be in the position to offer the same solution across (most) Member States at a European scale.

Now, how do these observations map to a vision for a possible solution within the existing European legal framework and compatible with existing national practices and subsidiarity? During the summer months, we have – with the help of the DSO association GEODE and other partners – gathered a group of electricity and gas grid operators, data-driven services, universities, regulatory and security experts to sketch the cornerstones of a fair distributed open-source and transparent common European data space for energy. We see a big opportunity to solve those issues with the means and practices already available now – in line with the European Commission’s Roadmap for the Digitalisation of the Energy Sector [5], with the Commission’s European Data Strategy [6] and the European Data Spaces Initiative [7] in mind.

Our key conclusions are:

We need a uniform European interface to data. Access to data of domestic customers will always need to accommodate with national data exchange requirements. Of course, this means considerable effort:
• Some countries run centralized systems or national data hubs to take care of online data distribution and other services. Others – like Energy Data Exchange Austria (EDA) – are utilizing de-centralized market communication infrastructure. Even others use hybrid scenarios.
• Data is expressed in different formats – sometimes you get it as CSV, sometimes as JSON, sometimes as XML under different schemas. A European interface to data should yield a pivotal format.
• Characteristics of data transfer are very diverse. Sometimes data can be downloaded through a REST API, sometimes you will be sent an encrypted “message” through B2B data exchange infrastructure, sometimes it is “streamed”, and in some areas you will get it as unencrypted emails.
• Onboarding procedures to even get in the position to request and receive energy data are quite diverse. If you want to access data from a data hub, you often need to register there to get an account. For de-centralised and hybrid data management environments, onboarding runs quite differently.

All of these options are established, viable and compliant with the European legal framework – therefore a uniform European interface to energy data must be able to deal with all them without intervening or attempting to disrupt national practices.

Potential of in-house data must be unlocked. With that respect, the Clean Energy Package (see Articles 19 and 20 of Directive (EU) 2019/944) only provisions that non-validated near real-time consumption data must be made available to final customers through a standardized local interface or remotely. That led to standardized interfaces following a variety of different standards used, being configured even in a multiplicity of ways. It is often very hard to get real-time data access to your meter. Even more, once data is flowing, it must be made available separately to data-driven services – meaning that legislation-wise, it is still locked in-house. In addition to this, we see a lot of valuable data streams measured by in-house sensors that would add a lot to remote services. Therefore, an implementation of a Data Space for Energy should not forget about the edge and provide means to give authorization to data-driven services to use that data. At the end, when data is flowing, it should be accessible via an integrated interface together with online data and interoperable public data like price feeds or a “current renewables share signal” as drafted for the recast of the Renewable Electricity Directive within the Fit for 55 Package [8].

A distributed, open-source solution is desirable. This means that we do not see a necessity for a “European Data Hub” or an instituted way to exchange data in a cross-border way – we favor a smart, competitive, distributed approach. We have sketched an architecture that allows data-driven services to talk to their customers and data-sharing infrastructures directly and without the need for central intermediaries or vendor lock-ins. It works the same for online data-sharing infrastructure as for in-house data access. The solution we are heading for is based on open-source code that can be downloaded and run with a single click on a student’s or energy community operator’s laptop, in an EV, on premises at a solution providers infrastructure, or installed through a single click in a cloud services market place. This means that service providers may run and pay for their own infrastructure and tailor it to their performance and scalability needs, use what they have if suitable or use the full potential of modern cloud-native infrastructure.

Interoperability with other data spaces is absolutely mandatory. There should be fruitful co-existence and a competition of ideas and concepts. Different Data Spaces should be developed in parallel and learn from each other –more importantly: they should be able to communicate with each other.

Feedback cycles between data-sharing infrastructures, legislation, data-driven services and customers must be established. In our discussions developing our proposal, data-sharing infrastructure operators learned from data-driven services about what they need and what blocks them, from customers how hard it is sometimes to even find out where to get access to their data.

For legislation and expert advice, we gathered a lot of experience on which procedures are actually implemented to which degree and in what manner in detail, strengthening our expertise for further legislative activities.

Also, peers learned a lot from each other. That is why we will – under the umbrella of our project – establish open best-practices round-tables for data-sharing infrastructure operators and also one for eligible parties. To create a room for these players to learn, improve and consolidate – and disseminate results to others.

Most involved parties are willing to go forward and co-operate to spin such a virtuous cycle and unlock the potential of energy data-driven services. Having all actors and legislation go hand-in-hand is a big advantage that we have here in Europe in comparison to other regions in the world. Hopefully, the establishment of a virtuous cycle as sketched before will also create beneficial pressure there.

What it takes is simply a forum and a focal point for needed activities to facilitate the development of energy data-driven solutions that work and compete in a common European market, beneficial for Active European Customers and the Energy Transition.

With our very active group of highly renowned organizations we want to drive that matter and if you would like to get in touch, I am looking forward to interesting discussions and potential options for future co-operation.

References
[1] Directive (EU) 2019/944: [open legislation in EUR-LEX]
[2] EU Smart Grids Task Force: [open on EC website]
[3] eIDAS Regulation (currently under revision): [open on EC website]
[4] The Lisbon Council for Economic Development and Social Renewal – A Roadmap for a fair Data Economy: https://lisboncouncil.net/publications/a-roadmap-for-a-fair-data-economy/
[5] EC’s Roadmap on the digitalisation of the Energy Sector: https://ec.europa.eu/info/news/action-plan-digitalisation-energy-sector-roadmap-launched-2021-jul-27_en
[6] EC’s European Data Strategy: https://ec.europa.eu/info/strategy/priorities-2019-2024/europe-fit-digital-age/european-data-strategy_en
[7] EC’s European Data Spaces Initiative: https://digital-strategy.ec.europa.eu/en/policies/strategy-data
[8] Fit for 55 Package (currently under negotiation): https://www.consilium.europa.eu/en/policies/green-deal/eu-plan-for-a-green-transition/