Multi-criteria Evaluation

Why multi-criteria evaluation is important for Smart Local Energy Systems?

Smart local energy systems (SLES) have the potential to deliver lower-carbon, more affordable energy including electricity, heating and transport. To ensure that this potential happens there is a need to be able to evaluate the performance of SLES projects to understand what works, for whom and in what situations.

Multi-criteria evaluation methods are used to analyse the performance of complex systems. For example, access to affordable energy may enhance lives and reduce poverty, but this should not be at the expense of the environment, so multi-criteria evaluation can be used to understand the trade-off between these factors. Multi-criteria evaluation can provide reliable information on the strengths and weaknesses of different SLES options, which help to inform decision making. A range of different metrics are used to do this, such as technical performance or financial viability.

The outputs of a multi-criteria evaluation can provide stakeholders including investors, project and technology developers, users, regulators and policy developers with impartial evidence to help them make decisions. The evaluation will also help to identify barriers preventing further SLES being developed which will help to increase confidence and reduce risk, potentially leading to the wider adoption of revolutionary SLES.

The challenge is that there is currently no clear consensus on what a “successful” SLES is. Measures to define success need to be identified across all areas, from environmental impact to social, technological and financial performance.

Evaluation methods like Technology Readiness Level (TRL) or Technology Performance Level (TPL) are already available which can be used to help determine where a technology sits on the pathway to success. These methods are limited and do not include measures to assess aspects relevant to SLES, such as:

  • bankability - how close to market the business models are, the types of financing and insurance required that would be needed for large scale deployment etc.;
  • regulatability - how complete the regulatory framework is to consent to such works, permits and controls required for large scale deployment etc.;
  • geographical potential - what kind of communities the technologies can be used with, how widely they are deployable across the country or restricted to specific types of community - urban/rural/island etc.;
  • community benefits – job creation, how lives will be improved, improvements in other sectors like welfare or healthcare provision etc.

EnergyREV will develop and build upon multi-criteria evaluation tools to meet SLES requirements so that they can be evaluated across all aspects in a fair, open and transparent way.

How will EnergyREV deliver unique and useful insights?

EnergyREV will develop, test and apply a multi-criteria evaluation methodology that can be used for SLES. This will build upon existing evaluation tools, together with other outputs from across EnergyREV, to develop a set of processes for a fair evaluation of SLES projects using a broad range measures to assess success. An early key piece of work will be to identify these measures that indicate success, which will be carried out by seeking agreement from across the EnergyREV consortium, the PFER Demonstrator projects and other relevant stakeholders.

The output of the evaluation methodology will be a snapshot of the performance of an SLES project across a number of areas of success. These areas will be aligned to the UN Sustainable Development Goals. This will enable stakeholders to focus on areas that are relevant to their project; for example, a system developed for a remote island with significant renewable energy resource can be expected to score very well on environmental sustainability, but poorly on potential for adaptation to an urban location. These would both be considered acceptable, as the latter is unavoidable, and the stakeholders might be more interested in focusing on areas like financial or social performance.

The evaluation processes will be demonstrated through testing on the PFER Demonstrator projects and other case studies. This will not only provide information about the performance of the Demonstrators for policy makers, stakeholders, end users and regulators, but will also provide information about the potential for SLES to meet the aspirations of the PFER programme: providing cleaner, cheaper energy, while creating high value jobs for the UK.

The evaluation will provide information for stakeholders to identify the key areas where there are opportunities for improvement in a SLES. Other themes within EnergyREV will provide insights into techniques to implement these improvements without causing a negative impact in other areas.

What will the evaluation team deliver for EnergyREV?

The evaluation team is responsible for developing and publishing the multi-criteria evaluation methodology for SLES with accompanying analysis tools. This will be freely available to help decision makers identify a snapshot of the performance of a SLES on the pathway to success.

Our initial focus is to conduct a series of workshops to gain insights from stakeholders, industry and the wider academic community on the key features of a successful SLES. These workshops will be held around the UK in the first 18 months of the project, and include a broad range of participants, from community groups to energy companies and investors. This engagement is critical to ensuring that the views of all stakeholders are considered in identifying the criteria by which SLES should be evaluated. Throughout the project we will also be collating information from across the EnergyREV consortium, ensuring that findings from other themes are used to inform the evaluation criteria and can provide numerical inputs to the evaluation process.

Key questions to be answered through the workshops and subsequent activities include:

  • What constitutes a successful SLES?
  • What are the key features of a successful SLES; for example, low-carbon, affordable etc.?
  • How are these features measured in existing multi-criteria evaluation tools?
  • What metrics can be used to measure each of these attributes?
  • How should these metrics be weighted in an evaluation?

Once we have developed a draft methodology we will work closely with the PFER Demonstrator projects, along with other case studies, to further develop and trial the evaluation methodology.

At the end of the research we will provide stakeholders involved in the case study projects with the results of the multi-criteria evaluation. By combining the outcomes from the evaluation with the outputs of the synthesis theme, stakeholders who wish to implement SLES will have a better understanding on how to deliver successful SLES projects.