Advanced Grid Modeling


The electric power industry has undergone extensive changes over the past several decades. It has integrated new technologies, adopted new market rules and regulatory policies, and encouraged new players as part of its operation. As a result, the modern power grid has become substantially more complex, dynamic, and uncertain. Integration of distributed energy resources has led to tighter interaction between transmission and distribution grids. New devices and sensors have made volumes of more detailed data available to system operators and utility engineers for operational use. Power grids have become tightly coupled with other systems such as communication networks that carry the data. Such changes have led to significant challenges in the planning and operation of the modern power grid. It has become more important than ever to ensure that the modernized grid is reliable, resilient, secure, affordable, sustainable and flexible.

The Advanced Grid Modeling Program (AGM) under Department of Energy’s Office of Electricity (OE) has led the path to address these needs ovemany years. AGM supports the nation’s foundational capacity to analyze the electric power system using big data, advanced mathematical theory and High Performance Computing (HPC), to assess the current state of the grid and understand future needs. The portfolio of projects funded by this program has enabled grid operators to optimize their decision-making in real-time giving the industry sophisticated tools to substantially improve reliability, resiliency and grid security.

LLNL’s expertise in the areas of advanced power grid modeling and simulation, mathematical methods and computation, optimization, uncertainty quantification, risk analysis and decision support, and data analytics and machine learning, helps support the OE/AGM program advance its objectives in addressing the challenges with power grids.

Decentralized Decision Making for Improved Grid Resiliency


This project will develop algorithms for power systems operations and planning that perform core functions (e.g., market clearing and reliability analyses) in a decentralized manner, avoiding the need for a single coordinating agent with global system knowledge. Demonstrations of proof-of-concept on illustrative grid functions and test systems will be conducted.

A key feature of modern power systems is that they are centrally operated and controlled, at both the transmission and distribution levels. The centrality of associated decision processes, including dispatch and commitment at the transmission level, leads to economically optimal results. Yet, such processes are inherently brittle, as they involve a single agent ingesting global data, making operational determinations, and emitting corresponding control signals.

  • Decentralized operations strategies are inherently more resilient to communications drop-outs and disruptions, for example, respectively due to reliability issues and cyber attacks.

  • Beyond improving grid resilience and reliability, decentralized operations strategies offer significant advantages in terms of market structure, information-sharing, and privacy protections.

Decentralized Decision Making


  • Develop algorithms based on decentralized optimization, for addressing key power systems operations and planning problems.

  • Conduct studies of the efficacy of the developed algorithms, in terms of time-to-solution, solution cost, and algorithm resilience.


  • Leverage algorithmic decomposition techniques for optimization models that are variants of the alternating direction method of multipliers (ADMM), specifically asynchronous variants previously investigated by the team.

  • Focus algorithmic research on achieving convergence given bounds on communication latency and frequency.

  • Develop canonical test problems for decentralized variants of grid operations planning problems, including non-linear and mixed-integer attributes.

  • Publish technical reports and journal papers describing test problems, developed decentralized optimization algorithms, and experimental results.

  • Release all test problems and codes as open source.


  • Specified initial set of decentralized power systems operations problems to be investigated

  • Implemented initial test problem generator for decentralized economic dispatch and power flow

Principal Investigator: watson61 [at] (Jean-Paul Watson)

Emergency Control and Protection in Large Scale Power Grids under Uncertainty

This project leverages recent developments in robust optimization and control and fast-sampled but spatially sparse measurements collected from available phasor measurement units (PMUs) to provide improved algorithms for emergency control response.

Emergency Control

Current operational paradigms for emergency control primarily rely on relay protection devices and automated schemes such as undervoltage load shedding that effectively restrict emergencies to local regions and prevent cascading outages. Many utilities implement Remedial Action Schemes (RAS) or Special Protection Schemes (SPS) to improve system reliability and performance during maintenance scenarios or in extreme events. These schemes may significantly improve the system’s response to failures and are believed to enable better integration of renewable energy sources. While the benefits of SPS have led to widespread adoption, the potentially deteriorating effects of maloperation and unintended interference between different schemes can pose a serious risk to system operations and a threat to energy security.

This project aims to design real-time extreme event monitoring and identification methods and provide the computation and implementation of fast control actions in a similarly short timeframe as the traditional special protection schemes but revisit the way those systems are designed. Specifically, we consider a context where increased uncertainty in the system state and more frequent extreme events (e.g., snowstorms and tornadoes) affecting grid topology and line parameters lead to greater demand and diversity in the required control actions. The ultimate goal is to develop advanced automatic control schemes protecting the system.

We will develop algorithms that maintain the benefits of slower, traditional AC optimal power flow while acting on a fast timescale, similar to SPS, thus complementing and improving existing control strategies and improving the special protection and remedial action scheme reliability. We will also leverage recently developed stochastic and robust optimization tools to manage and mitigate effects of renewable energy variability in the design of RAS and SPS schemes. To assess the performance and validity of our developed methods, the project team will utilize the industry-grade power grid dynamic simulator (i.e., GridDyn), which is developed at LLNL under DOE support, and the HELICS co-simulation platform to integrate with more complex scenarios and simulations.


  • Worked with the LANL team to generate datasets from dynamic simulations of different fault events.

  • Identified two test systems (i.e., the WECC 179-bus and the Texas A&M synthetic 200-bus ACTIVSg200 systems) for the initial demonstration. We have translated the simplified excitation system (SEXS) model (used in the ACTIVSg200’s case’s .raw file) into the IEEE DC1A excitation system (ESDC1A) model format so that the model can be run in GridDyn.

  • Wrote a .xml script that simulates a single-fault event on each load bus in the reduced WECC 179-bus system dynamic model. The output files include quantities like pre- and post-fault bus voltage magnitude/angle, current magnitude/angle, and bus real/reactive generation.


  • Y. Maximov, D. Deka, W. Li, P. Top, M. Korkali, L. Roald, J. Gorka, A. Hou, A. Botterud, and D. Lee, “Robust Real-Time Control, Monitoring, and Protection of Large-Scale Power Grids in Response to Extreme Events,” DOE AGM First-Year Roadmap Report, Nov. 2021.

Principal Investigator: top1 [at] (Philip Top)

GridDyn Usability Enhancements


GridDyn is an open source dynamic power system analysis platform, developed in C++ at for exploratory application and co-simulation, to address the lack of open high performance software in that space. The name is a concatenation of Grid Dynamics. It was created to meet a research need for exploring coupling between transmission, distribution, and communications system simulations. See GridDyn GitHub page.

Griddyn Logo


While good open source tools existed on the distribution side, the open source tools on the transmission side were limited in usability either in the language, platform, or simulation capability. Commercial tools, while quite capable, did not allow access to the internals required to conduct the research.

Building off prior efforts in grid simulation, GridDyn was designed to meet current and future research needs of various grid related research and computational efforts. It is written in C++, making use of recent improvements in the C++ standards. It is intended to be cross-platform with regard to operating systems and machine scale.


GridDyn is an open source dynamic power system analysis platform, developed in C++ at for exploratory application and co-simulation, to address the lack of open high performance software in that space. The objective of this project is to enhance the usability of that application for use by others specifically in University and research settings. This includes, additional documentation, capabilities, connectivity components; improvements to the software; and improvements to the build/install system and use of the GridDyn libraries. GridDyn is being used in several GMLC projects including the HELICS co-simulation project, and CYDER, and as part of the California Energy Systems for the 21st Century. It is also part of several proposals to DOE, DHS, and other agencies.


The project team is partnering with the University of Tennessee at Knoxville to explore the requirements to apply GridDyn in the large-scale testbed, and to work through a series of use cases and code reviews to identify areas of improvements. This effort results in a series of steps to address specific weaknesses in the software, integrate them, and identify aspects of the code that could be extracted and used elsewhere.

GridDyn Chart


GridDyn was used as part of the Parallel in Time project and continues to be used to test and explore parallel in time algorithms. It was used as a part of the testing and examples for HELICS in the GMLC 1.4.15. Many of this project’s software usability lessons learned contributed to the development of HELICS and have since fed back into GridDyn. GridDyn is being used for projects related to infrastructure security, T&D co-simulation with CYME, and hardware-in-the-loop simulations for GPS timing spoofing. The project team has also started to address some of the components needs for broader usability, including optimization components, support for a future graphical interface, automatic code generation of boilerplate code, and expanded continuous integration testing.


Principal Investigator: top1 [at] (Philip Top)



Grid Data Crossing (GriD-Xing) is a data management and analytics tool that integrates data processing, storage, visualization, and analysis functionalities for users in industry (e.g., EMS and DMS) and at universities. It aims to enhance cross-domain data sharing and cross-sector analytics capability to power systems applications, and to enable engineers and researchers to better identify the power of large cross-domain data (i.e., big data), especially during resilience events.

Grid data crossing flowchart


Today’s electric power systems produce high-volume, high-variety data across multiple infrastructure sectors. However, those data streams are not archived and analyzed in an integrated framework, causing stakeholders’ responses and investments to be guided by incomplete information. It is a critical yet challenging task to develop new, efficient tools for cross-domain data management and analytics.


This project has developed a data management and analytics tool named GriD-Xing that integrates data processing, storage, visualization, and analysis functionalities for users in industry (e.g., EMS and DMS) and at universities. It aims to increase the usability and usefulness of large sets of multi-source data from across domains (e.g., SCADA, PMU, µPMU, AMI, inverter, and natural gas data), especially during extreme events, and ultimately to improve the resilience and reliability of critical energy infrastructure. GriD-Xing is also a research platform to identify the usefulness of large multi-source data via case studies of interest to industry (e.g., 2011 southwest cold weather and 2016 Blue Cut fire events).

Technical Approach

  • Data Usability Enhancement: LLNL developed a new shared, distributed data storage system named UPS for transmission and distribution PMUs (T+D PMUs) in Go language and tested the prototype with the benchmark metrics covering QoS, flexibility, and scalability. UPS’s computing query speed is 10 times faster than the benchmark storage infrastructure.

  • Data Usefulness Enhancement: LLNL developed a set of data processing and integration algorithms for single-source and multi-source measurements in both Python and MATLAB. With WECC and IEEE test systems, the team demonstrated that the proposed algorithms achieve higher accuracy for traditional event detection and state estimation applications.

  • Synthetic Data Generation: The team also delivered a synthetic data generation tool, for generating synthetic data of signal and multiple grid events and grid-gas cascading failure events.


The project team delivered two software tools, the GriD-Xing tool and the Single & Multiple Grid Events Generation tool. The team actively introduced those tools to academia, industry, and community partners.

  • Academia: To date, the GriD-Xing research has been developed into eight journal papers and one patent, and the software capacity has been leveraged by six proposals through collaboration with university and industry partners.

  • Industry: GriD-Xing is being utilized by several DOE and DoD projects with real-world data, indicating its applicability to real utility and military sites. Feedbacks from demo sites will help the team further develop the proposed techniques.

  • Community: The team is actively involved with NASPI and IEEE PES working groups. GriD-Xing work was cited and presented in the 2019 NASPI annual report and the 2019 IEEE State Estimation Working Group.


Data Processing

  • C. Huang, C. Thimmisetty, X. Chen, M. Korkali, V. Donde, E. Stewart, P. Top, C. Tong, and L. Min, "Power Distribution System Synchrophasor Measurements with Non-Gaussian Noises: Real-World Data Testing and Analysis," IEEE Open Access Journal of Power and Energy, vol. 2, pp. 1-6, 2021.

  • Y. Xu, C. Huang, X. Chen, L. Mili, C. Tong, M. Korkali, L. Min, "Response-Surface-Based Bayesian Inference for Power System Dynamic Parameter Estimation," IEEE Transactions on Smart Grid, vol. 10, no. 6, pp. 5899-5909, Nov. 2019

  • Y. Xu, X. Chen, L. Mili, C. Huang and M. Korkali, "Polynomial-Chaos-Based Decentralized Dynamic Parameter Estimation Using Langevin MCMC," 2019 IEEE PES General Meeting, pp. 1-5.

  • US Patent: X. Chen, C Huang, L. Min, C. Thimmisetty, and C. Tong, Computational framework for modeling of physical process, US20200202057A1

Data Storage

  • Kosen, C. Huang, Z. Chen, X. Zhang, L. Min, and Y. Liu, "UPS: Unified PMU-Data Storage System to Enhance T+D PMU Data Usability," IEEE Transactions on Smart Grid, vol. 11, no. 1, pp. 739-748, Jan. 2020.

Data Integration

  • J. Zhao, C. Huang, L. Mili, Y. Zhang, and L. Min, "Robust Medium-Voltage Distribution System State Estimation using Multi-Source Data," IEEE PES ISGT, Washington, DC, USA, pp. 1-5, 2020.

New Data and Models for Interdependent Energy Infrastructure

  • Z. Chen, C. Huang, S. Mukhopadhyay, M. Nygaard, and L. Min, “Interdependent Expansion Planning for the Resilient Electricity and Natural Gas Networks,” IET Energy Systems Integration, submitted.

  • W. Jia, T. Ding, C. Huang, Z. Wang, Q. Zhou and M. Shahidehpour, "Convex Optimization of Integrated Power-Gas Energy Flow Model and Application to Probabilistic Energy Flow," IEEE Transactions on Power Systems, vol. 36, no. 2, pp. 1432-1441, March 2021.

  • T. Ding, Q. Ming, C. Huang, Z. Wang, P. Du, and M. Shahidehpour, "Multi-Period Active Distribution Network Planning Using Multi-Stage Stochastic Programming and Nested Decomposition by SDDIP," IEEE Transactions on Power Systems, in Press.

Principal Investigator: huang38 [at] (Can Huang)

Hybrid Learning of Black-Box and White-Box Power System for Large-Scale Adaptive Optimization under Uncertainty


The project couples the hybrid and adaptive surrogates with a high-fidelity system model for multi-objective optimization that will seek desirable tradeoffs between multiple objectives, while assessing and managing the risk of failures in power grids.


A power grid consists of a collection of subnetworks (e.g., power subtransmission and distribution systems) with different characteristics. Some of these subnetworks have known physical laws and structures that allow us to derive parametric static and dynamic models of their operating states. They may be viewed as physics-driven “white-box” model components with known parameters and prescribed prior probability density functions (pdfs), which is the case for high-voltage power transmission systems and medium-voltage feeders of power distribution systems. U.S. transmission grids are also well-metered, so their state vector can be estimated in real time. Other subsystems, referred to as “black-box” components, may have an unknown structure or may be driven by unknown physical laws. For power distribution systems, these “black-box” components are exemplified by low-voltage power distribution systems, consumer loads, and distributed energy resources. Black-box components are also common in constraints and in objective functions that are very expensive to evaluate and devoid of analytical or derivative information, such as those pertaining to environmental and socioeconomic costs as well as unseen and rare operating conditions.


This project aims to develop an efficient framework for risk-averse decision making in power systems by considering the impact of the risks and computational tractability, with an emphasis on the following objectives:

  • Identify and characterize different forms of risks and rare events, in the context of decision making under uncertainty

  • Establish objectives, constraints, and metrics for risk assessment and awareness, which are valued with respect to the grid operation and investment planning

  • Develop an iterative approach to adaptively refine model fidelity that would yield the optimal construction of risk-aware models, which can balance the inherent tradeoff between multiple objectives (e.g., security risk and system’s operation cost)

  • Design scalable and robust methods for solving this class of multi-objective optimization problems, while identifying conditions under which bounded optimality can be attained


The project team proposes computational frameworks to optimize hybrid problems, involving both incomplete/unknown information and mathematically known functions, in such a way that maximizes the interchange between the known (i.e., “white-box”) and unknown (i.e., “black-box”) components.

Specifically, the team proposes a risk-based multi-fidelity optimization framework that adaptively utilizes low- and high-fidelity data and a hybrid learning scheme that combines multi-fidelity data so as to efficiently solve large nonlinear power-system optimization problems, thus facilitating robust, data- and physics-driven real-time decision making under uncertainty.

Hybrid learning risk based multi-fidelity optimization framework flowchart


  • An iterative response-surface-based, copula-enhanced approach to solve the chance-constrained AC optimal power-flow (OPF) problem in the presence of dependent non-Gaussian uncertainties of wind power generation

  • An adaptive importance sampling (AIS) scheme under the Bayesian-inference framework that simultaneously estimates the topology, the outages, and the states of a distribution system with a very limited number of measurements


  • Y. Xu, M. Korkali, L. Mili, J. Valinejad, T. Chen, and X. Chen, “An Iterative Response-Surface-Based Approach for Chance-Constrained AC Optimal Power Flow Considering Dependent Uncertainty,” IEEE Transactions on Smart Grid, vol. 12, no. 3, pp. 2696–2707, May 2021.

  • Y. Xu, L. Mili, M. Korkali, X. Chen, J. Valinejad, and L. Peng, “A Surrogate-Enhanced Scheme in Decision Making under Uncertainty in Power Systems,” Proceedings of the 2021 IEEE Power and Energy Society General Meeting (PESGM), July 25-29, 2021 (virtual).

  • Y. Xu, M. Korkali, L. Mili, X. Chen, J. Valinejad, and Z. Zheng, “An Adaptive-Importance-Sampling-Enhanced Bayesian Approach for Topology Estimation in an Unbalanced Power Distribution System,” IEEE Transactions on Power Systems (under review).

  • Y. Xu, M. Korkali, L. Mili, K. Karra, J. Valinejad, and L. Peng, “An Efficient Data-Driven Framework for Decision Making under Uncertainty in Joint Chance-Constrained Optimal Power Flow,” IEEE Transactions on Sustainable Energy (under review).

Principal Investigator: chen73 [at] (Xiao Chen)

Robust Dynamic Load Modeling & Uncertainty Quantification


Load Sculptor, the software that will be developed in this project, will yield reliable dynamic load models that enable planning engineers and operators to make important real-time operational decisions for enhancing grid security and stability. This software will leverage the latest mathematical advancements and cutting-edge algorithms from machine learning, uncertainty quantification, and optimization theory.

Load Sculptor


Accurate and reliable composite dynamic load models play an essential role in helping utilities perform planning, stability assessment, and control. Dynamic load models are usually derived first using labor consuming bottom-up component-based approaches and then adjusted via measurements recorded from the field. To capture different load characteristics, more complex models have been derived, such as the WECC CMPLDW, yielding heavy computational burden for time-domain simulations. Complex load models also suffer from parameter uncertainty. Uncertain model parameters may cause misleading stability assessment outcome, security concerns, and economic losses, therefore their impact on dynamic simulation also needs to be quantified.


This project will develop a data-driven, multi-fidelity robust dynamic load modeling and uncertainty quantification tool called Load Sculptor, with the aim of enhancing utility planning and operational models and decision making. The two main objectives are:

  1. Develop computational cheap surrogate load models for dynamic simulation and security assessment without sacrificing the accuracy requirement;

  2. Derive robust load model parameter sets that are able to capture a wide range of operating conditions while quantifying the influences of parameter uncertainties on the simulation results.


Divide the available data sets into different groups/clusters that have similar pre-disturbance and post-disturbance operating conditions. K-means and its variants will be used and centroid matrix obtained. Dimension reduction-based on Kernel PCA will be carried out to improve computational efficiency in the case of large-scale data sets.

Model reduction via deep neural networks (DNN) surrogate modeling and uncertainty quantification. For each cluster, Bayesian parameter inference will be carried out to estimate parameters that yield consistent model outputs with the measured response (statistically less than 3% RMS error). DNN will be trained to construct the surrogate models and derive their associated robust parameter sets that yield different levels of fidelity. The structure of the DNN will be parameterized to recover a low-dimensional nonlinear manifold. This provides the ability to carry out fast dynamic simulations while preserving the dominant model dynamic characteristics. Since the training of DNN is computation intensive, the Quartz computing clusters at LLNL will be utilized to facilitate efficient training and validation of surrogate models.

Dynamic load model validation and uncertainty assessment. Extensive validations will be carried out to demonstrate the developed models under different operation conditions. The project team will also reach out to utility partners for validations using field data. The parameter uncertainties on model outputs will be quantified to yield better decision making via use cases, such as quantifying uncertainties of dynamic simulations and stochastic dynamic security assessment.


  • Developed a novel model reduction approach using the discrete empirical interpolation method enhanced proper orthogonal decomposition (DEIM-POD). DEIM-POD introduces an optimal selection procedure for the observation points in the state space so the projection error of the nonlinear functions is minimized. DEIM-POD provides a great solution for dynamic load model reduction because it not only effectively reduces the number of nonlinear function evaluations but also identifies the critical load locations where the parameters are important for the accuracy of dynamic load reduced-order models.

  • Developed a generalized polynomial chaos expansion framework that can effectively quantify the input uncertainties on dynamic system responses. It can deal with the correlations among input uncertainties and allows the project team to investigate the impacts of correlated unknown inputs on model responses. Conducted experiments on the IEEE 39-bus system considering the integration of Photovoltaics (PVs). Loads and PVs are assumed to follow Gaussian distributions with means being the original values and standard derivations being a certain portion (uncertainty level) of means. The robustness of the proposed method to different uncertain distributions was also tested.


  • N. Duan, J. Zhao, X. Chen, B. Wang, S. Wang, “Discrete Empirical Interpolation Method Based Dynamic Load Model Reduction,” IEEE PES General Meeting 2021.

  • K. Ye, J. Zhao, N. Duan, “Quantifying Impacts of Load and Correlated PV Uncertainties on Power System Dynamics” to be submitted.

Principal Investigator: duan4 [at] (Nan Duan )

Parallel in Time Algorithms for Solving Transient Stability Simulations for Power Systems


This project will develop multigrid reduction in time (MGRIT) methods for parallel acceleration of time dependent transmission grid simulations with discontinuities. The project is addressing problem formulation, software implementation, and performance assessments.

Parallel in time multi-grid in time v-cycle integration computer renderings


Sequential time stepping in transmission systems can pose a bottleneck for simulations in which a larger number of time steps is required, especially for renewable systems with longer time horizons (e.g., wind ramping). In addition, computer systems now advance speed through concurrency rather than processor clock rates. Simulations can be accelerated through use of concurrency, although parallelizing over buses often leads to inefficient simulations due to poor linear solvers. Moreover, a full restructuring of the code is often needed to incorporate such concurrency.


This project will investigate computationally efficient solutions for long-term dynamic simulations through incorporation of computational parallelism and development of a novel multigrid reduction in time (MGRIT) integration scheme for transmission power grid simulation. The main goal of the project is to provide significant speedups to these simulations. A major research objective of the project is to develop effective approaches for handling discontinuities in the system.


The MGRIT method pursued in this project makes use of current simulation time integrators by wrapping them into a multilevel time integration approach. The method leverages significant advances in parallel multilevel algorithms that have been developed in both theory and software over the last two decades. The MGRIT approach has been matured in the XBraid software developed at LLNL, which was designed to use existing time integration code and thus requires only modest code modification for its use. MGRIT has mostly been applied to problems with continuous behaviors, so a key part of this project’s work is adapting the MGRIT method to handling discontinuities in power grid systems, such as occur in equipment limit adjustments or load changes. Methods must be tested in software on high concurrency machines to understand potential benefits and weaknesses, so the approach also includes implementation of the MGRIT algorithms and capabilities into the GridDyn transmission code at LLNL and assessment of performance on LLNL’s high concurrency systems.


An important component of this project has been the question of handling discontinuities. The project team completed an initial integration of the MGRIT algorithm into GridDyn, and within GridDyn formulated both fixed and variable step MGRIT methods for handling systems with scheduled discontinuities. The team applied these methods to a version of the reduced WECC system with a discontinuous square load applied every two seconds and demonstrated uniform convergence of the MGRIT algorithm independent of the number of discontinuities. The largest problem had 460 discontinuities yet MGRIT’s coarsest grid only had four time points. We observed speedups of up to 53x with uniform time steps and up to 47x speedup with variable step methods. Convergence was insensitive to the load size, which ranged from 50MW to 200MW.

The project team recently extended its work to address problems with unscheduled events. In serial, the locations of these events are computed in a sequential procedural way, which does not immediately translate to the parallel-in-time setting where they must be computed simultaneously. The team’s approach incorporates root finding into the parallel algorithm to locate the events, and it combines this with temporal adaptivity to increase simulation accuracy. The team tested the new algorithm on a simple exciter model problem and demonstrated that it exhibits uniform convergence independent of the number of events. The team also demonstrated speedups of up to 15x.

Sdirk Speedup data graph


  • S Günther, R. D. Falgout, P. Top, C. S. Woodward, and J. B. Schroder, “Parallel-in-Time Solution of Power System with Unscheduled Events,” IEEE Power and Energy Society General Meeting (PESGM), 2020, in revision.

  • M. Lecouvez, R D. Falgout, C S. Woodward, “A Parallel-in-time Algorithm for Variable Step Multistep Methods, Journal of Computational Science, 37:101029, 2019.

  • J. B. Schroder, M. Lecouvez, R. D. Falgout, C. S. Woodward, and P. Top, “Parallel-in-Time Solution of Power Systems with Scheduled Events,” IEEE Power and Energy Society General Meeting (PESGM), pp. 1–5, 2018.

  • M. Lecouvez, R. D. Falgout, C. S. Woodward, and P. Top, “A Parallel Multigrid Reduction in Time Method for Power Systems,” IEEE Power and Energy Society General Meeting (PESGM), pp. 1–5, 2016.

Principal Investigator: falgout2 [at] (Rob Falgout)

Probabilistic Impact Scenarios for Extreme Weather Events


Extreme weather impacts on power grid components are inherently probabilistic. Thus, there is a need to develop probabilistic models of extreme weather events (SNL), evaluate their use in power systems operations (LLNL), and propose enhanced operations strategies (LLNL).

This project is a collaboration with SNL (project lead), who is focused on developing probabilistic models of extreme weather events. LLNL is tasked with evaluation of those scenarios in power systems operations problems, and developing enhanced operations strategies given the projections.


  • Develop probabilistic models of extreme weather event impacts on the electric power grid

  • Extend existing power systems operations models to incorporate probabilistic extreme weather impact models

  • Develop scalable solvers for these enhanced probabilistic weather impact models

  • Conduct analyses of the benefits of the new probabilistic operations models


Project effort is organized around the following tasks:

  • Work with SNL to develop specification for output of probabilistic extreme weather event impacts on the power grid

  • Extend commitment and dispatch optimization models (available in the EGRET open source optimization library for power systems) to incorporate probabilistic extreme weather scenarios

  • Develop scalable solvers for these extended probabilistic power systems operations models (in the context of the mpi-sppy open source library for stochastic optimization)

  • Conduct analyses (with SNL) demonstrating the benefits of the extended operations models relative to existing practice


Principal Investigator: watson61 [at] (Jean-Paul Watson )

Resilient Expansion Planning under Long-Term Hazard Uncertainty


This project will develop models and solution algorithms for power system expansion planning that directly address key long-term hazards, including climate change, cyber, and cyber-physical threats from adversaries. The effort is distinct from traditional methods for capacity expansion, which target reliability—as opposed to resilience—targets at minimum cost.


Traditional power system expansion planning approaches involve a design goal of reliability—ensuring that future infrastructure can meet projected changing regional demands from population growth and policy mandates. However, there are greater uncertainties than just demand and technology characteristics when considering resilience.

  • The nature of natural and adversary-initiated disasters will not remain static over the next few decades and the space of credible threats to the power system will change as extreme weather events become more frequent, nation-state threats shift, generation fleet mixes transform, and regional demand profiles evolve to meet new weather patterns and population movement.

  • Therefore, it is critical that investment decisions made now aim to both improve long-term resilience in this changing threat landscape and account for the expected costs of future grid disruptions when planning for resilient and low-cost infrastructure into the future.

long term hazards inform expansion planning to acheive target resilience goals image


  • Specify models for resilient power systems capacity expansion, specifically considering the long-term behavior of key threat classes.

  • Develop scalable solvers for resilient power systems capacity expansion, based on robust (bi-level) optimization and stochastic programming.


  • Model uncertainties involving long-term power grid hazards, including climate change, extreme weather (e.g., wildfire and heat waves), cyber-physical threats from nation-state actors, and price spikes induced by geo-political events.

  • Develop optimization algorithms for power systems capacity expansion, to obtain minimal-cost defensive investments to modeled long-term hazards.

  • Publish technical reports and journal papers describing test problems, developed expansion planning algorithms, and experimental results.

  • Release all test problems and codes as open source.


  • Surveyed key recent advances in defender-attacker-defender optimization models and solvers, which will serve as the basis for the project algorithm development effort.

  • Implemented state-of-the-art defender-attacker-defender algorithm to determine optimal investments to harden against intentional threats.

  • Initiated extension of defender-attacker-defender model to directly address uncertainties associated with long-term hazards.


Principal Investigator: watson61 [at] (Jean-Paul Watson)

Optimal Restoration Planning Under Uncertainty with Compromised Communications Systems (BlackOps)


This project is developing scalable algorithms with quality guarantees for optimizing black-start and restoration sequences.

Stochastic grid optimization chart


Power system operation practices are undergoing a major transformation following the integration of renewable energy resources and information technologies into the grid. This transformation, alongside the advent of increasingly frequent extreme weather events, has made power system operations vulnerable to large forecast errors, cyberattacks, and natural disasters. These events can result in large-scale blackouts, causing billions of dollars in damages and posing serious threats to health and public safety, among other consequences. This project seeks to answer one of the most critical questions under those circumstances: how can we restore power as fast as possible?


The project aims to develop predictable decision tools for the worst-case scenario, in which a major blackout has taken place and operators need to restore service. These decision tools will have the potential to improve power system resilience by increasing the effectiveness of restoration operations, indicating investment needs in critical infrastructure, and serving as basis for vulnerability detection tools.


The project team proposes mixed-integer optimization models for power system blackstart and the development of specialized branch-and-cut programming algorithms for solving such modes with quality guarantees. The team will implement these algorithms in high-performance computing environments and test them against real-size instances of power systems in blackout conditions, thereby ensuring their scalability and applicability in real blackout scenarios.


  • Proposed models for optimal power system restoration and black start allocation (deterministic and stochastic) and studied the theoretical properties of different formulations, finding equivalences and dominant models.

  • Developed robust approximations and relaxations of the power flow equations and a specialized branch-and-cut algorithm, based on the previous developments, which is capable of finding an optimal restoration sequence from a complete blackout for real power grids with up to 2000 buses, a two orders of magnitude improvement with respect to the state-of-the-art.

  • Extended the previous models and algorithms to account for the cyber-physical interactions between the power grid and its communications and control network.

  • Current work focuses on developing optimal policy functions that can be used in real time blackstart and restoration while accounting for the unobservability of the state (operational or damaged) of power grid components following an extreme weather event or adversarial attack.

file illinois 200 scene baseline
illinois 200 scene


  • G. Patsakis, I. Aravena, D. Rajan, S. Oren, "Formulations and Valid Inequalities for Optimal Black Start Allocation in Power Systems," Optimization Online, July 2020.

  • G. Patsakis, D. Rajan, I. Aravena, J. Rios and S.S. Oren, "Optimal Black Start Allocation for Power System Restoration," IEEE Transactions on Power Systems, vol. 33, no. 6, pp. 6766-6776, May 2018.

  • G. Patsakis, D. Rajan, I. Aravena and S.S. Oren. Strong mixed-integer formulations for power system islanding and restoration. IEEE Transactions on Power Systems, vol. 34, no. 6, pp. 4880-4888, Nov. 2019.

  • Aravena, D. Rajan and G. Patsakis, "Mixed-Integer Linear Approximations of AC Power Flow Equations for Systems Under Abnormal Operating Conditions," In 2018 IEEE Power & Energy Society General Meeting (PESGM), Portland, OR, August 2018.

  • G. Patsakis, I. Aravena and D. Rajan, "A Stochastic Program for Black Start Allocation," 52nd Annual Hawaii International Conference on System Sciences (HICSS’18), Wailea, Hawaii, January 2019.

  • G. Patsakis, "Optimal Black Start Allocation for Power System Restoration," INFORMS Optimization 2018, Denver, CO, March 24th, 2018.

  • I. Aravena, "Power System Restoration through Mixed Integer Linear Programming," FERC Software Conference 2018, Washington, DC, June 27th, 2018.

  • I. Aravena, "Optimizing Power System Restoration Using Mixed Integer Linear Programming," INFORMS Annual Meeting 2018, Phoenix, AZ, USA, November 4th, 2018.

  • G. Patsakis, "Strong Mixed-Integer Formulations for Power System Islanding and Restoration," INFORMS Annual Meeting 2018, Phoenix, AZ, November 6th, 2018.

  • Musselman, "A Scalable Mixed-Integer Decomposition Approach for Optimal Black Start and Restoration of Power Systems," FERC Software Conference 2018, Washington, DC, June 27th, 2019.

  • Musselman, "Optimal Power System Black-Start Planning With Damaged Communications," INFORMS Annual Meeting 2019, Seattle, WA, October 20th, 2019.

  • I. Aravena, D. Rajan, G. Patsakis, S.S. Oren and J. Rios. "A Scalable Mixed-Integer Decomposition Approach for Optimal Power System Restoration," Optimization Online, January 2019.

Principal Investigator: aravenasolis1 [at] (Ignacio Aravena Solis)

Implementing Coupled T&D Simulation


This project instantiates T&D co-simulation using Eaton’s commercial distribution planning software CYMDIST and LLNL’s open-source transmission grid simulator GridDyn in HELICS co-simulation environment while leveraging HPC. The project is funded by DOE’s Technology Commercialization Funding.

TCF TD Co-simulation map


Coupling between transmission and distribution grids is increasing due to the penetration of renewable generation and distributed energy resources (DER). The traditional approaches to analyzing power grids separately are not sufficient because of a tighter coupling between the grids. Accounting for the coupling requires efficient co-simulation. Industry currently lacks a commercial tool capable of coupled T&D simulation. This project aims to implement T&D co-simulation using CYMDIST commercial software in collaboration with Eaton Inc., while leveraging HELICS, GridDyn and LLNL’s HPC.

TCF TD Co-Silumation map


The objective of this project is to build a transmission and distribution grid co-simulation capability using a commercial distribution grid planning tool CYMDIST (by Eaton Inc.), along with the open-source co-simulation framework HELICS, transmission grid simulator GridDyn, and parallel computing, and to facilitate transition to the user base through a commercialization study. Specifically, this project will:

  • Develop a T&D co-simulation solution using a commercial grid planning tool CYMDIST

  • Facilitate the transition of the technology from LLNL to industry through a commercialization and product launch study


The project aims to first enable the execution to CYMDIST in an HPC environment and then integrate it into the HELICS platform for T&D co-simulation. The approach consists of four tasks.

  • HPC-enabled CYMDIST: Enable parallel execution of CYMDIST on LLNL’s HPC to obtain 100x speedup in a power flow analysis. The project team used WINE to create a Windows-like environment to install and execute CYMDIST on the Linux environment of HPC. One hundred scenarios of CYMDIST power flow were executed in parallel on HPC compute nodes.

  • Co-simulation: Integrate CYMDIST distribution grid power flow simulation into HELICS. CYMDIST and GridDyn were integrated into HELICS to perform time series power flow co-simulation.

  • Use case: Perform a case study to demonstrate the benefit of co-simulation. The team is working on a hosting capacity use case using the WECC bulk grid network and 100 California prototypical feeders that run in parallel on HPC. The prototypical feeders were obtained from NREL’s Open-DS project and the DiTTo tool for OpenDSS to CYMDIST model translation.

  • Commercialization Study: Commercialization and technology transfer study led by Eaton, Inc.

The T&D model consisted of up to 100 distribution feeders (CYMDIST) that are connected to the IEEE 179-bus transmission network representing WECC network (GridDyn). The work was focused on the T&D co-simulation proof of concept implementation in HELICS. The power flow in co-simulation was limited to a single time snapshot. During this performance period, the test case was scaled up to 100 distribution feeders in time series power flow co-simulation. NREL’s Smart-DS project has made available San Francisco bay area distribution feeders. By using diTTo tool, the OpenDSS model is converted to CYME model and is integrated to the T&D co-simulation.


  • This project is arguably a first effort to integrate a commercial distribution grid planning software (CYMDIST) into the HELICS co-simulation platform and successfully demonstrate the speedup benefit for coupled T&D simulations.

  • The project installed CYMDIST distribution planning software on LLNL’s HPC. Its power flow simulation speedup is demonstrated when 100 distribution circuits are used for distribution grid hosting capacity analysis. A speedup behavior reaching up to 100x is observed.

  • The project integrated CYMDIST into the HELICS co-simulation platform and a coupled T&D time series power flow simulation is demonstrated along with transmission simulation tool GridDyn. It is tested using WECC 179-bus transmission network and 100 standard distribution feeder networks. A theoretical upper bound for the speedup behavior is derived and compared with the test case results.

  • The project demonstrated a T&D hosting capacity analysis use case using WECC 179-bus transmission network and 100 prototypical California distribution feeder models. An overall performance speedup of 92.42 was observed when compared to an estimated sequential solution due to the use of HPC. A T&D grid wide hosting capacity analysis capability will provide utilities a practical tool to address challenges with scaling up the analysis and go beyond distribution systems only to also identify transmission grid impacts of DER integrated on distribution networks.

  • Eaton Inc. is performing a commercialization study to identify expected benefits to its customers.

  • The project team carried out various technology transfer activities during the project.


  • N. Duan, C. Huang, C.-C. Sun, V. Donde, “Parallel Hosting Capacity Analysis for Integrated Transmission and Distribution Planning,” IEEE PES General Meeting, 2021. (Best Paper Award)

  • N. Duan, C.-C. Sun, R. Mast, P. Sotorrio, V. Donde, W. Ren, I. Alvarez-Fernandez, “Parallel Transmission Distribution Co-Simulation Leveraging a Commercial Distribution Simulator”, IEEE PES ISGT Europe 2020.

  • Vaibhav Donde, “Implementing Coupled T&D Simulations,” Poster for National Lab Day on the Hill, August 2019.

  • Vaibhav Donde, Wei Ren, “HPC-Enabled Distribution Planning Tool,” panel presentation in IEEE PES ISGT conference panel, February 2018.

Principal Investigator: donde1 [at] (Vaibhav Donde)

Uncertainty Quantification for Grid Planning


This project advances the state of the art of quantitative risk assessment and uncertainty quantification (UQ) methods for operations, control, and planning in complex settings—i.e., under stochasticity (encompassing high-impact, low-probability (HILP) extreme events), high dimensionality, and steady-state and dynamic regimes—in order to provide solutions for large-scale power grids in (near-)real-time and with sufficient accuracy.

sunset with windmills, solar panels, and high wires


Faced with many independent and correlated system conditions, variables, parameters, and events, the probabilistic risk and reliability evaluation of a bulk power grid becomes quite involved and very complex. A particularly daunting task in this process is the capturing of low-probability events (e.g., N-k contingency and extreme weather events) that have potentially disastrous consequences, which entails an impossibly large number of samples. Thus, efficient quantification of risks brought by a wide array of uncertainties will enable enhanced planning and preparedness, and prioritization of investments for critical grid assets.


Develop novel algorithmic capabilities and analysis tools to arrive at credible and timely risk-informed, pdf*-provided decisions and to determine critical boundaries of the complex power grids subject to various uncertainties in system conditions, parameters, and high-impact rare events.

Facilitate robust and efficient risk, reliability, and security assessment to ensure safe and stable grid operation.


  • An enhanced polynomial-based Markov chain Monte Carlo (MCMC) by proposing a gradient-enabled polynomial-based MCMC to near-real-time inversion

  • Hybrid polynomial-based surrogates with a high-fidelity model for fast and accurate detection of rare events with high consequence and low probability

  • Multiple, low-fidelity surrogates to evaluate a large number of samples associated with high-probability (i.e., N-0 and N-1) topologies to screen out the key areas at the risk boundary

  • A hybrid procedure that employs high-fidelity, AC power-flow model to efficiently capture a very small portion of the probability space along the tails of the pdf of a quantity of interest (QoI) to fine-tune the estimation results

  • A nonparametric, reduced-order system representation using data-driven surrogate models based on Gaussian-process emulation and vine copula to

    • accelerate probabilistic power-flow analysis,

    • quantify the uncertainty in stochastic economic dispatch, and

    • realize a fast and an accurate probabilistic load-margin assessment under high-dimensional, spatiotemporally correlated uncertainty of wind power generation


  • Dynamic parameter estimation using a polynomial-based and gradient-enhanced surrogate model with a computational gain of two to three orders of magnitude (OOM) over the standard Monte-Carlo (MC) sampling approach

  • Risk assessment of power grids through multi-surrogates for nonlinear and stochastic power-flow analysis under topology changes caused by random branch outages and in the presence of load uncertainties

    • First attempt at using hybridized surrogates to detect low-probability rare events along a long-tail pdf of the selected quantities of interest (e.g., bus voltages and branch flows)

    • Accurate detection of very small failure probabilities (e.g., 10-3 or 10-4) within a few minutes

  • Probabilistic load-margin assessment under high-dimensional correlated wind power uncertainty by way of Gaussian processes and vine copula enabling more than two-OOM faster computation over the standard MC method


  • Y. Xu, M. Korkali, L. Mili, X. Chen, and L. Min, “Risk Assessment of Rare Events in Probabilistic Power Flow via Hybrid Multi-Surrogate Method,” IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1593–1603, Mar. 2020.

  • Y. Xu, M. Korkali, L. Mili, and X. Chen, “An Efficient Multifidelity Model for Assessing Risk Probabilities in Power Systems under Rare Events,” Proceedings of the 53rd Hawaii International Conference on System Sciences (HICSS) 2020, Jan. 7–10, 2020, Maui, HI.

  • Y. Xu, C. Huang, X. Chen, L. Mili, C. H. Tong, M. Korkali, and L. Min, “Response-Surface-Based Bayesian Inference for Power System Dynamic Parameter Estimation,” IEEE Transactions on Smart Grid, vol. 10, no. 6, pp. 5899-5909, Nov. 2019.

  • Y. Xu, L. Mili, X. Chen, M. Korkali, and L. Min, “A Bayesian Approach to Real-Time Dynamic Parameter Estimation Using PMU Measurement,” IEEE Transactions on Power Systems, vol. 35, no. 2, pp. 1109–1119, Mar. 2020.

  • Y. Xu, L. Mili, M. Korkali, and X. Chen, “An Adaptive Bayesian Parameter Estimation of a Synchronous Generator under Gross Errors,” IEEE Transactions on Industrial Informatics, vol. 16, no. 8, pp. 5088–5098, Aug. 2020.

  • Y. Xu, X. Chen, L. Mili, C. Huang, and M. Korkali, “Polynomial-Chaos-Based Decentralized Dynamic Parameter Estimation Using Langevin MCMC,” 2019 IEEE Power and Energy Society (PES) General Meeting (PESGM), Aug. 4–8, 2019, Atlanta, GA. [Best Paper Award]

  • Z. Hu, Y. Xu, M. Korkali, X. Chen, L. Mili, and C. H. Tong, “Uncertainty Quantification in Stochastic Economic Dispatch using Gaussian Process Emulation,” Innovative Smart Grid Technologies (ISGT) 2020, Feb. 17–20, 2020, Washington, D.C.

  • Y. Xu, Z. Hu, L. Mili, M. Korkali, and X. Chen, “Probabilistic Power Flow Based on a Gaussian Process Emulator,” IEEE Transactions on Power Systems, vol. 35, no. 4, pp. 3278–3281, July 2020.

  • Y. Xu, K. Karra, L. Mili, M. Korkali, X. Chen, and Z. Hu, “Probabilistic Load-Margin Assessment using Vine Copula and Gaussian Process Emulation,” 2020 IEEE PESGM, Aug. 2–6, 2020, Montreal, Canada.

  • Y. Xu, L. Mili, M. Korkali, K. Karra, Z. Zheng, and X. Chen, “A Data-Driven Nonparametric Approach for Probabilistic Load-Margin Assessment considering Wind Power Penetration,” IEEE Transactions on Power Systems, vol. 35, no. 6, pp. 4756–4767, Nov. 2020.

  • Z. Hu, Y. Xu, M. Korkali, X. Chen, L. Mili, J. Valinejad, and C. H. Tong, “A Bayesian Approach for Data-Driven Uncertainty Assessment in Stochastic Economic Dispatch considering Wind Power Penetration,” IEEE Transactions on Sustainable Energy, vol. 12, no. 1, pp. 671–681, Jan. 2021.

Principal Investigator: korkali1 [at] (Mert Korkali )

Vaibhav Donde

Vaibhav Donde

Associate Program Lead, Energy Infrastructure


  donde1 [at] (donde1[at]llnl[dot]gov)