Copyright 2023-2026 Udo Schmitz
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
file: docs/lit/workshoptourcgmes.jl purpose: Literate.jl source of the foreign-formats tour: the network formats of other tools that Sparlectra reads. CGMES (what an ENTSO-E delivery is, reading and analyzing it, bus-branch and node-breaker imports with and without a TP profile, #314, SV validation, the export round trip), PowSyBl IIDM (the same network as pypowsybl wrote it, compared with OpenLoadFlow) and the legacy DTF deck (FOR001 with its outage records). Runs on the official conformity test sets, fetched on demand, plus the shipped PowSyBl and DTF files.
The Sparlectra workshop tour: foreign formats
Note: This workshop was created with AI assistance and is reviewed and curated by the maintainer; it is not a fully machine-generated text.
Level: Expert. You should be comfortable importing and solving networks (basic tour); no prior knowledge of the formats is assumed, building that knowledge is the point of this tour.
Sparlectra has formats of its own: the Sparlectra Case Format (.scf.json, a power-grid-model dataset with a Sparlectra block), the plain power-grid-model JSON inside it, and MATPOWER, the lingua franca of the basic tour. Everything else is a FOREIGN format: the exchange format of another tool, with that tool's model of a network, its conventions and its blind spots. Three of them are read natively, and this tour takes each one apart:
- CGMES (Common Grid Model Exchange Standard), how European TSOs exchange grid models: a DELIVERY of several RDF/XML files, each carrying one PROFILE of the same network. Chapters 1 to 5 work on the official ENTSO-E conformity test sets, downloaded on demand (about 22 MB, once).
- PowSyBl IIDM (
.xiidm), the XML network model of the LF Energy grid framework, with node-breaker substations, tap changers and limits; chapter 6 reads a file that pypowsybl wrote and compares the result with OpenLoadFlow. - DTF, the fixed-column input deck (FOR001) of a legacy load-flow program, with its own outage records; chapter 7 reads the shipped demo deck and applies an outage from it.
- CGMES: anatomy of a delivery, profiles, and what a summary shows
- CGMES: when an import cannot work, the analysis report
- CGMES: bus-branch import, solve, and validation against the shipped state
- CGMES: node-breaker, with a TP profile and without one (the topology processor)
- CGMES: the export round trip
- PowSyBl IIDM: the same network as pypowsybl delivers it, compared with OpenLoadFlow
- DTF: a legacy input deck, its cards, and an outage from its records
Warm-up and the test data
Three data sources, one per format. The ENTSO-E conformity package bundles reference networks in several variants; ensureCGMESTestConfigurations downloads and extracts it once into a local cache and returns the extraction root (chapters 1 to 5). The PowSyBl file and the DTF demo deck ship with Sparlectra under data/powsybl and data/dtf_demo (chapters 6 and 7), nothing to fetch.
This cell defines what the chapters share (the using clause and all data paths: the conformity root with its download, the MicroGrid and MiniGrid deliveries, the PowSyBl file with its OpenLoadFlow reference and the DTF deck), so any chapter can run on its own after it; a code cell that uses a name from another cell says so in a comment. Julia compiles each function on first use. The cell's last line, warmup(), runs every path the chapters exercise once, output discarded (the first power-flow solve on a two-bus net, the CGMES summary, analysis, import with and without TP, the result tables, the SV comparison and the export, the IIDM reader with its OpenLoadFlow-like run, and the DTF deck with an outage), so the timing of the MicroGrid solve in Chapter 3 is the solve, not the compiler; its code is in docs/lit/warmup/workshop_tour_cgmes.jl of the package. How long the warm-up takes depends on the machine: a Colab session is several times slower than a desktop; every later cell then runs without compile pauses.
using Sparlectra
root = Sparlectra.CGMESImporter.ensureCGMESTestConfigurations()
println("conformity sets under: ", root)
# the two study deliveries of this tour
microgrid_be = joinpath(root, "MicroGrid", "BaseCase_BC", "CGMES_v2.4.15_MicroGridTestConfiguration_BC_BE_v2")
microgrid_bd = joinpath(root, "MicroGrid", "BaseCase_BC", "CGMES_v2.4.15_MicroGridTestConfiguration_BD_v2")
minigrid_nb = joinpath(root, "MiniGrid", "NodeBreaker", "CGMES_v2.4.15_MiniGridTestConfiguration_BaseCase_Complete_v3")
minigrid_bd = joinpath(root, "MiniGrid", "NodeBreaker", "CGMES_v2.4.15_MiniGridTestConfiguration_Boundary_v3")
# the shipped files of chapters 6 and 7
powsybl_dir = joinpath(pkgdir(Sparlectra), "data", "powsybl")
# the solution OpenLoadFlow computed for that file, kept with the tests
powsybl_reference = joinpath(pkgdir(Sparlectra), "test", "fixtures", "powsybl", "micro_grid_be.powsybl", "reference_buses.csv")
dtf_deck = joinpath(pkgdir(Sparlectra), "data", "dtf_demo", "sp_dtf5.DAT")
println("PowSyBl file: ", joinpath(powsybl_dir, "micro_grid_be.xiidm"))
println("DTF deck: ", dtf_deck)
# compile every path the chapters use once; the warm-up code is in
# docs/lit/warmup/workshop_tour_cgmes.jl of the installed package
include(pkgdir(Sparlectra, "docs", "lit", "warmup", "workshop_tour_cgmes.jl"))
warmup()conformity sets under: /home/runner/work/Sparlectra.jl/Sparlectra.jl/data/CGMES/extracted
PowSyBl file: /home/runner/work/Sparlectra.jl/Sparlectra.jl/data/powsybl/micro_grid_be.xiidm
DTF deck: /home/runner/work/Sparlectra.jl/Sparlectra.jl/data/dtf_demo/sp_dtf5.DAT
warm-up: compiles the paths of this workshop once; how long it takes depends on the machine (a Colab session is several times slower than a desktop)
warm-up solve: first 0.0 s (compiles), second 0.12 ms
further paths : 4.83 s (CGMES summary, analysis, import, export, IIDM, DTF); everything warmChapter 1: anatomy of a delivery
Example 1.1: what is in the box. MicroGrid BE is the Belgian half of a two-country reference model, as a sketch:
NL area (boundary) ~~~~ X-nodes ~~~~ BE area (this delivery)
380 kV ring: 2 substations
220/110 kV under 2 transformers
gens, loads, shunt, PSTOne delivery = one folder (or ZIP) of profile files. The important ones:
- EQ (equipment): what exists, lines, transformers, machines, their physical parameters. The skeleton.
- SSH (steady-state hypothesis): the operating point, set points, switch states, in-service flags.
- TP (topology): the sender's bus partition,
TopologicalNodes and which terminal sits on which node. - SV (state variables): the sender's solved state, voltages and flows. Reference material, not input.
- Boundary set (EQBD/TPBD): the X-nodes where areas stitch together; shipped separately so neighbors agree on the seam.
summarizeCGMES reads a delivery without importing it:
# uses: microgrid_bd, microgrid_be (warm-up cell)
s = summarizeCGMES(path = [microgrid_be, microgrid_bd])
print(s)CGMES summary
version: 2.4.15
files: 9 (6 read, 3 skipped)
- 20171002T0930Z_1D_BE_SSH_2.xml [SSH] read
- 20171002T0930Z_1D_BE_SV_2.xml [SV] read
- 20171002T0930Z_1D_BE_TP_2.xml [TP] read
- 20171002T0930Z_BE_DL_2.xml [DL] skip: profile not in import set
- 20171002T0930Z_BE_DY_2.xml [DY] skip: profile not in import set
- 20171002T0930Z_BE_EQ_2.xml [EQ,EQ_SC] read
- 20171002T0930Z_BE_GL_2.xml [GL] skip: profile not in import set
- 20171002T0930Z_ENTSO-E_EQ_BD_2.xml [EQ_BD,EQ_BD_OP] read
- 20171002T0930Z_ENTSO-E_TP_BD_2.xml [TP_BD] read
objects: 324 in 35 classes
CurrentLimit 100
Terminal 45
OperationalLimitSet 23
Line 13
SvPowerFlow 12
TopologicalNode 12
SvVoltage 11
BusbarSection 9
PowerTransformerEnd 9
OperationalLimitType 8
BaseVoltage 7
ACLineSegment 7
EnergySchedulingType 6
VoltageLevel 6
ConnectivityNode 6
… 20 more classes
unresolved references: 0Reading aid (Example 1.1): the summary names the files with their profiles, the CGMES version, object counts per class, and the declared model dependencies. Nothing is mapped yet; this is the "what did I just receive" view.
Chapter 2: when an import cannot work
Example 2.1: the missing boundary set. Deliveries reference objects they do not define, most prominently the boundary X-nodes. Import the BE files WITHOUT the boundary set and the import aborts; analyzeCGMES explains the gap in plain language instead of a bare error:
# uses: microgrid_be (warm-up cell)
a_report = analyzeCGMES(path = microgrid_be)
print(a_report)CGMES import analysis
=====================
Supplied models:
- 20171002T0930Z_1D_BE_SSH_2.xml: SSH v2.4.15, 70 objects, model 52b712d1-f3b0-4a59-9191-79f2fb1e4c4e
- 20171002T0930Z_1D_BE_SV_2.xml: SV v2.4.15, 30 objects, model c2960b34-0a04-4cd1-9c4d-f3112d85ec6c
- 20171002T0930Z_1D_BE_TP_2.xml: TP v2.4.15, 50 objects, model f2f43818-09c8-4252-9611-7af80c398d20
- 20171002T0930Z_BE_DL_2.xml: DL v2.4.15, 0 objects, model 0eb1540d-5ca3-40ab-9bf7-c91ec4c5d7c3 [skipped: profile not in import set]
- 20171002T0930Z_BE_DY_2.xml: DY v2.4.15, 0 objects, model 239caf0a-38bf-468b-a4d8-15f619886c58 [skipped: profile not in import set]
- 20171002T0930Z_BE_EQ_2.xml: EQ+EQ_SC v2.4.15, 256 objects, model d400c631-75a0-4c30-8aed-832b0d282e73
- 20171002T0930Z_BE_GL_2.xml: GL v2.4.15, 0 objects, model 9e383122-10cf-4da3-9785-f404b3541588 [skipped: profile not in import set]
Declared dependencies MISSING from the input (md:Model.DependentOn):
- model 2399cbd0-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_TP_2.xml, 20171002T0930Z_BE_EQ_2.xml
- model 2399cbd1-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_SV_2.xml, 20171002T0930Z_1D_BE_TP_2.xml
Unresolved references: 38 total
- Terminal.TopologicalNode: 10
- ACLineSegment.BaseVoltage: 7
- SvVoltage.TopologicalNode: 5
- EquivalentInjection.BaseVoltage: 5
- PowerTransformerEnd.BaseVoltage: 4
- VoltageLevel.BaseVoltage: 3
- TopologicalNode.BaseVoltage: 3
- TopologicalIsland.AngleRefTopologicalNode: 1
Verdict: the delivery declares 2 prerequisite model(s) that are not part of the input.
In real ENTSO-E deliveries these are typically the boundary files (EQ_BD/TP_BD) of the
matching date — the boundary carries the border X-nodes and the shared BaseVoltage catalog.
Obtain the matching boundary set and pass it as an additional path.
CGMES import analysis
=====================
Supplied models:
- 20171002T0930Z_1D_BE_SSH_2.xml: SSH v2.4.15, 70 objects, model 52b712d1-f3b0-4a59-9191-79f2fb1e4c4e
- 20171002T0930Z_1D_BE_SV_2.xml: SV v2.4.15, 30 objects, model c2960b34-0a04-4cd1-9c4d-f3112d85ec6c
- 20171002T0930Z_1D_BE_TP_2.xml: TP v2.4.15, 50 objects, model f2f43818-09c8-4252-9611-7af80c398d20
- 20171002T0930Z_BE_DL_2.xml: DL v2.4.15, 0 objects, model 0eb1540d-5ca3-40ab-9bf7-c91ec4c5d7c3 [skipped: profile not in import set]
- 20171002T0930Z_BE_DY_2.xml: DY v2.4.15, 0 objects, model 239caf0a-38bf-468b-a4d8-15f619886c58 [skipped: profile not in import set]
- 20171002T0930Z_BE_EQ_2.xml: EQ+EQ_SC v2.4.15, 256 objects, model d400c631-75a0-4c30-8aed-832b0d282e73
- 20171002T0930Z_BE_GL_2.xml: GL v2.4.15, 0 objects, model 9e383122-10cf-4da3-9785-f404b3541588 [skipped: profile not in import set]
Declared dependencies MISSING from the input (md:Model.DependentOn):
- model 2399cbd0-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_TP_2.xml, 20171002T0930Z_BE_EQ_2.xml
- model 2399cbd1-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_SV_2.xml, 20171002T0930Z_1D_BE_TP_2.xml
Unresolved references: 38 total
- Terminal.TopologicalNode: 10
- ACLineSegment.BaseVoltage: 7
- SvVoltage.TopologicalNode: 5
- EquivalentInjection.BaseVoltage: 5
- PowerTransformerEnd.BaseVoltage: 4
- VoltageLevel.BaseVoltage: 3
- TopologicalNode.BaseVoltage: 3
- TopologicalIsland.AngleRefTopologicalNode: 1
Verdict: the delivery declares 2 prerequisite model(s) that are not part of the input.
In real ENTSO-E deliveries these are typically the boundary files (EQ_BD/TP_BD) of the
matching date — the boundary carries the border X-nodes and the shared BaseVoltage catalog.
Obtain the matching boundary set and pass it as an additional path.Reading aid (Example 2.1): the analysis lists the supplied models, the declared dependencies (md:Model.DependentOn), which of them are missing from the input, and a histogram of unresolved references. The missing boundary set is named BY MODEL ID: exactly the file to ask the sender for. The same report lands in cgmes.log when a Web UI or service import aborts.
Chapter 3: bus-branch import, solve, validate
Example 3.1: the working import. With the boundary set the BE model (network of Example 1.1) imports into an ordinary Net; everything the other workshops do applies from here:
# uses: microgrid_bd, microgrid_be (warm-up cell)
res = importCGMES(path = [microgrid_be, microgrid_bd], name = "microgrid_be")
net = res.net
println("imported: ", length(net.nodeVec), " buses, ", length(net.branchVec), " branches, ", length(net.prosumpsVec), " injections")
etime = @elapsed ((ite, erg) = runpf!(net, 30, 1e-8, 0; islands_enabled = true))
println("solved: status ", erg, " in ", ite, " iterations")imported: 12 buses, 13 branches, 10 injections
solved: status 0 in 3 iterationsThe classical result tables, exactly as in the basic tour: calcNetLosses! derives branch flows and losses from the converged voltages, printACPFlowResults prints bus voltages and branch flows.
# uses: etime, ite, net (Example 3.1)
(@isdefined(etime) && @isdefined(ite) && @isdefined(net)) || error("Run Example 3.1 first: it sets up etime, ite, net.")
calcNetLosses!(net)
printACPFlowResults(net, etime, ite, 1e-8)================================================================================
| SPARLECTRA Version 0.50.0 - AC Power Flow Results |
================================================================================
Date : 19-Oct-26 18:2:0
Iterations : 3
Flatstart : No
Tolerance : 1.0e-08
Solver : NR
Total time : 0.000806 s
Case : microgrid_be
Cooldown iters : 1
Q-hysteresis : 0.0100 pu
Jacobian cond. : kappa1(J) = 35800.0, attainable accuracy ~ 7.9e-12, well conditioned (tol 1.0e-8 reachable)
BaseMVA : 100
Nodes : 12 (PV: 1 PQ: 10 (Aux: 1) Slack: 1
Grid connection: slack bus BE-Busbar_4
Branches : 13
Links : 0
HVDC links : 0
Lines : 2
Trafos : 11
Generators : 7
Loads : 3
Shunts : 2
Controllers : 0 (Tap: 0, Q(U): 0, P(U): 0)
PV→PQ locks : 0
PV→PQ events : 0
total network power balance (Σ S_branch): P = 40.532 [MW], Q = -24.091 [MVar]
==========================================================================================================================================================================================================================
| Nr | Bus | Vn [kV] | V [kV] | V [pu] | phi [deg] | Pg [MW] | Qg [MVar] | Pl [MW] | Ql [MVar] | Ps [MW] | Qs [MVar] | Type | Control | Tap Vm tgt |
==========================================================================================================================================================================================================================
| 1 | TN_Border_ST23 | 220.0 | 225.515 | 1.025 | -5.581 | 27.365 | -0.426 | | | | | PQ | - | |
| 2 | BE-Busbar_2 | 225.0 | 224.889 | 1.000 | -7.625 | | | 1.000 | | | | PQ | - | |
| 3 | BE_TR_BUS4 | 225.0 | 224.333 | 0.997 | -8.770 | | | 200.000 | 50.000 | | | PQ | - | |
| 4 | BE_TR_BUS2 | 380.0 | 413.020 | 1.087 | -6.781 | | | | | 1.194 | -59.023 | PQ | - | |
| 5 | TN_Border_AL11 | 400.0 | 411.182 | 1.028 | -6.589 | 46.817 | -79.194 | | | | | PQ | - | |
| 6 | TN_Border_MA11 | 400.0 | 412.642 | 1.032 | -6.748 | 43.687 | -84.877 | | | | | PQ | - | |
| 7 | TN_Border_GY11 | 400.0 | 410.913 | 1.027 | -6.577 | 90.037 | -148.604 | | | | | PQ | - | |
| 8 | TN_Border_ST24 | 220.0 | 224.984 | 1.023 | -5.639 | 26.805 | -1.490 | | | | | PQ | - | |
| 9 | BE-Busbar_6 | 110.0 | 115.508 | 1.050 | -9.392 | | | 200.000 | 90.000 | | -330.789 | PQ | - | |
| 10 | BE-Busbar_4 | 10.5 | 10.821 | 1.031 | -7.057 | 90.015 | -51.265 | | | | | SLACK | - | |
| 11 | AUX3WT_BE-TR3_1 | 380.0 | 412.690 | 1.086 | -7.354 | | | | | | | AUX | - | |
| 12 | BE-Busbar_5 | 21.0 | 21.987 | 1.047 | -6.650 | 118.000 | 91.952 | | | | | PV | - | |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
==========================================================================================================================================================================================================================
| Branch | Type | Connection | P [MW] | Q [MVar] | P [MW] | Q [MVar] | Pv [MW] | Qv [MVar] | Ctrl | P_tgt | TapPos | Ctrl status |
==========================================================================================================================================================================================================================
| B_2WT_220_1_2 | Trafo | TN_Border_ST23 -> BE-Bus… | 27.365 | -0.426 | -25.772 | -2.829 | 1.593 | -3.255 | - | - | - | - |
| B_ACL_225_3_2 | Line | BE_TR_BUS4 -> BE-Busbar_2 | -27.874 | -2.749 | 31.313 | 1.206 | 3.439 | -1.543 | - | - | - | - |
| B_2WT_380_4_5 | Trafo | BE_TR_BUS2 -> TN_Border_… | -36.589 | 54.182 | 46.817 | -79.194 | 10.228 | -25.012 | - | - | - | - |
| B_2WT_380_4_6 | Trafo | BE_TR_BUS2 -> TN_Border_… | -36.858 | 80.682 | 43.687 | -84.877 | 6.829 | -4.195 | - | - | - | - |
| B_2WT_380_4_7 | Trafo | BE_TR_BUS2 -> TN_Border_… | -82.839 | 138.449 | 90.037 | -148.604 | 7.198 | -10.154 | - | - | - | - |
| B_ACL_225_3_2 | Line | BE_TR_BUS4 -> BE-Busbar_2 | -11.221 | -1.076 | 17.296 | 0.353 | 6.075 | -0.723 | - | - | - | - |
| B_2WT_220_8_2 | Trafo | TN_Border_ST24 -> BE-Bus… | 26.805 | -1.490 | -23.837 | 1.270 | 2.968 | -0.220 | - | - | - | - |
| B_2WT_380_4_9 | Trafo | BE_TR_BUS2 -> BE-Busbar_6 | 55.868 | -217.509 | -55.067 | 221.807 | 0.802 | 4.298 | - | - | - | - |
| B_2WT_225_3_9 | Trafo | BE_TR_BUS4 -> BE-Busbar_6 | 55.302 | 39.251 | -55.233 | -38.308 | 0.070 | 0.943 | - | - | - | - |
| B_2WT_10_10_9 | Trafo | BE-Busbar_4 -> BE-Busbar… | 90.015 | -51.265 | -89.701 | 57.291 | 0.314 | 6.025 | - | - | - | - |
| B_2WT_380_11_4 | Trafo | AUX3WT_BE-TR3_1 -> BE_TR… | -99.172 | -2.639 | 99.224 | 3.218 | 0.052 | 0.579 | - | - | - | - |
| B_2WT_225_3_11 | Trafo | BE_TR_BUS4 -> AUX3WT_BE-… | -216.207 | -85.426 | 216.555 | 91.814 | 0.348 | 6.389 | - | - | - | - |
| B_2WT_380_11_12 | Trafo | AUX3WT_BE-TR3_1 -> BE-Bu… | -117.383 | -89.176 | 118.000 | 91.952 | 0.617 | 2.777 | - | - | - | - |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Control
-------
Transformer controls: none
Q-V characteristic: no non-physical generator states.The delivery also carries the sender's solved state (SV), which makes it its own reference: compareWithSV checks the re-solved voltages and flows against what the sender shipped:
# uses: res (Example 3.1)
@isdefined(res) || error("Run Example 3.1 first: it sets up res.")
cmp = compareWithSV(res)
println("compared ", cmp.n, " buses against the shipped SV: max |dVm| = ", round(cmp.max_dvm; sigdigits = 3), " pu, max |dVa| = ", round(cmp.max_dva; sigdigits = 3), " deg")compared 11 buses against the shipped SV: max |dVm| = 8.08e-5 pu, max |dVa| = 0.00103 degReading aid (Example 3.1): agreement to a few 1e-4 pu means the model mapped faithfully, the remaining difference is solver tolerance and rounding in the shipped SV. A LARGE deviation would point at a mapping or data problem; the per-bus table in the comparison names where.
Chapter 4: node-breaker, with and without TP
Example 4.1: node-breaker with the shipped TP. Node-breaker deliveries model every busbar, breaker, and disconnector explicitly; the bus partition is the result of aggregating connectivity across closed switches. Sketch of one MiniGrid substation:
busbar A ═══╤═══════╤═══ every ─[/]─ is a breaker or
[/] [/] disconnector with its own
busbar B ═══╧══╤════╧═══ SSH switch state; equipment
[/] connects through bays, not
line directly to a busbarAs long as the sender ships a TP profile, the importer simply consumes the sender's aggregation result:
# uses: minigrid_bd, minigrid_nb (warm-up cell)
res_tp = importCGMES(path = [minigrid_nb, minigrid_bd], name = "minigrid_nb")
println("with TP: ", length(res_tp.net.nodeVec), " buses, ", length(res_tp.net.branchVec), " branches")with TP: 15 buses, 19 branchesExample 4.2: node-breaker WITHOUT a TP profile. EMS and substation-level exports often ship EQ+SSH only, topology expressed as ConnectivityNodes plus switch states, and no topology processor ever ran. Since 0.9.16 Sparlectra derives the partition itself: connectivity nodes aggregate across closed switches (SSH open overriding EQ normalOpen, out-of-service counts as open), retained switches stay bus couplers, and the derived nodes feed the unchanged import pipeline. We simulate such a delivery by withholding the TP and SV files of Example 4.1's set:
# uses: minigrid_bd, minigrid_nb (warm-up cell); res_tp (Example 4.1)
@isdefined(res_tp) || error("Run Example 4.1 first: it sets up res_tp.")
files_no_tp = [f for f in readdir(minigrid_nb; join = true) if endswith(f, ".xml") && !occursin("_TP", basename(f)) && !occursin("_SV", basename(f))]
res_notp = importCGMES(path = vcat(files_no_tp, minigrid_bd), name = "minigrid_nb_no_tp")
println("without TP: ", length(res_notp.net.nodeVec), " buses, ", length(res_notp.net.branchVec), " branches")
for m in res_notp.messages
occursin("topology processor", m) && println(" ", m)
end
# the twins must describe the same network: solve both and compare
net_a = res_notp.net
net_b = res_tp.net
net_a.flatstart = true
net_b.flatstart = true
_, ea = runpf!(net_a, 40, 1e-8, 0; islands_enabled = true)
_, eb = runpf!(net_b, 40, 1e-8, 0; islands_enabled = true)
va = sort([n._vm_pu for n in net_a.nodeVec])
vb = sort([n._vm_pu for n in net_b.nodeVec])
println("both solve (", ea, "/", eb, "); max |Vm| difference across the sorted buses: ", maximum(abs.(va .- vb)))without TP: 15 buses, 19 branches
topology processor: derived 11 topological node(s) from 103 connectivity node(s) across 90 closed switch merge(s) (TP profile absent); 0 retained closed switch(es) kept as bus couplers, 0 open switch(es) split the graph, 2 boundary group(s) adopted their TP_BD node
both solve (0/0); max |Vm| difference across the sorted buses: 0.0Reading aid (Example 4.2): the derived partition reproduces the sender's TP bus for bus (up to naming: derived buses take their busbar names), and the solved voltages agree to numerical precision. The processor announces itself in the import messages and runs ONLY when no usable TP is present; TP-carrying deliveries take the unchanged path of Example 4.1. One honest caveat from the conformity sweep: a TP that assigns terminals of ONE connectivity node to DIFFERENT topological nodes (FullGrid's completeness set does this on a load node) cannot be derived from the connectivity graph; the processor then produces the graph-consistent partition and the affected injections sit one bus apart.
Chapter 5: the export round trip
Example 5.1: write it back out. The imported network (Example 3.1) exports as a complete CGMES 2.4.15 delivery (EQ+TP+SSH+SV) with roundtrip-stable identities: mRIDs recorded on import are reused, so renaming nothing changes nothing. The re-import of the export solves to the same power flow:
# uses: net (Example 3.1)
@isdefined(net) || error("Run Example 3.1 first: it sets up net.")
outdir = mktempdir()
files = writeCGMESFiles(net; path = outdir)
println(length(files), " profile files written")
res2 = importCGMES(path = outdir, name = "roundtrip")
ite2, erg2 = runpf!(res2.net, 30, 1e-8, 0; islands_enabled = true)
vm1 = sort([n._vm_pu for n in net.nodeVec])
vm2 = sort([n._vm_pu for n in res2.net.nodeVec])
println("re-import solves (status ", erg2, "); max |Vm| difference to the original: ", maximum(abs.(vm1 .- vm2)))4 profile files written
re-import solves (status 0); max |Vm| difference to the original: 2.220446049250313e-16Reading aid (Example 5.1): the export writes what the network IS, not what the original files said; the shipped SV of the export carries the CURRENT solved state, so a receiving tool starts from it. Details and the identity rules: CGMES Export.
Chapter 6: PowSyBl IIDM, the same network compared with OpenLoadFlow
What IIDM is. PowSyBl, the LF Energy grid framework, imports CGMES too and keeps its networks in its own XML model, IIDM (.xiidm). Where CGMES spreads one network over profiles and RDF references, IIDM is one document: substations with voltage levels, each level either a bus-breaker topology (named buses) or a node-breaker topology (numbered nodes joined by switches and internal connections), and the equipment attached to buses or nodes; tap changers with their step tables, operational limits, HVDC links and dangling lines sit right on the elements. What pypowsybl resolves before it hands out tables (the bus views of a node-breaker substation, the current tap step, the reactive limits at the setpoint), Sparlectra's reader resolves in Julia, so the file needs no Python and no conversion.
Example 6.1: an IIDM file. The MicroGrid BE of this tour ships with Sparlectra in that form under data/powsybl, as pypowsybl wrote it. The test suite keeps the solution OpenLoadFlow (PowSyBl's load flow) computed for it, reference_buses.csv, so the import can be checked against an independent solver:
# uses: powsybl_dir (warm-up cell)
using DelimitedFiles
tables = Sparlectra.read_iidm_tables(joinpath(powsybl_dir, "micro_grid_be.xiidm"))
println("IIDM ", tables.manifest["iidm_version"], ": ", length(tables.buses.id), " buses, ", length(tables.two_windings_transformers.id), " two-winding and ", length(tables.three_windings_transformers.id), " three-winding transformer(s), ", length(tables.dangling_lines.id), " dangling lines")IIDM 1_17: 6 buses, 3 two-winding and 1 three-winding transformer(s), 5 dangling linesThe generators of the MicroGrid regulate remote buses; OpenLoadFlow holds those buses at their targets, and remote_regulation = :remote attaches the same as outer-loop machine controls. The import report lists what was built, the slack decision and every notice:
# uses: tables (Example 6.1)
@isdefined(tables) || error("Run Example 6.1 first: it sets up tables.")
pnet, preport = build_net_from_powsybl(tables, PowsyblAdapterOptions(remote_regulation = :remote))
println(format_powsybl_report(preport))PowSyBl import
bus read 6 built 6 skipped 0
switch read 0 built 0 skipped 0
line read 2 built 2 skipped 0
2wt read 3 built 3 skipped 0
3wt read 1 built 1 skipped 0
generator read 2 built 2 skipped 0
load read 3 built 3 skipped 0
shunt read 2 built 2 skipped 0
svc read 0 built 0 skipped 0
dangling_line read 5 built 5 skipped 0
tie_line read 0 built 0 skipped 0
hvdc read 0 built 0 skipped 0
slack component 0: 550ebe0d-f2b2-48c1-991f-cebea43a21aa at bus f96d552a-618d-4d0c-a39a-2dea3c411dee (largest unit among locally regulating units)
notice: bus state of the file not used as the start: 2 tap changer(s) or shunt(s) were solved at another position than the one imported (solvedTapPosition, solvedSectionCount); the import starts flat
notice: load types: UNDEFINED 3
notice: generator 3a3b27be-b18b-4385-b557-6735d733baf0: remote voltage control of bus 5c74cb26-ce2f-40c6-951d-89091eb781b6 at 1.05 pu attached (outer loop)Reading aid (Example 6.1): the notice about the file's bus state is the reader being careful: the file was solved at other tap positions (solvedTapPosition) than the ones it carries as the current position, so that state belongs to another network and the import starts flat.
Example 6.2: the comparison. OpenLoadFlow's defaults are: every synchronous component solved, the active-power mismatch shared over the generators in proportion to max_p (the importer carries that as participation factors), and reactive limits switched without hysteresis. The matching Sparlectra configuration, then the bus voltages against the reference file:
# uses: powsybl_reference (warm-up cell); pnet (Example 6.1)
@isdefined(pnet) || error("Run Example 6.1 first: it sets up pnet.")
olf_like = SparlectraConfig(
powerflow = PowerFlowConfig(max_iter = 60, tol = 1e-9, islands_enabled = true, qlimits = Sparlectra.QLimitConfig(hysteresis_pu = 1e-6), distributed_slack = Sparlectra.DistributedSlackConfig(enabled = true, p_mode = :imported)),
output = OutputConfig(logfile_results = :off, startup_latency_hint = false),
control = ControlConfig(),
)
pres = run_sparlectra(net = pnet, config = olf_like)
@assert pres.numerical_converged
ref, hdr = readdlm(powsybl_reference, ',', String; quotes = true, header = true)
cols = Dict(Symbol(strip(h)) => k for (k, h) in enumerate(vec(hdr)))
worst = 0.0
for i in axes(ref, 1)
bus = String(ref[i, cols[:bus_breaker_id]])
haskey(pnet.busDict, bus) || continue
dv = abs(pnet.nodeVec[pnet.busDict[bus]]._vm_pu - parse(Float64, ref[i, cols[:v_pu]]))
global worst = max(worst, dv)
end
println("worst |dV| against OpenLoadFlow over ", size(ref, 1), " buses: ", round(worst; sigdigits = 3), " pu")
@assert worst < 1e-5worst |dV| against OpenLoadFlow over 6 buses: 1.63e-9 puReading aid (Example 6.2): two load-flow programs with different conventions (ratio at the from side against rho at side 1, the magnetizing admittance placement, the slack distribution) agree to 1e-9 pu once the conventions are mapped, which is what the PowSyBl import settled. The same comparison runs for any .xiidm of your own next to a reference written with pypowsybl; without one, the file alone imports just the same. Details: PowSyBl Import.
Chapter 7: DTF, a legacy input deck with its own outage records
What DTF is. A fixed-column text deck of a legacy load-flow program: FOR001 is the input (the network plus run parameters), FOR002 the program's printed result report. The deck of this chapter, data/dtf_demo/sp_dtf5.DAT, is a self-built five-bus network in that layout (a ring of 110 kV lines, one 110/20 kV transformer with a tap control, two outage records); decks of other origin are external data and do not ship. A deck is a sequence of CARDS in a fixed order: parameter and text cards, the nominal voltages the voltage-level indices refer to, a size card with the bus and branch counts and the NAMED slack bus, the branch cards (L for a line, T for a transformer, with impedances in ohm and admittances in siemens at the voltage level the card names), compensation cards, transformer-control cards (winding voltages, the longitudinal tap range and step, an optional skew-angle regulator), the bus cards (type, level index, name, start voltage, load, generation), and after the buses an optional block of OUTAGE records between AUSFALL and ENDE, one branch outage per line. There is no per-terminal switch: a branch is in or out. Sparlectra reads the deck into typed records, raw lines included, and builds the network from them.
Example 7.1: reading the deck. read_dtf parses the cards; everything the deck says stays on the case object before any network exists:
# uses: dtf_deck (warm-up cell)
case = Sparlectra.DTFImporter.read_dtf(dtf_deck)
println("deck: base ", case.baseMVA, " MVA, ", length(case.buses), " buses, ", length(case.branches), " branches (", count(b -> b.kind == 'T', case.branches), " transformers), ", length(case.transformer_controls), " transformer control(s), ", length(case.outages), " outage record(s)")
println("nominal voltages (kV): ", join(case.nominal_voltages_kv, ", "), "; slack bus of the size card: ", case.size.slack)
@assert (length(case.buses), length(case.branches), length(case.transformer_controls), length(case.outages)) == (5, 6, 1, 2)deck: base 100.0 MVA, 5 buses, 6 branches (1 transformers), 1 transformer control(s), 2 outage record(s)
nominal voltages (kV): 110.0, 20.0; slack bus of the size card: NORDExample 7.2: the network and its solve. build_net turns the records into a Net (transformer transverse admittance on the branch, the named slack as the reference, PQ buses kept as fixed injections); from there it is the ordinary solver:
# uses: case (Example 7.1)
@isdefined(case) || error("Run Example 7.1 first: it sets up case.")
dnet = Sparlectra.DTFImporter.build_net(case)
println("network: ", length(dnet.nodeVec), " buses, ", length(dnet.branchVec), " branches")
@assert length(dnet.nodeVec) == length(case.buses)
dite, derg = runpf!(dnet, 50, 1e-8, 0)
@assert derg == 0
calcNetLosses!(dnet)
println("solved in ", dite, " iterations; losses ", round(dnet.totalLosses[end][1]; digits = 3), " MW")network: 5 buses, 6 branches
solved in 4 iterations; losses 0.484 MWReading aid (Example 7.2): a deck of the legacy program comes with its FOR002 report, the program's result for the same deck; the example examples/dtf/dtf_validation_base.jl parses such a report and compares bus voltages, branch flows and generator reactive power line by line. That comparison is how the DTF path was validated, and the place to look when a deck of your own disagrees with its old report. The demo deck has no such report: it was never run by the legacy program.
Example 7.3: an outage from the deck. The records between AUSFALL and ENDE name branches by kind, level, parallel identifier and the two bus names; find_outage_branch_indices resolves one to the base network's branches (exactly one match is required, a miss or an ambiguity is a diagnostic, not a guess), apply_single_branch_outage! takes the branch out, and the solve repeats:
# uses: case (Example 7.1); dnet (Example 7.2)
@isdefined(case) || error("Run Example 7.1 first: it sets up case.")
@isdefined(dnet) || error("Run Example 7.2 first: it sets up dnet.")
if isempty(case.outages)
println("this deck carries no outage records")
else
outage = first(case.outages)
matches = Sparlectra.DTFImporter.find_outage_branch_indices(case, outage)
println("first outage record resolves to branch index(es) ", matches, ": ", Sparlectra.DTFImporter.outage_match_diagnostic(case, outage, matches))
if length(matches) == 1
onet = Sparlectra.DTFImporter.build_net(case)
Sparlectra.DTFImporter.apply_single_branch_outage!(onet, matches[1])
markIsolatedBuses!(net = onet, log = false)
oite, oerg = runpf!(onet, 50, 1e-8, 0; islands_enabled = true)
@assert oerg == 0
dv = maximum(abs(onet.nodeVec[i]._vm_pu - dnet.nodeVec[i]._vm_pu) for i in eachindex(dnet.nodeVec))
println("with the outage: solved in ", oite, " iterations, largest |Vm| change ", round(dv; sigdigits = 3), " pu")
end
endfirst outage record resolves to branch index(es) [5]: Matched native DTF branch 5 for outage L1 WEST -> OST
with the outage: solved in 4 iterations, largest |Vm| change 0.00356 puReading aid (Example 7.3): the outage records are part of the deck, so a DTF study is reproducible from the file alone; the Web UI and the service apply them the same way (for001Contingencies). What the deck cannot say (a one-sided open branch, a switch) has no record, and the importer does not invent one.
Where to go next
- PowSyBl Import: what the IIDM reader resolves, the conventions against OpenLoadFlow.
- DTF Format: the cards, the outage records, the validation workflow against FOR002.
- CGMES Import: the mapping reference, config keys, placeholder guards, and the topology processor.
- CGMES Export: profiles written, identity stability, provenance.
- Workshop tour, basic and advanced: everything you can do with the imported network.
- Feature Matrix