Copyright 2023-2026 Udo Schmitz
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
file: docs/lit/workshoptourcgmes.jl purpose: Literate.jl source of the CGMES tour: what an ENTSO-E delivery is, reading and analyzing it, bus-branch and node-breaker imports (with and without a TP profile, #314), SV validation, and the export round trip. Runs on the official conformity test sets, fetched on demand.
The Sparlectra workshop tour: CGMES
Note: This workshop was created with AI assistance and is reviewed and curated by the maintainer; it is not a fully machine-generated text.
Level: Expert. You should be comfortable importing and solving networks (basic tour); no prior CGMES knowledge is assumed, building that knowledge is the point of this tour.
CGMES (Common Grid Model Exchange Standard) is how European TSOs exchange grid models: a DELIVERY of several RDF/XML files, each carrying one PROFILE of the same network. This tour works on the official ENTSO-E conformity test sets, downloaded on demand (about 22 MB, once):
- Anatomy of a delivery: profiles, and what a summary shows
- When an import cannot work: the analysis report
- Bus-branch import, solve, and validation against the shipped state
- Node-breaker: with a TP profile, and without one (the topology processor)
- The export round trip
Warm-up and the test data
The ENTSO-E conformity package bundles reference networks in several variants; ensureCGMESTestConfigurations downloads and extracts it once into a local cache and returns the extraction root. Everything below works on those files.
using Sparlectra
root = Sparlectra.CGMESImporter.ensureCGMESTestConfigurations()
println("conformity sets under: ", root)
# the two study deliveries of this tour
microgrid_be = joinpath(root, "MicroGrid", "BaseCase_BC", "CGMES_v2.4.15_MicroGridTestConfiguration_BC_BE_v2")
microgrid_bd = joinpath(root, "MicroGrid", "BaseCase_BC", "CGMES_v2.4.15_MicroGridTestConfiguration_BD_v2")
minigrid_nb = joinpath(root, "MiniGrid", "NodeBreaker", "CGMES_v2.4.15_MiniGridTestConfiguration_BaseCase_Complete_v3")
minigrid_bd = joinpath(root, "MiniGrid", "NodeBreaker", "CGMES_v2.4.15_MiniGridTestConfiguration_Boundary_v3")"/home/runner/work/Sparlectra.jl/Sparlectra.jl/data/CGMES/extracted/MiniGrid/NodeBreaker/CGMES_v2.4.15_MiniGridTestConfiguration_Boundary_v3"Chapter 1: anatomy of a delivery
Example 1.1: what is in the box. MicroGrid BE is the Belgian half of a two-country reference model, as a sketch:
NL area (boundary) ~~~~ X-nodes ~~~~ BE area (this delivery)
380 kV ring: 2 substations
220/110 kV under 2 transformers
gens, loads, shunt, PSTOne delivery = one folder (or ZIP) of profile files. The important ones:
- EQ (equipment): what exists, lines, transformers, machines, their physical parameters. The skeleton.
- SSH (steady-state hypothesis): the operating point, set points, switch states, in-service flags.
- TP (topology): the sender's bus partition,
TopologicalNodes and which terminal sits on which node. - SV (state variables): the sender's solved state, voltages and flows. Reference material, not input.
- Boundary set (EQBD/TPBD): the X-nodes where areas stitch together; shipped separately so neighbors agree on the seam.
summarizeCGMES reads a delivery without importing it:
s = summarizeCGMES(path = [microgrid_be, microgrid_bd])
print(s)CGMES summary
version: 2.4.15
files: 9 (6 read, 3 skipped)
- 20171002T0930Z_1D_BE_SSH_2.xml [SSH] read
- 20171002T0930Z_1D_BE_SV_2.xml [SV] read
- 20171002T0930Z_1D_BE_TP_2.xml [TP] read
- 20171002T0930Z_BE_DL_2.xml [DL] skip: profile not in import set
- 20171002T0930Z_BE_DY_2.xml [DY] skip: profile not in import set
- 20171002T0930Z_BE_EQ_2.xml [EQ,EQ_SC] read
- 20171002T0930Z_BE_GL_2.xml [GL] skip: profile not in import set
- 20171002T0930Z_ENTSO-E_EQ_BD_2.xml [EQ_BD,EQ_BD_OP] read
- 20171002T0930Z_ENTSO-E_TP_BD_2.xml [TP_BD] read
objects: 324 in 35 classes
CurrentLimit 100
Terminal 45
OperationalLimitSet 23
Line 13
SvPowerFlow 12
TopologicalNode 12
SvVoltage 11
BusbarSection 9
PowerTransformerEnd 9
OperationalLimitType 8
BaseVoltage 7
ACLineSegment 7
EnergySchedulingType 6
VoltageLevel 6
ConnectivityNode 6
… 20 more classes
unresolved references: 0Reading aid (Example 1.1): the summary names the files with their profiles, the CGMES version, object counts per class, and the declared model dependencies. Nothing is mapped yet; this is the "what did I just receive" view.
Chapter 2: when an import cannot work
Example 2.1: the missing boundary set. Deliveries reference objects they do not define, most prominently the boundary X-nodes. Import the BE files WITHOUT the boundary set and the import aborts; analyzeCGMES explains the gap in plain language instead of a bare error:
a_report = analyzeCGMES(path = microgrid_be)
print(a_report)CGMES import analysis
=====================
Supplied models:
- 20171002T0930Z_1D_BE_SSH_2.xml: SSH v2.4.15, 70 objects, model 52b712d1-f3b0-4a59-9191-79f2fb1e4c4e
- 20171002T0930Z_1D_BE_SV_2.xml: SV v2.4.15, 30 objects, model c2960b34-0a04-4cd1-9c4d-f3112d85ec6c
- 20171002T0930Z_1D_BE_TP_2.xml: TP v2.4.15, 50 objects, model f2f43818-09c8-4252-9611-7af80c398d20
- 20171002T0930Z_BE_DL_2.xml: DL v2.4.15, 0 objects, model 0eb1540d-5ca3-40ab-9bf7-c91ec4c5d7c3 [skipped: profile not in import set]
- 20171002T0930Z_BE_DY_2.xml: DY v2.4.15, 0 objects, model 239caf0a-38bf-468b-a4d8-15f619886c58 [skipped: profile not in import set]
- 20171002T0930Z_BE_EQ_2.xml: EQ+EQ_SC v2.4.15, 256 objects, model d400c631-75a0-4c30-8aed-832b0d282e73
- 20171002T0930Z_BE_GL_2.xml: GL v2.4.15, 0 objects, model 9e383122-10cf-4da3-9785-f404b3541588 [skipped: profile not in import set]
Declared dependencies MISSING from the input (md:Model.DependentOn):
- model 2399cbd0-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_TP_2.xml, 20171002T0930Z_BE_EQ_2.xml
- model 2399cbd1-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_SV_2.xml, 20171002T0930Z_1D_BE_TP_2.xml
Unresolved references: 38 total
- Terminal.TopologicalNode: 10
- ACLineSegment.BaseVoltage: 7
- SvVoltage.TopologicalNode: 5
- EquivalentInjection.BaseVoltage: 5
- PowerTransformerEnd.BaseVoltage: 4
- VoltageLevel.BaseVoltage: 3
- TopologicalNode.BaseVoltage: 3
- TopologicalIsland.AngleRefTopologicalNode: 1
Verdict: the delivery declares 2 prerequisite model(s) that are not part of the input.
In real ENTSO-E deliveries these are typically the boundary files (EQ_BD/TP_BD) of the
matching date — the boundary carries the border X-nodes and the shared BaseVoltage catalog.
Obtain the matching boundary set and pass it as an additional path.
CGMES import analysis
=====================
Supplied models:
- 20171002T0930Z_1D_BE_SSH_2.xml: SSH v2.4.15, 70 objects, model 52b712d1-f3b0-4a59-9191-79f2fb1e4c4e
- 20171002T0930Z_1D_BE_SV_2.xml: SV v2.4.15, 30 objects, model c2960b34-0a04-4cd1-9c4d-f3112d85ec6c
- 20171002T0930Z_1D_BE_TP_2.xml: TP v2.4.15, 50 objects, model f2f43818-09c8-4252-9611-7af80c398d20
- 20171002T0930Z_BE_DL_2.xml: DL v2.4.15, 0 objects, model 0eb1540d-5ca3-40ab-9bf7-c91ec4c5d7c3 [skipped: profile not in import set]
- 20171002T0930Z_BE_DY_2.xml: DY v2.4.15, 0 objects, model 239caf0a-38bf-468b-a4d8-15f619886c58 [skipped: profile not in import set]
- 20171002T0930Z_BE_EQ_2.xml: EQ+EQ_SC v2.4.15, 256 objects, model d400c631-75a0-4c30-8aed-832b0d282e73
- 20171002T0930Z_BE_GL_2.xml: GL v2.4.15, 0 objects, model 9e383122-10cf-4da3-9785-f404b3541588 [skipped: profile not in import set]
Declared dependencies MISSING from the input (md:Model.DependentOn):
- model 2399cbd0-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_TP_2.xml, 20171002T0930Z_BE_EQ_2.xml
- model 2399cbd1-9a39-11e0-aa80-0800200c9a66 — required by 20171002T0930Z_1D_BE_SV_2.xml, 20171002T0930Z_1D_BE_TP_2.xml
Unresolved references: 38 total
- Terminal.TopologicalNode: 10
- ACLineSegment.BaseVoltage: 7
- SvVoltage.TopologicalNode: 5
- EquivalentInjection.BaseVoltage: 5
- PowerTransformerEnd.BaseVoltage: 4
- VoltageLevel.BaseVoltage: 3
- TopologicalNode.BaseVoltage: 3
- TopologicalIsland.AngleRefTopologicalNode: 1
Verdict: the delivery declares 2 prerequisite model(s) that are not part of the input.
In real ENTSO-E deliveries these are typically the boundary files (EQ_BD/TP_BD) of the
matching date — the boundary carries the border X-nodes and the shared BaseVoltage catalog.
Obtain the matching boundary set and pass it as an additional path.Reading aid (Example 2.1): the analysis lists the supplied models, the declared dependencies (md:Model.DependentOn), which of them are missing from the input, and a histogram of unresolved references. The missing boundary set is named BY MODEL ID: exactly the file to ask the sender for. The same report lands in cgmes.log when a Web UI or service import aborts.
Chapter 3: bus-branch import, solve, validate
Example 3.1: the working import. With the boundary set the BE model (network of Example 1.1) imports into an ordinary Net; everything the other workshops do applies from here:
res = importCGMES(path = [microgrid_be, microgrid_bd], name = "microgrid_be")
net = res.net
println("imported: ", length(net.nodeVec), " buses, ", length(net.branchVec), " branches, ", length(net.prosumpsVec), " injections")
etime = @elapsed ((ite, erg) = runpf!(net, 30, 1e-8, 0; islands_enabled = true))
println("solved: status ", erg, " in ", ite, " iterations")imported: 12 buses, 13 branches, 10 injections
solved: status 0 in 4 iterationsThe classical result tables, exactly as in the basic tour: calcNetLosses! derives branch flows and losses from the converged voltages, printACPFlowResults prints bus voltages and branch flows.
calcNetLosses!(net)
printACPFlowResults(net, etime, ite, 1e-8)================================================================================
| SPARLECTRA Version 0.9.19 - AC Power Flow Results |
================================================================================
Date : 25-Aug-26 16:19:44
Iterations : 4
Flatstart : No
Tolerance : 1.0e-08
Solver : NR
Total time : 0.335358 s
Case : microgrid_be
Cooldown iters : 0
Q-hysteresis : 0.0000 pu
Jacobian cond. : kappa1(J) = 35600.0, attainable accuracy ~ 7.9e-12, well conditioned (tol 1.0e-8 reachable)
BaseMVA : 100
Nodes : 12 (PV: 1 PQ: 10 Slack: 1)
Grid connection: slack bus BE-Busbar_4
Branches : 13
Links : 0
HVDC links : 0
Lines : 2
Trafos : 11
Generators : 7
Loads : 3
Shunts : 2
Controllers : 0 (Tap: 0, Q(U): 0, P(U): 0)
PV→PQ locks : 0
PV→PQ events : 0
total network power balance (Σ S_branch): P = 40.528 [MW], Q = -24.110 [MVar]
==========================================================================================================================================================================================================================
| Nr | Bus | Vn [kV] | V [kV] | V [pu] | phi [deg] | Pg [MW] | Qg [MVar] | Pl [MW] | Ql [MVar] | Ps [MW] | Qs [MVar] | Type | Control | Tap Vm tgt |
==========================================================================================================================================================================================================================
| 1 | TN_Border_ST23 | 220.0 | 225.492 | 1.025 | -5.544 | 27.365 | -0.426 | | | | | PQ | - | |
| 2 | BE-Busbar_2 | 225.0 | 224.866 | 0.999 | -7.589 | | | 1.000 | | | | PQ | - | |
| 3 | BE_TR_BUS4 | 225.0 | 224.310 | 0.997 | -8.734 | | | 200.000 | 50.000 | | | PQ | - | |
| 4 | BE_TR_BUS2 | 380.0 | 412.977 | 1.087 | -6.745 | | | | | 1.194 | -59.010 | PQ | - | |
| 5 | TN_Border_AL11 | 400.0 | 411.139 | 1.028 | -6.552 | 46.817 | -79.194 | | | | | PQ | - | |
| 6 | TN_Border_MA11 | 400.0 | 412.599 | 1.031 | -6.711 | 43.687 | -84.877 | | | | | PQ | - | |
| 7 | TN_Border_GY11 | 400.0 | 410.869 | 1.027 | -6.541 | 90.037 | -148.604 | | | | | PQ | - | |
| 8 | TN_Border_ST24 | 220.0 | 224.961 | 1.023 | -5.602 | 26.805 | -1.490 | | | | | PQ | - | |
| 9 | BE-Busbar_6 | 110.0 | 115.492 | 1.050 | -9.355 | | | 200.000 | 90.000 | | -330.699 | PQ | - | |
| 10 | BE-Busbar_4 | 10.5 | 10.821 | 1.031 | -7.057 | 90.010 | -51.813 | | | | | SLACK | - | |
| 11 | AUX3WT_BE-TR3_1 | 380.0 | 412.658 | 1.086 | -7.318 | | | | | | | PQ | - | |
| 12 | BE-Busbar_5 | 21.0 | 21.987 | 1.047 | -6.615 | 118.000 | 18.720 | | | | | PV | - | |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
==========================================================================================================================================================================================================================
| Branch | Type | Connection | P [MW] | Q [MVar] | P [MW] | Q [MVar] | Pv [MW] | Qv [MVar] | Ctrl | P_tgt | TapPos | Ctrl status |
==========================================================================================================================================================================================================================
| B_2WT_220_1_2 | Trafo | TN_Border_ST23 -> BE-Bus… | 27.365 | -0.426 | -25.773 | -2.828 | 1.592 | -3.254 | - | - | - | - |
| B_ACL_225_3_2 | Line | BE_TR_BUS4 -> BE-Busbar_2 | -27.876 | -2.748 | 31.314 | 1.205 | 3.438 | -1.543 | - | - | - | - |
| B_2WT_380_4_5 | Trafo | BE_TR_BUS2 -> TN_Border_… | -36.591 | 54.188 | 46.817 | -79.194 | 10.226 | -25.006 | - | - | - | - |
| B_2WT_380_4_6 | Trafo | BE_TR_BUS2 -> TN_Border_… | -36.860 | 80.683 | 43.687 | -84.877 | 6.828 | -4.194 | - | - | - | - |
| B_2WT_380_4_7 | Trafo | BE_TR_BUS2 -> TN_Border_… | -82.841 | 138.452 | 90.037 | -148.604 | 7.196 | -10.152 | - | - | - | - |
| B_ACL_225_3_2 | Line | BE_TR_BUS4 -> BE-Busbar_2 | -11.223 | -1.075 | 17.296 | 0.353 | 6.074 | -0.723 | - | - | - | - |
| B_2WT_220_8_2 | Trafo | TN_Border_ST24 -> BE-Bus… | 26.805 | -1.490 | -23.838 | 1.271 | 2.967 | -0.219 | - | - | - | - |
| B_2WT_380_4_9 | Trafo | BE_TR_BUS2 -> BE-Busbar_6 | 55.894 | -217.082 | -55.095 | 221.365 | 0.799 | 4.284 | - | - | - | - |
| B_2WT_225_3_9 | Trafo | BE_TR_BUS4 -> BE-Busbar_6 | 55.281 | 39.414 | -55.212 | -38.469 | 0.070 | 0.945 | - | - | - | - |
| B_2WT_110_9_10 | Trafo | BE-Busbar_6 -> BE-Busbar… | -89.693 | 57.803 | 90.010 | -51.813 | 0.317 | 5.990 | - | - | - | - |
| B_2WT_380_11_4 | Trafo | AUX3WT_BE-TR3_1 -> BE_TR… | -99.151 | -2.191 | 99.203 | 2.770 | 0.052 | 0.578 | - | - | - | - |
| B_2WT_380_11_3 | Trafo | AUX3WT_BE-TR3_1 -> BE_TR… | 216.531 | 91.983 | -216.183 | -85.591 | 0.348 | 6.392 | - | - | - | - |
| B_2WT_380_11_12 | Trafo | AUX3WT_BE-TR3_1 -> BE-Bu… | -117.380 | -89.792 | 118.000 | 92.583 | 0.620 | 2.791 | - | - | - | - |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Control
-------
Transformer controls: noneThe delivery also carries the sender's solved state (SV), which makes it its own reference: compareWithSV checks the re-solved voltages and flows against what the sender shipped:
cmp = compareWithSV(res)
println("compared ", cmp.n, " buses against the shipped SV: max |dVm| = ", round(cmp.max_dvm; sigdigits = 3), " pu, max |dVa| = ", round(cmp.max_dva; sigdigits = 3), " deg")compared 11 buses against the shipped SV: max |dVm| = 7.25e-5 pu, max |dVa| = 0.0358 degReading aid (Example 3.1): agreement to a few 1e-4 pu means the model mapped faithfully, the remaining difference is solver tolerance and rounding in the shipped SV. A LARGE deviation would point at a mapping or data problem; the per-bus table in the comparison names where.
Chapter 4: node-breaker, with and without TP
Example 4.1: node-breaker with the shipped TP. Node-breaker deliveries model every busbar, breaker, and disconnector explicitly; the bus partition is the result of aggregating connectivity across closed switches. Sketch of one MiniGrid substation:
busbar A ═══╤═══════╤═══ every ─[/]─ is a breaker or
[/] [/] disconnector with its own
busbar B ═══╧══╤════╧═══ SSH switch state; equipment
[/] connects through bays, not
line directly to a busbarAs long as the sender ships a TP profile, the importer simply consumes the sender's aggregation result:
res_tp = importCGMES(path = [minigrid_nb, minigrid_bd], name = "minigrid_nb")
println("with TP: ", length(res_tp.net.nodeVec), " buses, ", length(res_tp.net.branchVec), " branches")with TP: 15 buses, 19 branchesExample 4.2: node-breaker WITHOUT a TP profile. EMS and substation-level exports often ship EQ+SSH only, topology expressed as ConnectivityNodes plus switch states, and no topology processor ever ran. Since 0.9.16 Sparlectra derives the partition itself: connectivity nodes aggregate across closed switches (SSH open overriding EQ normalOpen, out-of-service counts as open), retained switches stay bus couplers, and the derived nodes feed the unchanged import pipeline. We simulate such a delivery by withholding the TP and SV files of Example 4.1's set:
files_no_tp = [f for f in readdir(minigrid_nb; join = true) if endswith(f, ".xml") && !occursin("_TP", basename(f)) && !occursin("_SV", basename(f))]
res_notp = importCGMES(path = vcat(files_no_tp, minigrid_bd), name = "minigrid_nb_no_tp")
println("without TP: ", length(res_notp.net.nodeVec), " buses, ", length(res_notp.net.branchVec), " branches")
for m in res_notp.messages
occursin("topology processor", m) && println(" ", m)
end
# the twins must describe the same network: solve both and compare
net_a = res_notp.net
net_b = res_tp.net
net_a.flatstart = true
net_b.flatstart = true
_, ea = runpf!(net_a, 40, 1e-8, 0; islands_enabled = true)
_, eb = runpf!(net_b, 40, 1e-8, 0; islands_enabled = true)
va = sort([n._vm_pu for n in net_a.nodeVec])
vb = sort([n._vm_pu for n in net_b.nodeVec])
println("both solve (", ea, "/", eb, "); max |Vm| difference across the sorted buses: ", maximum(abs.(va .- vb)))without TP: 15 buses, 19 branches
topology processor: derived 11 topological node(s) from 103 connectivity node(s) across 90 closed switch merge(s) (TP profile absent); 0 retained closed switch(es) kept as bus couplers, 0 open switch(es) split the graph, 2 boundary group(s) adopted their TP_BD node
both solve (0/0); max |Vm| difference across the sorted buses: 0.0Reading aid (Example 4.2): the derived partition reproduces the sender's TP bus for bus (up to naming: derived buses take their busbar names), and the solved voltages agree to numerical precision. The processor announces itself in the import messages and runs ONLY when no usable TP is present; TP-carrying deliveries take the unchanged path of Example 4.1. One honest caveat from the conformity sweep: a TP that assigns terminals of ONE connectivity node to DIFFERENT topological nodes (FullGrid's completeness set does this on a load node) cannot be derived from the connectivity graph; the processor then produces the graph-consistent partition and the affected injections sit one bus apart.
Chapter 5: the export round trip
Example 5.1: write it back out. The imported network (Example 3.1) exports as a complete CGMES 2.4.15 delivery (EQ+TP+SSH+SV) with roundtrip-stable identities: mRIDs recorded on import are reused, so renaming nothing changes nothing. The re-import of the export solves to the same power flow:
outdir = mktempdir()
files = writeCGMESFiles(net; path = outdir)
println(length(files), " profile files written")
res2 = importCGMES(path = outdir, name = "roundtrip")
ite2, erg2 = runpf!(res2.net, 30, 1e-8, 0; islands_enabled = true)
vm1 = sort([n._vm_pu for n in net.nodeVec])
vm2 = sort([n._vm_pu for n in res2.net.nodeVec])
println("re-import solves (status ", erg2, "); max |Vm| difference to the original: ", maximum(abs.(vm1 .- vm2)))4 profile files written
re-import solves (status 0); max |Vm| difference to the original: 1.9984014443252818e-15Reading aid (Example 5.1): the export writes what the network IS, not what the original files said; the shipped SV of the export carries the CURRENT solved state, so a receiving tool starts from it. Details and the identity rules: CGMES Export.
Where to go next
- CGMES Import: the mapping reference, config keys, placeholder guards, and the topology processor.
- CGMES Export: profiles written, identity stability, provenance.
- Workshop tour, basic and advanced: everything you can do with the imported network.
- Feature Matrix