Test Suite

Sparlectra uses profile-aware test loading through test/runtests.jl. Profile selection precedence is:

1. CLI argument (julia --project=. test/runtests.jl <profile>)
2. SPARLECTRA_TEST_PROFILE
3. default fast

Test profiles

Fast versus extended ownership

Pkg.test() uses the same test runner and therefore the default fast profile unless SPARLECTRA_TEST_PROFILE is set:

julia --project=. -e 'using Pkg; Pkg.test()'

Fast profile groups

Extended profile additions

The extended profile is extended-only: it does not run the fast profile first. Use all when a single invocation must execute both fast and extended suites.

Current extended-only groups are:

• core_model_extended
• programmatic_api_extended
• webui_extended
• repository_hygiene
• apslf
• dtf_extended

dtf_extended group

dtf_extended is a normal extended-only group. It owns exactly five DTF test files and invokes each of their run functions once:

extended includes dtf_extended, and all runs fast once followed by the complete extended profile once, so the DTF tests run exactly once per extended or all invocation. Optional external FOR001/FOR002 fixtures continue to skip cleanly (as Broken, not failed) when absent.

test/extended/runtests_extended.jl remains an optional standalone DTF-focused runner. It includes the same five files directly and is useful for iterating on DTF coverage without running the rest of the extended profile:

julia --project=. test/extended/runtests_extended.jl

Native DTF validation examples with FOR002 reference reports

The native DTF examples with FOR002 reference reports validate Testnetz13 through the direct DTF format path: the current Julia compatibility module DTFImporter reads the DTF format with DTFImporter.read_dtf -> DTFImporter.build_net -> runpf!. They deliberately avoid MATPOWER import/export and the generated FOR001 builder so that native DTF format parsing, Net construction, and solved branch-flow reporting are exercised directly. FOR002 is used as a legacy textual reference report.

External FOR001/FOR002 validation datasets are not shipped with Sparlectra. Place local validation files under data/DTF/ or pass explicit paths with --dtf-file and --for002-file. If the files are absent, the optional validation scripts stop with a clear missing-data message instead of substituting unrelated data or claiming that validation was executed.

examples/validate_dtf_suite.jl is the shared CLI runner for all three checks (plus the DTF import audit). It bundles cases and modes into one command and dispatches to library modules under examples/internal/ (dtf_validation_base.jl, dtf_validation_outages.jl, dtf_validation_matpower.jl, dtf_validation_audit.jl), so there is one implementation of each check shared by the CLI, the suite, and the extended test files.

Run the base-case validation with:

julia --project=. examples/validate_dtf_suite.jl --mode=base --case=A --data-dir=data/DTF --output-dir=examples/_out/dtf_for002_native_validation --write-csv=true --write-markdown=true

Run the outage validation with:

julia --project=. examples/validate_dtf_suite.jl --mode=outages --case=A --data-dir=data/DTF --output-dir=examples/_out/dtf_for002_native_outages --write-csv=true --write-markdown=true

Each command writes concise console output plus CSV and Markdown files in the requested --output-dir. The Markdown files summarize the run, while CSV files keep row-level bus, generator, branch, KCL, state-residual, and metric diagnostics.

The MATPOWER export validation example checks a different question: whether an already-built native DTF Net can be exported by Sparlectra's existing MATPOWER exporter and re-imported without materially changing the solved Sparlectra result. Its required path is DTFImporter.read_dtf -> DTFImporter.build_net -> writeMatpowerCasefile -> createNetFromMatPowerFile; it does not implement a DTF-specific exporter and does not use FOR002 as the primary roundtrip reference. Run it with:

julia --project=. examples/validate_dtf_suite.jl --mode=matpower --case=A --data-dir=data/DTF --output-dir=examples/_out/dtf_matpower_export_testnetz13 --write-csv=true --write-markdown=true --write-matpower=true --run-outages=true

Each examples/internal/dtf_validation_*.jl module is also directly runnable as its own single-purpose CLI entry point (for example julia --project=. examples/internal/dtf_validation_base.jl --dtf-file=... --for002-file=...), which is what the extended example tests drive for exact console-output coverage; prefer the suite above for everyday case/mode selection.

The command writes dtf_matpower_export_summary.md, dtf_matpower_export_metrics.csv, bus/branch/generator comparison CSV files, and exported MATPOWER .m cases in the selected output directory. The exporter uses Sparlectra's established optional MATPOWER metadata fields mpc.bus_name, mpc.branch_name, mpc.branch_kind, and mpc.for001_contingencies when that information is available, so DTF bus names, stable branch labels, DTF branch kind (L/T), and the parsed outage cards can be recovered by MatpowerIO.read_case_m. Standard MATPOWER bus, generator, and branch data remain sufficient for solving files that do not contain that metadata. The roundtrip proves preservation through Sparlectra's MATPOWER export/import path; it does not independently certify agreement with external FOR002 outage blocks.

The roundtrip import disables MATPOWER PQ-generator controller reinterpretation for this diagnostic so DTF PQ generators remain fixed injections. The exporter also writes MATPOWER TAP as 0.0 for line rows and as the explicit Sparlectra ratio for transformer rows, matching MATPOWER's line/transformer convention.

Important metrics:

• converged: whether runpf! reported successful convergence.
• iterations: Newton iterations used by the native solve.
• final mismatch: infinity-norm mismatch recomputed from the solved state.
• max voltage deviation: largest voltage magnitude or angle difference from FOR002 printed bus values.
• max branch P/Q deviation: largest directed branch endpoint-flow difference from FOR002.
• max generator/slack Q deviation: difference between solved generator/slack reactive output and FOR002 generator-Q reporting.
• state residual P/Q: injection residual from forcing FOR002 printed voltage magnitudes/angles into the native Ybus and comparing the calculated injections with the FOR002 bus table. Reported only in the CSV diagnostics (dtf_state_residual.csv, dtf_validation_metrics.csv, and the outage equivalents), not in console or Markdown summaries.

Current Testnetz13 interpretation:

• Branch-flow deviations are small and are the strongest validation signal for the native DTF path.
• Slack Q is solved by the power flow and should not be compared with the specified input Q as if it were fixed.
• State residuals are gross-error diagnostics only: their rounding-noise floor (FOR002 prints rounded voltages/angles, and transformer-adjacent buses amplify tiny differences) sits far above real model deviations, so they are kept in the row-level CSVs but excluded from console and Markdown summaries.
• Outage validation currently executes only the outages listed in DTF.
• FOR002 may contain more outage blocks than FOR001 lists; unmatched FOR002 blocks are treated as reference text, not executed scenarios.

Offline and runtime expectations

The default fast profile is intended to be offline-safe and should not download MATPOWER cases or run benchmark loops. The Q-limit large-case comparison workflow is a manual diagnostic tool: it resolves optional large MATPOWER cases through Sparlectra's case registry/download support, compares case × start_profile × qlimit_mode, and uses its CSV/JSON summaries as the primary output. Real runs for cases such as case13659pegase.m and case_SyntheticUSA.m are intentionally excluded from the fast tests; automated coverage uses stubs for case resolution and API execution. The experimental large-case Q-limit comparison test block is suppressed from the normal fast profile; run test/test_qlimit_large_case_comparison.jl manually when maintaining that tool.

Default fast-profile output is intentionally compact: the runner prints the selected profile, one [n/8] marker per group, and Julia's final test summary. MATPOWER import diagnostics, auto-profile tables, runtime casefile banners, Q-limit tables, and similar artifact-oriented diagnostic blocks are suppressed in normal test stdout so progress remains scannable.

Fast profile example on Windows / Julia 1.12.6: 934 tests passed in approximately 95 seconds. Runtime is machine-dependent and is not a CI threshold.

Use an explicit verbose opt-in when debugging a noisy test path:

SPARLECTRA_TEST_VERBOSE=1 julia --project=. test/runtests.jl
julia --project=. test/runtests.jl --verbose

Verbose mode does not change the selected profile. Existing profile selection remains available, for example:

julia --project=. test/runtests.jl fast --verbose
julia --project=. test/runtests.jl extended --verbose

The fast profile runs normal unit and focused integration tests. The extended profile runs only extended repository-hygiene, documentation-coverage, example, and fixture-heavy checks. The all profile runs fast followed by extended. Set SPARLECTRA_TEST_GC_BETWEEN_GROUPS=1 to request a GC cycle after each completed group; per-group output reports elapsed seconds, allocated MiB, and GC seconds.

The extended profile may include MATPOWER/example/output-heavy tests and native DTF diagnostic-example checks with FOR002 reference reports. These tests stay isolated from the default profile.

Use fast during normal development. Use extended before merging changes that affect configuration, MATPOWER import, output formatting, performance reporting, or broader integration paths.

Fast-profile volume review

The fast profile currently contains a mix of true unit/smoke coverage and several integration-style service/UI paths:

• True unit or focused smoke tests: configuration key/value validation, MATPOWER auto-profile decision rules on tiny synthetic cases, rectangular solver API checks with small fixtures, core model invariants, state-estimation smoke/regression cases, and control-loop unit/regression checks.
• Integration-style tests that remain in fast because they protect recent public behavior: focused API service request/metadata smoke checks, Web UI form rendering and stubbed routing, allowlisted documentation/help routing, operation-log safety, run deletion safety, and one small API service run.
• Heavier or broader tests already isolated in extended: MATPOWER example runner coverage, synthetic-grid regressions, configuration documentation consistency, PV residual integration coverage, structural remove/delete behavior, real asynchronous Web UI/API job lifecycles, and artifact lifecycle matrices.
• Expensive or duplicate candidates to watch: repeated API/Web UI artifact-generation parity checks, broad service-path status/recovery assertions that overlap between test_api.jl and test_webui.jl, and any future large-case or repeated auto-profile scans. These should move to extended if they become slow, require network/cached large cases, or duplicate a smaller fast regression.

No tests were moved in this review. The current fast-profile volume is acceptable as long as default stdout remains quiet and the service/UI cases continue to use small offline fixtures. Future large-case regressions such as case13659pegase.m should use a small reproducible proxy in fast and keep the real large-case check in extended/manual verification unless the case is already cached and cheap.

Pre-merge verification gate (config / MATPOWER / output / performance / docs changes)

For branches that touch central configuration, MATPOWER runner behavior, output routing/formatting, performance reporting, or documentation/config consistency, complete this checklist before merge:

Bash

julia --project=. -e 'using Pkg; Pkg.instantiate()'
julia --project=. test/runtests.jl
SPARLECTRA_TEST_PROFILE=extended julia --project=. test/runtests.jl
julia --project=docs docs/make.jl

PowerShell

julia --project=. -e 'using Pkg; Pkg.instantiate()'
julia --project=. test/runtests.jl

$env:SPARLECTRA_TEST_PROFILE="extended"
julia --project=. test/runtests.jl
Remove-Item Env:SPARLECTRA_TEST_PROFILE

julia --project=docs docs/make.jl

This keeps the default local workflow fast while making the extended profile and docs build an explicit pre-merge gate for integration-heavy changes.

Output-summary regression note

The fast profile includes a regression for printACPFlowResults(...; toFile=true, result_mode=:summary). It verifies that the result file is closed/flushed before the function returns and that the summary contains Q-limit counter labels. Equivalent environment-variable usage remains supported:

SPARLECTRA_TEST_PROFILE=extended julia --project=. test/runtests.jl

Progress output

The runner prints a lightweight progress view, for example:

Test framework: fast
[1/8] core_model
[2/8] powerflow_rectangular
...
[8/8] controls

Julia's final Test Summary remains unchanged and visible at the end.

Rectangular/Q-limit diagnostics in tests

The rectangular convergence and Q-limit active-set diagnostic block is not printed in normal test runs. Those diagnostics remain available only through explicit diagnostic requests (for example solver verbose > 0 paths used during focused debugging).

Experimental/internal DTF Web/API input path

The PowerFlow service and Web UI include an experimental/internal DTF format input path for diagnostics and validation. This path is deliberately cautious and is not yet announced as a public supported file format. Use case_format = :dtf_for001 for explicit native input, or case_format = :auto only when the FOR001 markers are unambiguous; ambiguous .DAT files are rejected instead of being silently interpreted.

The native API path uses DTFImporter.read_dtf -> DTFImporter.build_net -> Sparlectra.Net -> run_sparlectra/runpf! and does not solve through a MATPOWER intermediate conversion. The Web UI places the selector in the advanced/internal Input format section with the cautious label "DTF diagnostics (experimental/internal)". FOR002 is treated as a reference/result file rather than a runnable input case: FOR002-like .DAT files are hidden from the primary case selector, can be selected or typed as an optional FOR002 reference file when available in the case cache, and must not be auto-paired with FOR001. DTF-listed outages can be requested explicitly as all outages or selected outage labels/indices; the default remains base-case-only.

Generated artifacts use the existing PowerFlow run artifact mechanism. Stable DTF-path artifact filenames include dtf_import_summary.md, dtf_import_summary.csv, dtf_for002_base_comparison.md, dtf_for002_base_metrics.csv, dtf_native_matpower_export.m, and per-outage files such as dtf_outage_1_summary.md and dtf_outage_1_metrics.csv.

Limitation/TODO: DC lines, HVDC links, and active MATPOWER mpc.dcline data are not modeled by this native DTF/MATPOWER power-flow path. They fail clearly with structured unsupported-DC-line diagnostics instead of being approximated or dropped.