API Reference

Configuration and Data Structures

Args

GraphNetSim.Args — Type
Args

Configuration structure for training and evaluating Graph Neural Network simulators.

Fields

Network Architecture

  • mps::Integer=15: Number of message passing steps (higher = more expressive but slower)
  • layer_size::Integer=128: Latent dimension for MLP hidden layers
  • hidden_layers::Integer=2: Number of hidden layers in each MLP module

Training Configuration

  • epochs::Integer=1: Number of passes over the entire dataset
  • steps::Integer=10e6: Total number of training steps
  • checkpoint::Integer=10000: Interval (in steps) for saving checkpoints
  • norm_steps::Integer=1000: Steps to accumulate normalization statistics before weight updates
  • batchsize::Integer=1: Batch size (currently limited to 1 - full trajectory per batch)

Data Augmentation

  • types_updated::Vector{Integer}=[1]: Node types whose features are predicted
  • types_noisy::Vector{Integer}=Int[]: Node types to which noise is added during training
  • noise_stddevs::Vector{Float32}=[0.0f0]: Standard deviations for Gaussian noise (per type or broadcast)

Training Strategy

  • training_strategy::TrainingStrategy=DerivativeTraining(): Method for computing loss

Hardware and Optimization

  • use_cuda::Bool=true: Enable CUDA GPU acceleration (if available)
  • gpu_device::Union{Nothing,CuDevice}: CUDA device to use (auto-selected if CUDA functional)
  • optimizer_learning_rate_start::Float32=1.0f-4: Initial learning rate
  • optimizer_learning_rate_stop::Union{Nothing,Float32}=nothing: Final learning rate (for decay schedule)

Normalization

  • norm_type::Symbol=:online: Normalization strategy for Float32 features. :online (accumulate stats during training), :minmax (requires datamin/datamax in meta.json), :meanstd (requires datamean/datastd in meta.json).

Validation

  • show_progress_bars::Bool=true: Show training progress bars
  • use_valid::Bool=true: Load validation checkpoint (best loss) instead of final checkpoint
  • solver_valid::OrdinaryDiffEqAlgorithm=Tsit5(): ODE solver for validation rollouts
  • solver_valid_dt::Union{Nothing,Float32}=nothing: Fixed timestep for validation solver
  • reset_valid::Bool=false: Reset validation after loading checkpoint
  • save_step::Bool=false: No-op since GraphNetCore >= 0.4 (the per-step df_step DataFrame was removed upstream). Kept for backwards-compatible Args construction.
source

Dataset

GraphNetSim.Dataset — Type
Dataset

A mutable structure containing trajectory data and associated metadata for training, validation, or testing.

Fields

  • meta::Dict{String,Any}: Dictionary containing all metadata for the dataset, including feature names, trajectory information, and device settings.
  • datafile::String: Path to the data file (usually .h5 or .jld2 format).
  • lock::ReentrantLock: Lock for thread-safe access to the datafile during concurrent operations.
source

Model Configuration

GraphNetSim.ModelConfig — Type
ModelConfig

Persists the minimal set of parameters required to reconstruct the GNN model from a checkpoint without re-specifying them at the call site.

The architecture fields (mps, layer_size, hidden_layers, bounded) affect weight shapes and must match on resume; bounded additionally must agree with the dataset's meta["bounds"] presence at load time (a bounded model has a wider encoder input). The remaining fields are derived from meta.json or the JLD2 checkpoint and are saved as documentation; they may legitimately differ between training phases.

Fields

  • mps: Number of message passing steps.
  • layer_size: Hidden layer width.
  • hidden_layers: Number of hidden layers per MLP block.
  • norm_steps: Steps before weight updates start (online normalizer warm-up).
  • types_updated: Node types whose outputs are predicted.
  • types_noisy: Node types receiving noise injection during training.
  • noise_stddevs: Per-type noise standard deviations.
  • norm_type: Normalization strategy for Float32 features (:online, :minmax, :meanstd).
  • bounded: Whether the model was trained with the boundary (wall-distance) node feature, i.e. whether the training dataset defined meta["bounds"]. Part of the architecture: a bounded model has a wider encoder input than an unbounded one, so this must match the dataset at load time.
source
GraphNetSim.save_model_config — Function
save_model_config(cfg, cp_path)

Write cfg to cp_path/model_config.json.

On the first call the file is created. On subsequent calls the architecture fields (mps, layer_size, hidden_layers) are validated against the saved values — a mismatch with an existing checkpoint would cause a weight-shape incompatibility and is therefore an error. Training fields are updated silently, as they may change between training phases.

source
GraphNetSim.load_model_config — Function
load_model_config(cp_path) -> Union{ModelConfig, Nothing}

Read cp_path/model_config.json and return a ModelConfig, or nothing if the file does not exist. Returns nothing (with a warning) if the file exists but cannot be parsed, preserving backwards compatibility with checkpoints that predate this feature.

source

Dataset Constructors

GraphNetSim.Dataset — Method
Dataset(datafile::String, metafile::String, args)

Construct a Dataset from separate data and metadata files.

Validates that both files exist and have correct formats (.h5 or .jld2 for data, .json for metadata), then loads trajectories and merges metadata with provided arguments.

Arguments

  • datafile::String: Path to the data file (.h5 or .jld2 containing trajectory data).
  • metafile::String: Path to the metadata file (.json containing dataset configuration).
  • args: A structure or object whose fields will be merged into the metadata dictionary.

Returns

  • Dataset: A new Dataset object initialized with the provided data and metadata.

Throws

  • ArgumentError: If datafile or metafile do not exist or have invalid formats.
source
GraphNetSim.Dataset — Method
Dataset(split::Symbol, path::String, args)

Construct a Dataset by specifying a split type and directory path.

Locates and loads the appropriate data file based on the split type (:train, :valid, or :test), expecting a "meta.json" file in the given directory.

Arguments

  • split::Symbol: The dataset split, one of :train, :valid, or :test.
  • path::String: Directory path containing the metadata file "meta.json" and the corresponding data file.
  • args: A structure or object whose fields will be merged into the metadata dictionary.

Returns

  • Dataset: A new Dataset object initialized with data from the specified split.

Throws

  • ArgumentError: If split is invalid, if meta.json is not found, or if the corresponding data file cannot be found.
source
GraphNetSim.get_file — Function
get_file(split::Symbol, path::String)

Locate the data file corresponding to a dataset split in the given directory.

Attempts to find a .jld2 file first, then a .h5 file with the split name (e.g., "train.jld2" or "train.h5").

Arguments

  • split::Symbol: The dataset split name (converted to string).
  • path::String: Directory path to search for the data file.

Returns

  • String: Full path to the located data file.

Throws

  • ArgumentError: If no data file with the specified split name is found in the directory.
source

Training and Evaluation

Main Training Function

GraphNetSim.train_network — Function
train_network(opt, ds_path::String, cp_path::String; kws...)

Train a Graph Neural Network simulator on trajectory data.

Initializes a graph network, loads dataset, computes normalization statistics, and performs supervised training using the specified training strategy. Validates periodically on validation set and saves checkpoints for the best model.

Arguments

  • opt: Optimizer configuration (e.g., Optimisers.Adam(1f-4)).
  • ds_path::String: Path to dataset directory (must contain train, valid, test splits).
  • cp_path::String: Path where checkpoints and logs are saved.

Keyword Arguments

  • mps::Int=15: Number of message passing steps in the network.
  • layer_size::Int=128: Latent dimension of hidden MLP layers.
  • hidden_layers::Int=2: Number of hidden layers in each MLP.
  • batchsize::Int=1: Batch size for training (default uses full trajectories).
  • epochs::Int=1: Number of training epochs.
  • steps::Int=10e6: Total number of training steps.
  • checkpoint::Int=10000: Create checkpoint every N steps.
  • norm_steps::Int=1000: Steps for accumulating normalization statistics without updates.
  • types_updated::Vector{Int}=[1]: Node types whose features are predicted.
  • types_noisy::Vector{Int}=Int[]: Node types to which noise is added.
  • noise_stddevs::Vector{Float32}=[0.0f0]: Standard deviations for Gaussian noise.
  • training_strategy::TrainingStrategy=DerivativeTraining(): Training method to use.
  • use_cuda::Bool=true: Use CUDA GPU if available.
  • solver_valid::OrdinaryDiffEqAlgorithm=Tsit5(): ODE solver for validation rollouts.
  • solver_valid_dt::Union{Nothing,Float32}=nothing: Fixed timestep for validation (if set).
  • optimizer_learning_rate_start::Float32=1.0f-4: Initial learning rate.
  • optimizer_learning_rate_stop::Union{Nothing,Float32}=nothing: Final learning rate for decay schedule.
  • show_progress_bars::Bool=true: Show training progress bars.
  • use_valid::Bool=true: Use validation checkpoint for early stopping.

Returns

  • Float32: Minimum validation loss achieved during training.

Training Strategies

  • DerivativeTraining: Train on current step derivatives (collocation)
  • BatchingStrategy: Custom batching of trajectory segments

Example #TODO add example with different training strategies

train_network(
    Optimisers.Adam(1f-4),
    "./data",
    "./checkpoints";
    epochs=10,
    steps=50000,
    mps=15,
    layer_size=128,
    training_strategy=DerivativeTraining()
)
source

Internal Training Functions

GraphNetSim.train_gns! — Function
train_gns!(gns::GraphNetwork, ds_train::Dataset, ds_valid::Dataset, df_train, df_valid, device::Function, cp_path::String, args::Args)

Execute the main training loop for a Graph Neural Network simulator.

Performs supervised learning with periodic validation, checkpoint saving, and learning rate scheduling. Supports various training strategies (derivative, batching) and handles feature noise injection, gradient accumulation over multiple time steps, and online normalizer updates.

Arguments

  • gns::GraphNetwork: Graph network model containing parameters, optimiser state, and normalizers.
  • ds_train::Dataset: Training dataset with trajectories and metadata.
  • ds_valid::Dataset: Validation dataset for monitoring training progress.
  • df_train: DataFrame storing training loss at checkpoints.
  • df_valid: DataFrame storing best validation losses.
  • device::Function: Device placement function (cpudevice or gpudevice).
  • cp_path::String: Path for saving checkpoints and logs.
  • args::Args: Configuration including optimizer, strategy, and training parameters.

Algorithm

For each training step:

  1. Prepare input data and compute node features
  2. For each time step in trajectory:
    • Build computation graph with neighborhood connections
    • Forward pass through network
    • Compute loss depending on training strategy
    • Accumulate gradients
  3. Update parameters after accumulation window
  4. Optionally decay learning rate
  5. Every N steps: validate on full trajectories and save checkpoint if improved

Returns

  • Float32: Minimum validation loss achieved.

Notes

  • First norm_steps steps only accumulate normalization statistics
  • Validation using long trajectory rollouts occurs at checkpoint intervals
  • Checkpoints saved to cp_path/valid when validation loss improves
  • Final checkpoint always saved at cp_path
source

Normalization Setup

GraphNetSim.calc_norms — Function
calc_norms(dataset::Dataset, device::Function, args::Args)

Initialize and compute feature normalizers from dataset statistics.

Computes normalization statistics for edge features, node features, and output features based on metadata specifications. Supports offline min/max and mean/std normalization, boolean encoding, one-hot encoded features, and online accumulation strategies.

Arguments

  • dataset::Dataset: Dataset object containing feature metadata and specifications.
  • device::Function: Device placement function (cpudevice or gpudevice).
  • args::Args: Configuration struct with norm_steps for online normalizer accumulation.

Returns

  • Tuple: (quantities, enorms, nnorms, o_norms)
    • quantities::Int: Total number of input feature dimensions
    • e_norms: Dictionary or single normalizer for edge features
    • n_norms::Dict: Dictionary mapping node feature names to normalizers
    • o_norms::Dict: Dictionary mapping output feature names to normalizers

Normalizer Types

  • NormaliserOfflineMinMax: Fixed min/max normalization with learnable target range
  • NormaliserOfflineMeanStd: Fixed mean/standard deviation normalization
  • NormaliserOnline: Accumulates statistics online during training

Notes

  • One-hot encoded Int32 features are expanded to multiple dimensions
  • Boolean features are mapped to [0.0, 1.0] range
  • Distance features add dimensions for domain boundary constraints
source

Main Evaluation Function

GraphNetSim.eval_network — Function
eval_network(ds_path::String, cp_path::String, out_path::String, solver=nothing; start, stop, dt=nothing, saves, mse_steps, kws...)

Evaluate a trained Graph Neural Network simulator on test trajectories.

Loads a trained network from checkpoint, performs long-term trajectory rollouts, computes error metrics against ground truth, and saves results to disk.

Arguments

  • ds_path::String: Path to dataset directory containing test split.
  • cp_path::String: Path to checkpoint directory (contains model parameters).
  • out_path::String: Path where evaluation results are saved.
  • solver: ODE solver for long-term predictions (e.g., Tsit5()).

Keyword Arguments

  • start::Real: Start time for evaluation.
  • stop::Real: End time for evaluation.
  • saves::AbstractVector: Time points where solution is saved.
  • mse_steps::AbstractVector: Time points where error metrics are computed.
  • dt::Union{Nothing,Real}=nothing: Fixed timestep for solver (if applicable).
  • mps::Int=15: Number of message passing steps (must match training config).
  • layer_size::Int=128: Hidden layer size (must match training config).
  • hidden_layers::Int=2: Number of hidden layers (must match training config).
  • types_updated::Vector{Int}=[1]: Updated node types (must match training config).
  • use_cuda::Bool=true: Use CUDA GPU if available.
  • use_valid::Bool=true: Load from best validation checkpoint instead of final checkpoint.

Output

Saves results to out_path/{solver_name}/trajectories.h5:

  • Ground truth positions, velocities, accelerations
  • Predicted positions, velocities, accelerations
  • Prediction errors for each trajectory

Example

eval_network(
    "./data",
    "./checkpoints",
    "./results";
    solver=Tsit5(),
    start=0.0f0,
    stop=1.0f0,
    dt=0.01f0,
    saves=0.0:0.01:1.0,
    mse_steps=0.0:0.1:1.0
)
source
GraphNetSim.eval_network! — Function
eval_network!(solver, gns::GraphNetwork, ds_test::Dataset, device::Function, out_path::String, start::Real, stop::Real, dt, saves, mse_steps, args::Args)

Perform evaluation loops and trajectory rollouts for all test samples.

Executes long-term predictions for each test trajectory, computes performance metrics relative to ground truth, and saves trajectories and errors to HDF5 format.

Arguments

  • solver: ODE solver for trajectory rollouts (or nothing for collocation).
  • gns::GraphNetwork: Trained graph network model.
  • ds_test::Dataset: Test dataset with trajectories.
  • device::Function: Device placement function.
  • out_path::String: Output directory for results.
  • start::Real: Evaluation start time.
  • stop::Real: Evaluation end time.
  • dt: Fixed timestep (or nothing for adaptive).
  • saves: Time points to save solution.
  • mse_steps: Time points to compute errors.
  • args::Args: Configuration parameters.

Algorithm

For each test trajectory:

  1. Extract initial conditions from data
  2. Create computation graph with graph network
  3. Roll out trajectory using ODE solver for specified duration
  4. Extract position, velocity, and acceleration from solution
  5. Compute mean squared error against ground truth
  6. Report cumulative error at specified time points

Returns

  • Tuple: (traj_ops, errors)
    • traj_ops::Dict: Dictionary of trajectories with ground truth and predictions
    • errors::Dict: Squared errors for each trajectory

Output Files

Creates {out_path}/{solver_name}/trajectories.h5 containing:

  • Ground truth and predicted trajectories
  • Error fields for each time step
  • Properly indexed for easy post-processing
source

Extrapolation Beyond Ground Truth

GraphNetSim.extrapolate_network — Function
extrapolate_network(ds_path, cp_path::String, out_path::String, solver;
                    start, stop, dt, saves,
                    mse_steps=Float32[], has_ground_truth::Bool=true, kws...)

Evaluate a trained network by rolling out trajectories that may extend past the ground truth horizon (or run with no ground truth at all).

Unlike eval_network, the user-supplied start, stop, dt, saves, and mse_steps are never overridden by values derived from the test trajectories. This lets callers integrate further than (trajectory_length - 1) * dt.

Arguments

  • ds_path: Path to the dataset (test split is used as the source of initial conditions).
  • cp_path::String: Checkpoint directory containing the trained model.
  • out_path::String: Output directory for the HDF5 result file.
  • solver: ODE solver (e.g. Tsit5(), Euler()) or nothing for collocation.
  • start: Evaluation start time; the initial condition is taken from the test trajectory at frame round(Int, (start/dt) + 1).
  • stop: Evaluation end time (may exceed (trajectory_length - 1) * dt).
  • dt: Fixed timestep. Must equal the dataset's native dt.
  • saves: Time points at which the ODE solution is saved.
  • mse_steps=Float32[]: Time points for MSE reporting. Steps beyond the ground-truth horizon are skipped automatically.
  • has_ground_truth::Bool=true: When false, all ground-truth reads, MSE reporting, and the gt/ HDF5 group are skipped.

HDF5 Output

Written to {out_path}/{solver_name}/trajectories.h5:

trajectory_N/
  timesteps: Npred
  prediction/
    pos[1..Npred], vel[1..Npred], acc[1..Npred]
    err[1..Npred]          # NaN32-filled for k > Noverlap (only when has_ground_truth)
  gt/                      # omitted when has_ground_truth=false or no overlap
    pos[1..Noverlap], vel[1..Noverlap], acc[1..Noverlap]

Noverlap = min(Npred, trajectory_length - stepstart + 1) is the number of saved steps covered by ground truth.

Notes

  • Online feature normalisers are frozen for the duration of the rollout so the network's out-of-distribution predictions do not corrupt the stored statistics.
  • Integrator stability past the training horizon is not guaranteed — extrapolated rollouts should be treated as qualitative unless independently validated.
  • Visualising the gt/ subgroup of an extrapolated trajectory can throw past Noverlap because the top-level timesteps field equals Npred. Use eval_network when you need to visualise ground truth.
source
GraphNetSim.extrapolate_network! — Function
extrapolate_network!(solver, gns::GraphNetwork, ds_test::Dataset, device::Function,
                     out_path, start, stop, dt, saves, mse_steps, args::Args;
                     has_ground_truth::Bool=true)

Inner evaluation loop for extrapolate_network. Mirrors eval_network! but (a) by default never overrides the user-supplied time-grid arguments, (b) tolerates predictions longer than the ground truth (or missing ground truth entirely), and (c) freezes online feature normalisers during the rollout to prevent out-of-distribution drift.

Per-trajectory horizons

When extrapolate_factor is set (e.g. 2.0), the global stop/saves are ignored and each trajectory instead rolls out to extrapolate_factor times its OWN ground-truth time window: a trajectory of L frames (GT window dt*(L-1)) is simulated over [start, start + f*dt*(L-1)], i.e. round(Int, f*(L-1)) + 1 frames. This is the correct setting for a dataset of varying-length trajectories, where a single global grid would extrapolate short trajectories far past 2× while barely extrapolating the longest. mse_steps is still matched per trajectory against its own overlap region.

Returns

  • Tuple: (predictions::Dict{Int,NamedTuple}, errors::Dict{Int,Array{Float32,3}}). errors is empty when has_ground_truth=false.
source

Training Strategies

Abstract Base Type

GraphNetSim.prepare_training — Function
prepare_training(strategy)

Function that is executed once before training. Can be overwritten by training strategies if necessary.

Arguments

  • strategy: Used training strategy.

Returns

  • Tuple containing the results of the function.
source
GraphNetSim.get_delta — Function
get_delta(strategy, trajectory_length)

Returns the delta between samples in the training data.

Arguments

  • strategy: Used training strategy.
  • Trajectory length (used for Derivative strategies).

Returns

  • Delta between samples in the training data.
source
get_delta(::SolverStrategy, ::Integer)

Returns the delta (step size) for solver-based training strategies.

For most solver-based strategies, returns 1 (advancing by single timestep). Can be overridden by specific strategies (e.g., BatchingStrategy).

Arguments

  • strategy::SolverStrategy: Solver-based training strategy.
  • trajectory_length::Integer: Length of the trajectory (unused in base implementation).

Returns

  • Integer delta between training samples.
source
get_delta(strategy::BatchingStrategy, ::Integer)

Returns the number of steps per batch.

Arguments

  • strategy::BatchingStrategy: BatchingStrategy instance.
  • Unused trajectory length parameter.

Returns

  • Integer: Number of steps per batch (strategy.steps).
source
get_delta(strategy::DerivativeStrategy, trajectory_length)

Returns the effective trajectory length for derivative training.

If strategy windowsize > 0 and smaller than trajectorylength, returns windowsize. Otherwise returns the full trajectorylength.

Arguments

  • strategy::DerivativeStrategy: Derivative-based strategy.
  • trajectory_length::Integer: Length of the trajectory.
source
GraphNetSim.init_train_step — Function
init_train_step(strategy, t)

Function that is executed before each training sample.

Arguments

  • strategy: Used training strategy.
  • t: Tuple containing the variables necessary for initializing training.
  • ta: Tuple with additional variables that is returned from prepare_training.

Returns

source
init_train_step(strategy::SolverStrategy, t::Tuple)

Initializes a training step for solver-based strategies.

Extracts initial conditions, packs state into ComponentArray format, and prepares ground truth data for ODE problem setup.

Arguments

  • strategy::SolverStrategy: Solver-based training strategy.
  • t::Tuple: Input tuple containing (gns, data, position, velocity, meta, outputfields, targetfields, node_type, mask, device, ...).

Returns

  • Tuple: Initialized data for training step.
source
init_train_step(strategy::BatchingStrategy, t::Tuple)

Initializes a training step for the BatchingStrategy.

Receives the scheduler-selected batch index from the caller, extracts initial conditions for that time window, packs state into ComponentArray, and prepares ground truth data.

Arguments

  • strategy::BatchingStrategy: BatchingStrategy instance.
  • t::Tuple: Input tuple (gns, data, meta, outputfields, targetfields, nodetype, mask, valmask, device, batchindex, batches, showprogress_bars).

Returns

  • Tuple: Initialized batch data for training step.
source
init_train_step(strategy::DerivativeStrategy, t::Tuple)

Initializes a training step for derivative-based strategies.

Extracts target derivatives at a single datapoint and normalizes using network feature normalizers.

Arguments

  • strategy::DerivativeStrategy: Derivative-based strategy.
  • t::Tuple: Input tuple with network, data, and sampling information.
source
GraphNetSim.train_step — Function
train_step(strategy, t)

Performs a single training step and return the resulting gradients and loss.

Arguments

  • strategy: Solver strategy that is used for training.
  • t: Tuple that is returned from init_train_step.

Returns

  • Gradients for optimization step.
  • Loss for optimization step.
source
train_step(strategy::SolverStrategy, t::Tuple)

Performs one training step for solver-based strategies.

Constructs an ODE problem from the GNS model, solves it using the strategy's solver, and computes gradients via sensitivity analysis (adjoint method).

Arguments

  • strategy::SolverStrategy: Solver-based training strategy.
  • t::Tuple: Initialized data from init_train_step().

Returns

  • Tuple: (gradients, loss) - Gradients for optimization and scalar training loss.

Algorithm

  1. Create ODE right-hand side function using ode_func_train().
  2. Setup ODE problem with initial conditions and parameters.
  3. Compute loss via train_loss().
  4. Backpropagate through ODE solver using sensitivity algorithm.
  5. Return gradients and loss.
source
train_step(strategy::BatchingStrategy, t::Tuple)

Performs one training step for the BatchingStrategy.

Constructs an ODE problem for the selected batch, solves it using the strategy's solver, and computes gradients via sensitivity analysis. Updates batch loss.

Arguments

  • strategy::BatchingStrategy: BatchingStrategy instance.
  • t::Tuple: Data tuple from inittrainstep().
source
train_step(strategy::DerivativeStrategy, t::Tuple)

Performs one training step for derivative-based strategies.

Evaluates network on graph at a single timepoint and computes loss against target derivatives. Computes gradients via backpropagation.

Arguments

  • strategy::DerivativeStrategy: Derivative-based strategy.
  • t::Tuple: Data tuple from inittrainstep().
source
GraphNetSim.validation_step — Function
validation_step(strategy, t)

Performs validation of a single trajectory. Should be overwritten by training strategies to determine simulation and data interval before calling the inner function _validation_step.

Arguments

  • strategy: Type of training strategy (used for dispatch).
  • t: Tuple containing the variables necessary for validation.

Returns

source
validation_step(strategy::SolverStrategy, t::Tuple)

Validation step for solver-based strategies.

Computes validation loss by rolling out the GNS model over the full validation trajectory and comparing predicted outputs with ground truth.

Arguments

  • strategy::SolverStrategy: Solver-based training strategy.
  • t::Tuple: Validation data tuple containing (gns, data, meta, ...).

Returns

  • Float32: Validation loss (mean squared error).
source
validation_step(strategy::DerivativeStrategy, t::Tuple)

Validation step for derivative-based strategies.

Computes validation loss by rolling out GNS model over trajectory window and comparing derivatives with ground truth.

Arguments

  • strategy::DerivativeStrategy: Derivative-based strategy.
  • t::Tuple: Validation data tuple.
source
GraphNetSim._validation_step — Function
_validation_step(t, sim_interval, data_interval)

Inner function for validation of a single trajectory.

Arguments

  • t: Tuple containing the variables necessary for validation.
  • sim_interval: Interval that determines the simulated time for the validation.
  • data_interval: Interval that determines the indices of the timesteps in ground truth and prediction data.

Returns

  • Loss calculated on the difference between ground truth and prediction (via mse).
  • Ground truth data with data_interval as timesteps.
  • Prediction data with data_interval as timesteps.
source
GraphNetSim.batchTrajectory — Function
batchTrajectory(strategy::BatchingStrategy, data::Dict)

Partitions a trajectory into time intervals (batches) for sequential training.

Divides the full trajectory duration into equal-sized time intervals, creating one Batch object per interval. Used for memory-efficient training on long sequences.

Arguments

  • strategy::BatchingStrategy: Batching strategy with interval specifications.
  • data::Dict: Data dictionary containing "dt" (timestep) and "trajectory_length".

Returns

  • Vector{Batch}: Array of Batch objects partitioning the trajectory.
source

Concrete Strategies

GraphNetSim.SingleShooting — Type
SingleShooting(tstart, dt, tstop, solver; sense = InterpolatingAdjoint(autojacvec = ZygoteVJP()), solargs...)

The default solver based training that is normally used for NeuralODEs. Simulates the system from tstart to tstop and calculates the loss based on the difference between the prediction and the ground truth at the timesteps tstart:dt:tstop.

Arguments

  • tstart: Start time of the simulation.
  • dt: Interval at which the simulation is saved.
  • tstop: Stop time of the simulation.
  • solver: Solver that is used for simulating the system.

Keyword Arguments

  • sense: The sensitivity algorithm used for calculating the sensitivities. Defaults to _default_solver_sense(solver) (InterpolatingAdjoint with checkpointing on for adaptive solvers, off for fixed-timestep solvers such as Euler()).
  • solargs: Keyword arguments that are passed on to the solver.
source
GraphNetSim.MultipleShooting — Type
MultipleShooting(tstart, dt, tstop, solver, interval_size, continuity_term = 100; sense = InterpolatingAdjoint(autojacvec = ZygoteVJP(), checkpointing = true), solargs...)

Similar to SingleShooting, but splits the trajectory into intervals that are solved independently and then combines them for loss calculation. Useful if the network tends to get stuck in a local minimum if SingleShooting is used.

Arguments

  • tstart: Start time of the simulation.
  • dt: Interval at which the simulation is saved.
  • tstop: Stop time of the simulation.
  • solver: Solver that is used for simulating the system.
  • interval_size: Size of the intervals (i.e. number of datapoints in one interval).
  • continuity_term = 100: Factor by which the error between points of concurrent intervals is multiplied.

Keyword Arguments

  • sense: The sensitivity algorithm used for calculating the sensitivities. Defaults to _default_solver_sense(solver) (InterpolatingAdjoint with checkpointing on for adaptive solvers, off for fixed-timestep solvers such as Euler()).
  • solargs: Keyword arguments that are passed on to the solver.
source
GraphNetSim.DerivativeTraining — Type
struct DerivativeTraining <: DerivativeStrategy

Derivative-based training strategy using finite-difference ground truth.

Compares network output with finite-difference derivatives from data. Faster than solver-based training, useful for initial model training. Supports temporal windowing, optional random shuffling, and a pluggable per-pass Scheduler.

source
GraphNetSim.BatchingStrategy — Type
struct BatchingStrategy <: SolverStrategy

Solver-based training strategy that batches long trajectories into segments.

Divides long trajectories into time intervals and solves/trains on each segment independently. Useful for memory-efficient training on long sequences.

source

Default Sensitivity Algorithm

BatchingStrategy, SingleShooting and MultipleShooting default their sense keyword to the algorithm returned here.

GraphNetSim._default_solver_sense — Function
_default_solver_sense(solver)

Default sensitivity algorithm for solver-based strategies: InterpolatingAdjoint(autojacvec=ZygoteVJP(), checkpointing=false).

checkpointing=true re-solves the forward problem between checkpoints during the reverse pass, which breaks in two ways for the short-interval solves these strategies use:

  • Fixed-step solvers (e.g. Euler()): the internal re-solve forwards neither dt nor tstops (see SciMLSensitivity interpolating_adjoint.jl), so it throws ArgumentError: Fixed timestep methods require a choice of dt or choosing the tstops.
  • Adaptive solvers (e.g. Tsit5()) over short batches: when a checkpoint segment's forward solve holds a single time point, the reverse pass evaluates dt = abs(cpsol_t[end] - cpsol_t[end - 1]) and hits a BoundsError at index [0] (interpolating_adjoint.jl).

Disabling checkpointing skips that re-solve entirely — the forward solve's dt reaches the backward adjoint solve and the full forward trajectory is stored — so every solver works. Pass an explicit sense=InterpolatingAdjoint(...; checkpointing=true) to opt back into checkpointing when batches are long enough and GPU memory is tight.

source

Schedulers

Per-pass selection rules used by DerivativeTraining and BatchingStrategy to choose which training unit (timestep or batch) is processed at each inner iteration of a trajectory pass.

GraphNetSim.Scheduler — Type
abstract type Scheduler

Per-pass selection rule shared across training strategies that iterate over a pool of discrete training units (timesteps for DerivativeTraining, batches for BatchingStrategy). Each scheduler implements:

  • total_iters(sched, n_units): outer-loop iterations per trajectory pass. The training loop may still override this via strategy-level outer_iters (e.g. BatchingStrategy uses its strategy.steps field).
  • next_index(sched, i, n_units, losses, norm_active): unit index for iteration i (1-based).
  • update_state!(sched, i, actual, loss, n_units, losses): hook after the training step completes; may mutate the per-unit losses buffer.
  • tracks_losses(sched): whether the training loop must allocate the per-unit losses buffer before the trajectory pass begins.

See Sequential, WorstLoss, and UniqueWorst for the built-in schedulers.

source

Stateless and loss-buffer schedulers

GraphNetSim.Sequential — Type
Sequential()

Linear-cycle scheduler: picks unit ((i-1) % n_units) + 1. For strategies whose outer-iteration budget equals n_units (e.g. DerivativeTraining without reruns) this is a single ordered pass; for strategies whose budget exceeds n_units it wraps around cyclically. No reruns, no loss tracking.

source
GraphNetSim.Shuffled — Type
Shuffled()

Random-permutation scheduler: each pass over the unit pool visits every unit exactly once, but in a fresh random order. A new permutation is drawn at the start of every cycle (whenever the cyclic position resets to 1), so a strategy whose budget exceeds n_units re-shuffles on each wrap-around.

Like Sequential it performs no reruns and tracks no losses; it differs only in ordering, which decorrelates consecutive gradient steps without biasing any unit's visit count. Safe for every training strategy and unaffected by the norm-accumulation window (the training loop bypasses the scheduler there).

source
GraphNetSim.WorstLoss — Type
WorstLoss(rerun_steps)

After the base pass, take rerun_steps extra gradient steps on argmax(losses_per_unit). The same unit can be selected repeatedly as long as it remains the worst; each visit overwrites the recorded loss.

When paired with an Inf32-initialised losses buffer, the base pass reduces to argmax too — this reproduces BatchingStrategy's historical "uncomputed-first-else-highest-loss" curriculum exactly.

source
GraphNetSim.UniqueWorst — Type
UniqueWorst(rerun_steps)

Like WorstLoss, but each rerun-phase pick sets the selected entry to -Inf32 so it cannot be picked again in the same trajectory pass. Total rerun budget is clamped to min(rerun_steps, n_units) so candidates are never exhausted mid-pass.

source
GraphNetSim.WeightedWorst — Type
WeightedWorst(rerun_steps; temperature=1.0f0)

Stochastic loss-aware scheduler. After the base pass, take rerun_steps extra gradient steps, each on a unit sampled with probability proportional to its loss raised to 1/temperature:

  • temperature → 0 concentrates on the single worst unit (≈ WorstLoss).
  • temperature == 1 samples in direct proportion to loss.
  • temperature → ∞ approaches uniform sampling.

Unlike WorstLoss, attention is spread across several high-loss units instead of fixating on the current argmax, which can avoid over-fitting a single hard unit. Non-finite (uncomputed) entries are always selected first via argmax, preserving the Inf32-init "uncomputed-first" invariant.

source
GraphNetSim.TopKWorst — Type
TopKWorst(rerun_steps, k)

After the base pass, distribute rerun_steps extra gradient steps round-robin across the current k worst-loss units rather than repeatedly hammering the single argmax (WorstLoss) or visiting each worst unit only once (UniqueWorst). The top-k set is recomputed from the live losses buffer on every rerun pick, so it tracks the worst units as their losses change.

k is clamped to 1:n_units. With k == 1 this reduces to WorstLoss; with k == n_units and rerun_steps == n_units it sweeps every unit once more in worst-first order. Non-finite (uncomputed) entries sort first, preserving the Inf32-init "uncomputed-first" invariant.

source
GraphNetSim.EpsilonGreedy — Type
EpsilonGreedy(rerun_steps; epsilon=0.1f0)

ε-greedy loss-aware scheduler. After the base pass, take rerun_steps extra gradient steps; each picks the current worst unit (argmax) with probability 1 - epsilon ("exploit") and a uniformly random unit with probability epsilon ("explore"). This keeps WorstLoss's focus on the hardest unit while periodically sampling elsewhere to avoid starving units that the greedy rule never revisits.

epsilon is clamped to [0, 1]; epsilon == 0 recovers WorstLoss. Non-finite (uncomputed) entries are selected first via argmax, preserving the Inf32-init "uncomputed-first" invariant.

source
GraphNetSim.PercentileWorst — Type
PercentileWorst(rerun_steps; q=0.9f0)

Robust top-quantile scheduler. Each rerun samples uniformly among the units whose loss is at or above the q-quantile of the current losses buffer. Unlike TopKWorst's fixed-k set, the candidate pool adapts to the loss distribution — widening when many units are equally bad and narrowing to the true tail when one dominates — which is less sensitive to a single outlier than argmax. Stateless; tracks losses. Non-finite (uncomputed) entries are selected first.

source

Stateful schedulers (per-pass state)

These keep per-unit state in mutable fields and rebuild it at the start of every trajectory pass, mirroring the training loop's fresh losses buffer.

GraphNetSim.UCB — Type
UCB(rerun_steps=0; c=1.0f0)

Upper-confidence-bound bandit scheduler. After the base pass (which visits each unit once and seeds its running mean loss), each rerun picks the unit maximising mean_loss + c·sqrt(ln t / visits). The exploration bonus grows for rarely-visited units, so — unlike WorstLoss — no unit is starved indefinitely; c trades exploitation (0 ≈ greedy mean loss) against exploration. Per-pass state: visit counts and loss sums, reset each trajectory.

source
GraphNetSim.LearningProgress — Type
LearningProgress(rerun_steps=0)

Prioritise units by learning progress — the absolute change in loss between consecutive visits, |loss - prev_loss| — rather than by loss level. A unit whose loss is high but stuck is deprioritised relative to one that is actively moving, focusing reruns where gradient steps still pay off. Units with only one sample so far carry Inf32 progress, so each is revisited at least once to establish a delta (preserving the uncomputed-first invariant). Per-pass state: previous-loss and progress buffers, reset each trajectory.

source
GraphNetSim.Staleness — Type
Staleness(rerun_steps=0; weight=1.0f0)

Recency-aware anti-starvation scheduler. Each rerun picks argmax(loss + weight · steps_since_last_visit), so high-loss units are still favoured but any unit left untouched long enough eventually wins regardless of its loss. weight tunes the trade-off: 0 recovers WorstLoss, large values approach round-robin coverage. A simpler, single-knob alternative to UCB. Per-pass state: per-unit last-visit timestamps, reset each trajectory.

source

Annealing schedulers (global step counter)

These keep a step counter across the whole run instead of resetting per pass, so their behaviour shifts as training progresses.

GraphNetSim.ProgressiveHorizon — Type
ProgressiveHorizon(rerun_steps=0; warmup_frac=0.25f0, growth_steps=1000)

Time-horizon curriculum. Restricts every selection to a growing prefix 1:h of the unit pool, where h = ceil(n_units · frac) and frac ramps linearly from warmup_frac to 1 over the first growth_steps active gradient steps. Within the prefix it cycles linearly (((i-1) % h) + 1).

For autoregressive simulators this learns near-term dynamics before long horizons — the same intuition behind multiple-shooting, applied as a sampling curriculum. Most natural for BatchingStrategy, where units are consecutive time intervals, but valid for any strategy. No loss tracking; stateful in the global step counter only.

source
GraphNetSim.AnnealedWeighted — Type
AnnealedWeighted(rerun_steps=0; t0=2.0f0, t1=0.1f0, decay_steps=1000)

Temperature-annealed variant of WeightedWorst. Reruns sample a unit with probability ∝ loss^(1/T), where the temperature T decays linearly from t0 (exploratory, near-uniform) to t1 (greedy, near-argmax) over the first decay_steps active gradient steps. Mirrors simulated annealing: spread attention early, sharpen onto the worst units late. Reuses WeightedWorst's sampler; stateful in the global step counter only.

source

Gradient-pooling scheduler

Pools a window of neighbouring timesteps into a single gradient step. Supported by DerivativeTraining only; BatchingStrategy rejects it because it already pools temporally over its ODE interval.

GraphNetSim.TemporalWindow — Type
TemporalWindow(rerun_steps; radius=2)

Rerun-phase scheduler that pools the loss over the ±radius temporal neighbours of the worst timestep. Each rerun iteration runs 2*radius + 1 GNN forward passes and one pooled backward pass — a single gradient update informed by the whole hard region rather than a single timestep.

The scalar pick is identical to WorstLoss: a linear base pass (which, with the Inf32-initialised losses buffer, preserves the "uncomputed-first" invariant) followed by argmax(losses) reruns. The pooling happens through the window_indices hook, which the training loop expands into a multi-forward window only during the rerun phase — the base pass and the norm-accumulation window stay length-1, so initial-pass throughput is unchanged.

Naming vs. `DerivativeTraining.window_size`

window_size is the trajectory-slicing field on DerivativeTraining (how many timesteps per trajectory). TemporalWindow's radius is the neighbourhood half-width around the argmax pick during a rerun. They are unrelated concepts and must not be confused.

Buffer semantics

After a windowed rerun, losses[centre] holds the window-pooled mean, not the single-step loss at centre; neighbour entries are untouched. Writing only the centre index is the cheapest way to advance the argmax state without a second no-grad forward per member.

Combination guards

Requires sequential timesteps, so only supported on DerivativeTraining(; random=false, …) — with random=true the trajectory arrays are shuffled in-place and index neighbours no longer correspond to adjacent physics steps (the constructor throws via requires_sequential). Not supported on BatchingStrategy (rejected via pools_in_inner_loop); that strategy already pools temporally through its ODE interval.

source
GraphNetSim.window_indices — Function
window_indices(sched, centre, n_units)

Training units pooled into one gradient step when sched selects centre. Defaults to the singleton (centre,); only pooling schedulers (e.g. TemporalWindow) override it to return a wider neighbourhood.

source

Normalization and Data Statistics

Computing Normalization Statistics

GraphNetSim.data_minmax — Function
data_minmax(path; kws...)

Calculates the minimum and maximum values for each numeric feature across all dataset partitions.

Iterates through training, validation, and test datasets to compute global min/max bounds for all numeric features (Int32 and Float32 types).

Arguments

  • path: Path to the dataset files.

Keyword Arguments

All keyword arguments are forwarded to Args to configure dataset loading (e.g. types_updated, types_noisy, noise_stddevs).

Returns

  • Dict: Dictionary mapping feature names to [min, max] value pairs computed from all datasets.
source
GraphNetSim.data_meanstd — Function
data_meanstd(path; kws...)

Calculates the mean and standard deviation for each feature in the given part of the dataset.

Arguments

  • path: Path to the dataset files.

Keyword Arguments

All keyword arguments are forwarded to Args to configure dataset loading (e.g. types_updated, types_noisy, noise_stddevs).

Returns

  • Mean and standard deviation in training, validation and test set
source
GraphNetSim.der_minmax — Function
der_minmax(path)

Calculates the minimum and maximum across training, validation, and test sets for each numeric feature.

Combines results from both training/validation and test data to compute overall min/max bounds.

Arguments

  • path: Path to the dataset files.

Returns

  • Dict: Dictionary mapping feature names to [min, max] value pairs across all datasets.
source
GraphNetSim.update_meta! — Function
update_meta!(path, norm_type)

Compute normalization statistics and write them into the dataset's meta.json.

Calls data_minmax or data_meanstd depending on norm_type, then updates the per-feature entries in meta.json with the computed statistics. Conflicting statistics from a different normalization type are removed.

Arguments

  • path::String: Dataset directory containing meta.json, train.h5, valid.h5, test.h5.
  • norm_type::Symbol: One of :online, :minmax, or :meanstd. :online is a no-op (no precomputed statistics needed).

Returns

  • String: Path to the written meta.json file.
source

Graph Construction and ODE Solving

Building Computation Graphs

GraphNetSim.build_graph — Function
build_graph(gns::GraphNetCore.GraphNetwork, data::Dict{String,Any}, datapoint::Integer, meta, node_type, device)

Construct a FeatureGraph from trajectory data at a specific time step.

Extracts position and velocity data from the trajectory dictionary at the given time point, then delegates to the second method to construct the graph with edge connectivity and normalized features.

Arguments

  • gns::GraphNetCore.GraphNetwork: Graph network model containing normalizers for features.
  • data::Dict{String,Any}: Dictionary containing trajectory data (position, velocity, etc.).
  • datapoint::Integer: Time step index to extract from the trajectory.
  • meta::Dict{String,Any}: Metadata dictionary with connectivity and feature settings.
  • node_type: One-hot encoded node type features.
  • device::Function: Device placement function (cpu or gpu).

Returns

  • GraphNetCore.FeatureGraph: Constructed graph with normalized node and edge features.
source
build_graph(gns::GraphNetCore.GraphNetwork, position, velocity, meta, node_type, mask, device)

Construct a FeatureGraph from position and velocity data with edge connectivity.

Computes edges based on spatial proximity using GPU-accelerated neighborhood search, calculates relative displacements and normalized distances. Node features are constructed from position, velocity, node type, and distance bounds to domain boundaries. All features are normalized using the normalizers stored in the model.

Arguments

  • gns::GraphNetCore.GraphNetwork: Graph network model containing normalizers.
  • position::AbstractArray: Particle positions with shape (dims, n_particles).
  • velocity::AbstractArray: Particle velocities with shape (dims, n_particles).
  • meta::Dict{String,Any}: Metadata with defaultconnectivityradius, bounds, dims, input_features, and device settings.
  • node_type::AbstractArray: One-hot encoded node type features.
  • mask::AbstractVector: Indices of particles to include in the graph (fluid particles).
  • device::Function: Device placement function.

Returns

  • GraphNetCore.FeatureGraph: Graph with normalized node features, normalized edge features, sender and receiver indices.
source

ODE Integration

GraphNetSim.rollout — Function
rollout(solver, gns::GraphNetwork, initial_state, output_fields, meta, target_fields, node_type, mask, val_mask, start, stop, dt, saves, device; pr=nothing)

Solves the ODE problem for a Graph Neural Network simulator using the given solver and computes solution at specified timesteps.

Solves the ODEProblem of the GNS model over the specified time interval. The function handles both fixed and adaptive timestep solvers, with optional progress reporting.

Arguments

  • solver: ODE solver algorithm (e.g., Tsit5(), RK4()) from OrdinaryDiffEq.jl.
  • gns::GraphNetwork: Graph neural network model to evaluate for dynamics.
  • initial_state::Dict: Dictionary with "position" and "velocity" arrays for initial conditions.
  • output_fields::Vector{String}: Names of output features predicted by the network.
  • meta::Dict: Dataset metadata containing feature dimensions and specifications.
  • target_fields::Vector{String}: Names of target (output) features.
  • node_type::Vector: One-hot encoded node type indicators.
  • mask::Vector: Boolean mask for valid nodes in graph.
  • val_mask::Vector: Validation/evaluation mask for output features.
  • start::Float32: Start time of ODE integration.
  • stop::Float32: Stop time of ODE integration.
  • dt::Union{Nothing,Float32}: Fixed timestep (if nothing, uses adaptive timestepping).
  • saves::Vector: Timesteps where solution should be saved.
  • device::Function: Device placement function (cpudevice or gpudevice).

Keyword Arguments

  • pr::Union{Nothing,ProgressBar}=nothing: Progress bar for tracking ODE solve.

Returns

  • sol: Solution object containing state trajectories at specified saves timesteps.

Notes

  • Uses ode_step_eval() as the right-hand side function for the ODE.
  • State is packed as ComponentArray with x (position) and dx (velocity) fields.
  • Output features are denormalized using stored normalizers before return.
source

Data Utilities and Conversion

Dataset Loading

GraphNetSim.keystraj — Function
keystraj(datafile::String)

Extract trajectory keys from a data file.

Opens either a .jld2 or .h5 file and returns all top-level keys, representing individual trajectories stored in the file.

Arguments

  • datafile::String: Path to the data file (.h5 or .jld2).

Returns

  • Array{String,1}: Array of trajectory keys from the file.
source
MLCore.getobs! — Function
MLUtils.getobs!(buffer::Dict{String,Any}, ds::Dataset, idx::Int)

Load trajectory data into a pre-allocated buffer using the MLUtils interface.

Retrieves a single trajectory by index, populates all metadata and features into the provided buffer dictionary, and applies trajectory preparation (device transfer, masking, validation masks). Modifies the buffer in-place.

Arguments

  • buffer::Dict{String,Any}: Pre-allocated dictionary to store trajectory data (modified in-place).
  • ds::Dataset: The dataset object.
  • idx::Int: Index of the trajectory to retrieve (1-indexed).

Returns

  • Dict{String,Any}: The modified buffer containing the trajectory data.
source

Format Conversion

GraphNetSim.csv_to_hdf5 — Function
csv_to_hdf5(source::String, output::String;
             dt::Float64=0.01, 
             n_trajectories::Int=1,
             dims::Vector{Int}=[1, 2],
             groupby_col::Symbol=:Idp,
             interpolation_scheme::String="pchip",
             pos_col_prefix::String="Points",
             vel_col_prefix::String="Vel",
             type_col::Symbol=:Type,
             extra_fields::Vector{Symbol}=Symbol[])

Convert particle trajectory data from CSV to HDF5 format with computed accelerations.

Arguments

  • source::String: Path to input CSV file
  • output::String: Path to output HDF5 file

Keyword Arguments

  • dt::Float64: Time step (default: 0.01)
  • n_trajectories::Int: Number of trajectories to process (default: 1)
  • dims::Vector{Int}: Spatial dimensions to extract, e.g. [1, 2] or [1, 3].(default: [1, 2])
  • groupby_col::Symbol: Column name for grouping particles (default: :Idp)
  • interpolation_scheme::String: Acceleration calculation method (default: "pchip")
    • "central_diff": Central difference scheme
    • "forward_diff": Forward difference scheme
    • "backward_diff": Backward difference scheme
    • "from_pos": Acceleration from position (2nd order central difference)
    • "pchip": PCHIP interpolation of velocity derivatives
    • "linear": LinearInterpolation
    • "quadratic": QuadraticInterpolation
    • "cubic_spline": CubicSpline
    • "quadratic_spline": QuadraticSpline
    • "cubic_hermite": CubicHermiteSpline
    • "lagrange": LagrangeInterpolation
    • "akima": AkimaInterpolation
  • pos_col_prefix::String: Prefix for position columns (default: "Points")
  • vel_col_prefix::String: Prefix for velocity columns (default: "Vel")
  • type_col::Symbol: Column name for particle type (default: :Type)
  • extra_fields::Vector{Symbol}: Additional CSV columns to copy to HDF5 (e.g. [:Mass, :Temperature])

Example

# 3D simulation with PCHIP interpolation
csv_to_hdf5("data/dam_break.csv", "data/dam_break_first.h5"; 
            dt=0.01, dims=[1, 2, 3], interpolation_scheme="pchip")

# 2D (skip y-dimension), with extra fields
csv_to_hdf5("data/input.csv", "output.h5"; 
            dims=[1, 3], interpolation_scheme="cubic_spline",
            extra_fields=[:Mass, :Pressure])
source
GraphNetSim.vtk_to_hdf5 — Function
vtk_to_hdf5(sources, output::String; kwargs...)

Convert VTK particle trajectories into the HDF5 layout GraphNetSim trains on. Each entry in sources becomes one trajectory group (trajectory_1, trajectory_2, …) in output. Supported inputs:

  • ParaView / DualSPHysics XML (.pvd collections, .pvtu parallel wrappers, .vtu pieces), and
  • legacy BINARY .vtk (POLYDATA, e.g. DualSPHysics PartAll_*.vtk).

The .pvd/.pvtu wrappers are parsed internally; leaf .vtu pieces are decoded via ReadVTK while legacy .vtk files use a built-in reader (ReadVTK handles neither the wrappers nor legacy .vtk). Empty (0-point) frames are skipped and particles are kept row-aligned across time by sorting on id_field when present.

Arguments

  • sources: a single .pvd, a directory (of .pvtu/.vtu/.vtk), a .pvtu/.vtu/.vtk, or a vector of such (one per trajectory). A directory's leaves are ordered lexicographically.
  • output::String: destination .h5 file.

Keyword Arguments

  • dims::Vector{Int}=[1,2,3]: which of the 3 VTK point components to keep.
  • velocity_field="Vel", type_field="phaseID", id_field="prtlID", acc_field="Acc": PointData array names for each role; auto-detected from common aliases when absent.
  • extra_fields::Vector{String}=String[]: extra scalar PointData arrays to copy through.
  • recompute_acc::Bool=false: derive acceleration from velocity/position instead of reading acc_field. Recomputation also happens automatically when no acceleration field is present (e.g. legacy .vtk).
  • interpolation_scheme::String="pchip": scheme used when (re)computing acceleration (see csv_to_hdf5).
  • dt::Union{Nothing,Real}=nothing: fixed timestep; inferred from .pvd timesteps when nothing. A directory/glob source carries no timestamps, so pass dt explicitly there (it defaults to 1.0).
  • meta::Union{Nothing,String}=nothing: path to an existing meta.json to validate + copy next to output (Tier A).
  • write_meta::Bool=false: emit a computed skeleton meta.json instead (Tier B).
  • connectivity_radius::Union{Nothing,Real}=nothing: value for the skeleton meta (estimated + warned when nothing).
source

Visualization

VTK Export

GraphNetSim.visualize — Function
visualize(inPath, outFolder, Position, subgroupTrajectory, Parameters, Trajectorys, NumberOfTimesteps)

Reads HDF5 file into dictionary and writes VTK HDF5 file format with dictToVTKHDF().

Input file must have a group for each trajectory and subgroups for all trajectories used. Datasets are linked to each subgroup of trajectory. If only certain trajectories should be written, specify them with a Vector{Int} (minimum is 1). Validation contains the names of the subgroups which will be written and each trajectory must have the same number of timesteps.

Timesteps can be automatically detected if dataset "timesteps" is linked to each trajectory group. This dataset must contain the number for the largest timestep and timesteps 1:1:max will be read. When using timesteps, all datasets to be read need "[Int]" appended.

Arguments

  • inPath::String: Complete path ending with .h5 file.
  • outFolder::String: Complete path for output folder.
  • Position::String: Name of HDF5 dataset for position data.
  • subgroupTrajectory::String: Name of subgroup within trajectory groups.
  • Parameters::Vector{String}: Names of other HDF5 datasets to read (optional).
  • Trajectorys::Union{Vector{Int},Nothing}: Trajectory indices to write (optional, auto-detected if nothing).
  • NumberOfTimesteps::Union{Vector{Int},Nothing}: Timesteps to write (optional, auto-detected if nothing).

Returns

  • Dict: Dictionary mapping (trajectory, dataset_name, timestep) tuples to numerical arrays.
source
GraphNetSim.visualize_eval — Function
visualize_eval(inPath, outFolder; Position, gt_subgroup, gt_parameters,
               pred_subgroup, pred_parameters, Trajectorys)

One-shot VTK export for evaluation outputs: writes both the rollout and the ground-truth subgroup. Safe for extrapolation — GT files are produced for every timestep present in the gt/ group, even when the rollout is longer.

Arguments

  • inPath::String: Complete path ending with .h5 file.
  • outFolder::String: Complete path for output folder.
  • Position::String: HDF5 dataset name for position data (default "pos").
  • gt_subgroup::String: Subgroup name for ground truth (default "gt").
  • gt_parameters::Vector{<:AbstractString}: Additional GT datasets (default ["vel", "acc"]).
  • pred_subgroup::String: Subgroup name for rollout (default "prediction").
  • pred_parameters::Vector{<:AbstractString}: Additional rollout datasets (default ["vel", "acc", "err"]).
  • Trajectorys::Union{Vector{Int},Nothing}: Trajectory indices to write (optional, auto-detected if nothing).
source