Physics-Guided Optimization Instrumentation

This page documents the reusable Wave 5.2R loss-interaction diagnostics, weight adapters, conflict-aware gradient composition, schedules, and deterministic dataloader helpers.

Reusable loss-interaction instrumentation for physics-guided TE training.

class scripts.training.physics_guided_optimization_instrumentation.LossActivationSchedule(start_step=0, full_weight_step=0, end_step=None)[source]

Describe the staged activation interval for one loss component.

Parameters:

  • start_step (int)

  • full_weight_step (int)

  • end_step (int | None)

start_step

First optimization step with a non-zero multiplier.

Type:

int

full_weight_step

First step at which the multiplier reaches one.

Type:

int

end_step

Optional first step at which the component becomes inactive.

Type:

int | None

resolve_multiplier(optimization_step)[source]

Resolve the deterministic activation multiplier for one step.

Parameters:

optimization_step (int) – Zero-based optimizer-step index.

Returns:

Component multiplier inside the closed interval from zero to one.

Return type:

float

class scripts.training.physics_guided_optimization_instrumentation.LossComponentConfiguration(name, unit_label, normalization_scale=1.0, fixed_weight=1.0, role='auxiliary', activation_schedule=LossActivationSchedule(start_step=0, full_weight_step=0, end_step=None))[source]

Define one named, normalized, and scheduled loss component.

Parameters:

  • name (str)

  • unit_label (str)

  • normalization_scale (float)

  • fixed_weight (float)

  • role (str)

  • activation_schedule (LossActivationSchedule)

name

Stable machine-facing component name.

Type:

str

unit_label

Human-readable raw unit, such as normalized TE squared.

Type:

str

normalization_scale

Positive divisor that maps the raw loss to an order-one comparison scale.

Type:

float

fixed_weight

Baseline coefficient used by fixed controls and as the reference coefficient for adaptive adapters.

Type:

float

role

Either main for the protected data loss or auxiliary.

Type:

str

activation_schedule

Optional staged activation contract.

Type:

scripts.training.physics_guided_optimization_instrumentation.LossActivationSchedule

class scripts.training.physics_guided_optimization_instrumentation.ParameterFreezeSchedule(parameter_name_token_list, freeze_before_step)[source]

Define a name-matched parameter freeze-unfreeze interval.

Parameters:

  • parameter_name_token_list (tuple[str, ...])

  • freeze_before_step (int)

parameter_name_token_list

Tokens matched against model parameter names.

Type:

tuple[str, …]

freeze_before_step

Parameters are frozen before this optimizer step.

Type:

int

apply(named_parameter_iterable, optimization_step)[source]

Apply the freeze state and return the matched parameter names.

Parameters:

  • named_parameter_iterable (Iterable[tuple[str, Parameter]]) – Model parameter names and tensors.

  • optimization_step (int) – Zero-based optimizer-step index.

Returns:

Names of parameters matched by the schedule.

Return type:

list[str]

class scripts.training.physics_guided_optimization_instrumentation.PhysicsGuidedOptimizationInstrumentation(component_configuration_list, ema_decay=0.95, epsilon=DEFAULT_EPSILON, random_seed=314159)[source]

Measure and adapt named loss interactions on shared parameters.

The class keeps scalar history only. Per-loss gradients are computed with functional autograd and therefore do not populate or overwrite parameter .grad buffers.

Parameters:
__init__(component_configuration_list, ema_decay=0.95, epsilon=DEFAULT_EPSILON, random_seed=314159)[source]

Initialize the loss-interaction instrumentation.

Parameters:

  • component_configuration_list (Sequence[LossComponentConfiguration]) – Complete named component contract.

  • ema_decay (float) – Exponential moving-average retention coefficient.

  • epsilon (float) – Positive numerical-stability floor.

  • random_seed (int) – Seed used by deterministic ReLoBRaLo-style lookbacks.

Return type:

None

validate_loss_dictionary(raw_loss_dictionary)[source]

Validate exact component coverage and scalar tensor structure.

Parameters:

raw_loss_dictionary (Mapping[str, Tensor])

Return type:

None

normalize_loss_dictionary(raw_loss_dictionary)[source]

Normalize all loss components into declared comparison units.

Parameters:

raw_loss_dictionary (Mapping[str, Tensor])

Return type:

dict[str, Tensor]

update_loss_ema_dictionary(normalized_loss_dictionary)[source]

Update and return normalized per-component loss EMAs.

Parameters:

normalized_loss_dictionary (Mapping[str, Tensor])

Return type:

dict[str, float]

resolve_activation_multiplier_dictionary(optimization_step)[source]

Resolve the activation multiplier of every component.

Parameters:

optimization_step (int)

Return type:

dict[str, float]

compute_component_gradient_dictionary(normalized_loss_dictionary, shared_parameter_sequence)[source]

Compute one flattened gradient vector per normalized loss.

Parameters:

  • normalized_loss_dictionary (Mapping[str, Tensor]) – Scalar normalized losses.

  • shared_parameter_sequence (Sequence[Parameter]) – Declared shared trainable parameters.

Returns:

Component names mapped to flattened detached gradient vectors.

Return type:

dict[str, Tensor]

compute_gradient_norm_dictionary(component_gradient_dictionary)[source]

Compute the L2 norm of each component gradient.

Parameters:

component_gradient_dictionary (Mapping[str, Tensor])

Return type:

dict[str, float]

compute_pairwise_gradient_cosine_dictionary(component_gradient_dictionary)[source]

Compute every unique pairwise gradient cosine similarity.

Parameters:

component_gradient_dictionary (Mapping[str, Tensor])

Return type:

dict[str, float]

resolve_weight_dictionary(adapter_name, normalized_loss_dictionary, gradient_norm_dictionary, optimization_step, gradient_weight_minimum=0.05, gradient_weight_maximum=20.0, relobralo_temperature=0.25, relobralo_lookback_probability=0.95, relobralo_previous_weight_retention=0.90)[source]

Resolve component weights for one supported adapter.

Parameters:

  • adapter_name (str) – Registered adapter name.

  • normalized_loss_dictionary (Mapping[str, Tensor]) – Current normalized scalar losses.

  • gradient_norm_dictionary (Mapping[str, float]) – Current per-component gradient norms.

  • optimization_step (int) – Zero-based optimizer-step index.

  • gradient_weight_minimum (float) – Lower adaptive-weight clamp.

  • gradient_weight_maximum (float) – Upper adaptive-weight clamp.

  • relobralo_temperature (float) – Positive relative-progress softmax scale.

  • relobralo_lookback_probability (float) – Probability of comparing with the initial rather than immediately previous loss state.

  • relobralo_previous_weight_retention (float) – Previous-weight blend factor.

Returns:

Component names mapped to finite non-negative weights.

Return type:

dict[str, float]

compose_weighted_loss(normalized_loss_dictionary, weight_dictionary)[source]

Compose a scalar weighted loss without gradient surgery.

Parameters:

  • normalized_loss_dictionary (Mapping[str, Tensor])

  • weight_dictionary (Mapping[str, float])

Return type:

Tensor

compose_main_loss_preserving_gradient(component_gradient_dictionary, weight_dictionary)[source]

Project conflicting auxiliary gradients while preserving the main.

Each auxiliary gradient with a negative dot product against the main gradient is projected onto the main-gradient normal plane. The protected main gradient is never altered.

Parameters:

  • component_gradient_dictionary (Mapping[str, Tensor]) – Flattened component gradients.

  • weight_dictionary (Mapping[str, float]) – Scalar component coefficients.

Returns:

Combined flattened gradient and per-auxiliary projection records.

Return type:

tuple[Tensor, dict[str, dict[str, float | bool]]]

build_diagnostic_record(raw_loss_dictionary, shared_parameter_sequence, adapter_name, optimization_step)[source]

Build one complete scalar loss-interaction diagnostic record.

Parameters:

  • raw_loss_dictionary (Mapping[str, Tensor])

  • shared_parameter_sequence (Sequence[Parameter])

  • adapter_name (str)

  • optimization_step (int)

Return type:

dict[str, Any]

scripts.training.physics_guided_optimization_instrumentation.capture_trainable_parameter_vector(parameter_sequence)[source]

Capture one detached flattened vector of trainable parameters.

Parameters:

parameter_sequence (Sequence[Parameter])

Return type:

Tensor

scripts.training.physics_guided_optimization_instrumentation.assign_flat_gradient_to_parameters(flattened_gradient_tensor, parameter_sequence)[source]

Assign one flattened gradient vector to trainable parameters.

Parameters:

  • flattened_gradient_tensor (Tensor) – Gradient vector following parameter order.

  • parameter_sequence (Sequence[Parameter]) – Parameters whose .grad buffers will be assigned.

Return type:

None

scripts.training.physics_guided_optimization_instrumentation.compute_update_to_parameter_ratio(parameter_vector_before_step, parameter_vector_after_step, epsilon=DEFAULT_EPSILON)[source]

Compute the optimizer update norm divided by parameter norm.

Parameters:

  • parameter_vector_before_step (Tensor)

  • parameter_vector_after_step (Tensor)

  • epsilon (float)

Return type:

float

scripts.training.physics_guided_optimization_instrumentation.configure_deterministic_execution(random_seed)[source]

Configure deterministic PyTorch execution and return a data generator.

Notes

PyTorch does not guarantee identical results across releases, platforms, or CPU/GPU devices. This helper establishes reproducibility inside one frozen execution environment.

Parameters:

random_seed (int)

Return type:

Generator

scripts.training.physics_guided_optimization_instrumentation.build_deterministic_dataloader(dataset, batch_size, random_seed, shuffle=True, num_workers=0)[source]

Build a seeded dataloader with an explicit generator.

Parameters:

  • dataset (Dataset) – PyTorch dataset.

  • batch_size (int) – Positive batch size.

  • random_seed (int) – Seed assigned to the dataloader generator.

  • shuffle (bool) – Whether to shuffle the dataset.

  • num_workers (int) – Non-negative worker count.

Returns:

Deterministically configured dataloader.

Return type:

DataLoader

scripts.training.physics_guided_optimization_instrumentation.compute_dataloader_fingerprint(dataloader)[source]

Compute an exact ordered-batch SHA-256 fingerprint.

Parameters:

dataloader (DataLoader)

Return type:

str