refnx.reflect.extra

class refnx.reflect.extra.CompiledModel(model: Callable, jacfwd: Callable, x0: Array, param_names: list[str], params_to_slabs: Callable, setp: Callable, n_free: int)[source]

Bases: object

The output of compile_model.

Mirrors the interface of CompiledObjective but exposes the forward model R(q; free) rather than the log-likelihood, so that gradients of the reflectivity with respect to free parameters are accessible directly. Useful for sensitivity analysis, uncertainty propagation via linearisation, and visualising how R(q) changes with each parameter.

model

Pure JAX function (q, free, q_err=None) -> R(q) (shape (N,)), paralleling refnx.reflect.ReflectModel.__call__. JIT-compiled. Resolution smearing is applied when q_err is supplied and non-zero.

Type:

callable

jacfwd

Forward-mode Jacobian (q, free, q_err=None) -> dR/d(free) (shape (N, n_free)). JIT-compiled. Differentiates w.r.t. free only; q and q_err are treated as fixed inputs.

Type:

callable

x0

Initial free-parameter values.

Type:

jnp.ndarray

param_names

Names of the free parameters, in the same order as x0.

Type:

list of str

params_to_slabs

Pure JAX function mapping the free vector to the (N, 4) layers array (thick, mixed_sld_real, mixed_sld_imag, rough) used by the reflectivity kernel. Solvent volume-fraction averaging has already been applied. Useful for inspecting the SLD profile under AD.

Type:

callable

setp

setp(x) pushes the values in x back into the stateful objective after use.

Type:

callable

n_free

Number of free parameters.

Type:

int

class refnx.reflect.extra.CompiledObjective(logl: Callable, grad_logl: Callable, value_and_grad: Callable, params_to_slabs: Callable, generative: Callable, generative_fwd: Callable, x0: Array, param_names: list[str], setp: Callable, n_free: int)[source]

Bases: object

The output of compile_objective.

logl

Pure JAX log-likelihood, logl(free). JIT-compiled.

Type:

callable

grad_logl

Exact gradient of logl w.r.t. the free-parameter vector. JIT-compiled.

Type:

callable

value_and_grad

jax.value_and_grad(logl), JIT-compiled. Preferred for optimisers that need both simultaneously.

Type:

callable

params_to_slabs

Pure JAX function mapping the free vector to the (N, 4) layers array (thick, mixed_sld_real, mixed_sld_imag, rough) used by the reflectivity kernel. Solvent volume-fraction averaging has already been applied. Useful for inspecting the SLD profile under AD.

Type:

callable

generative

Pure JAX forward model free -> R(q) (shape (N,)). JIT-compiled. Evaluates the reflectivity curve at the Q values and resolution of the objective’s dataset. Useful for posterior predictive checks and plotting the model curve at optimised parameters. For a refnx.analysis.GlobalObjective this is the concatenation of the per-objective generative functions, matching the order of GlobalObjective.objectives.

Type:

callable

generative_fwd

Forward-mode (jvp) compatible twin of generative, with the same signature and result. The default kernel used by generative only supports reverse-mode AD, so jax.jvp/jax.jacfwd raise on it. generative_fwd always uses a pure-JAX kernel instead, so it works under forward-mode AD (e.g. a pytensor pushforward/Rop, or jax.jacfwd). JIT-compiled.

Type:

callable

x0

Initial free-parameter values extracted from the objective.

Type:

jnp.ndarray

param_names

Names of the free parameters, in the same order as x0.

Type:

list of str

setp

Thin wrapper around refnx.analysis.Objective.setp so you can push optimised values back into the stateful object graph after optimisation.

Type:

callable

n_free

Number of free parameters.

Type:

int

class refnx.reflect.extra.GenerativeOp(compiled_objective)[source]

Bases: Op

A pytensor Op that wraps CompiledObjective.generative.

Takes the tuple of free-parameter pytensor variables and returns a vector of predicted reflectivity values R(q) of shape (N_q,). Supports both reverse-mode (pullback) and forward-mode (pushforward) differentiation.

Parameters:

compiled_objective (CompiledObjective) – A compiled objective, as returned by compile_objective or compile_global_objective.

make_node(inputs)[source]

Construct an Apply node that represent the application of this operation to the given inputs.

This must be implemented by sub-classes.

Returns:

node – The constructed Apply node.

Return type:

Apply

perform(node, inputs, outputs)[source]

Calculate the function on the inputs and put the variables in the output storage.

Parameters:
  • node – The symbolic Apply node that represents this computation.

  • inputs – Immutable sequence of non-symbolic/numeric inputs. These are the values of each Variable in node.inputs.

  • output_storage – List of mutable single-element lists (do not change the length of these lists). Each sub-list corresponds to value of each Variable in node.outputs. The primary purpose of this method is to set the values of these sub-lists.

Notes

The output_storage list might contain data. If an element of output_storage is not None, it has to be of the right type, for instance, for a TensorVariable, it has to be a NumPy ndarray with the right number of dimensions and the correct dtype. Its shape and stride pattern can be arbitrary. It is not guaranteed that such pre-set values were produced by a previous call to this Op.perform; they could’ve been allocated by another Op’s perform method. An Op is free to reuse output_storage as it sees fit, or to discard it and allocate new memory.

pullback(inputs, outputs, cotangents)[source]

Construct a graph for the vector-Jacobian product (pullback).

Given a function \(f\) implemented by this Op with inputs \(x\) and outputs \(y = f(x)\), the pullback computes \(\bar{x} = \bar{y}^T J\) where \(J\) is the Jacobian \(\frac{\partial f}{\partial x}\) and \(\bar{y}\) are the cotangent vectors (upstream gradients).

This is the core method for reverse-mode automatic differentiation.

If the output is not differentiable with respect to an input, return a variable of type DisconnectedType for that input. If the gradient is not implemented for some input, return a variable of type NullType (see pytensor.gradient.grad_not_implemented and pytensor.gradient.grad_undefined).

Parameters:
  • inputs (Sequence[Variable]) – The input variables of the Apply node using this Op.

  • outputs (Sequence[Variable]) – The output variables of the Apply node using this Op.

  • cotangents (Sequence[Variable]) – The cotangent vectors (gradients w.r.t. each output).

Returns:

input_cotangents – The cotangent vectors w.r.t. each input. One Variable per input.

Return type:

list of Variable

pushforward(inputs, outputs, tangents)[source]

Construct a graph for the Jacobian-vector product (pushforward).

Given a function \(f\) implemented by this Op with inputs \(x\) and outputs \(y = f(x)\), the pushforward computes \(\dot{y} = J \dot{x}\) where \(J\) is the Jacobian \(\frac{\partial f}{\partial x}\) and \(\dot{x}\) are the tangent vectors.

This is the core method for forward-mode automatic differentiation.

If an output is not differentiable with respect to any input, return a variable of type DisconnectedType for that output. Unlike the legacy R_op method, pushforward must never use None to indicate disconnected outputs.

Parameters:
  • inputs (Sequence[Variable]) – The input variables of the Apply node using this Op.

  • outputs (Sequence[Variable]) – The output variables of the Apply node using this Op.

  • tangents (Sequence[Variable]) – The tangent vectors. One per input. A variable of DisconnectedType indicates that the corresponding input is not being differentiated.

Returns:

output_tangents – The tangent vectors w.r.t. each output. One Variable per output.

Return type:

list of Variable

refnx.reflect.extra.compile_global_objective(global_objective, reflect_fn: Callable | None = None) → CompiledObjective[source]

Compile a refnx.analysis.GlobalObjective into a pure JAX log-likelihood.

The compiled log-likelihood is the weighted sum of the individual objective log-likelihoods, mirroring refnx.analysis.GlobalObjective.logl:

logl = sum( lambda_i * logl_i(free) )

Because all child objectives are compiled against a single shared free-parameter index built from global_objective.varying_parameters(), parameters that are shared between objectives (e.g. a polymer SLD refined simultaneously against multiple contrasts) appear exactly once in the free vector and their gradients are summed correctly by JAX’s autodiff — no special handling is required.

Parameters:
Returns:

compiled – The compiled log-likelihood, its gradient, and the forward model. The params_to_slabs field is not populated for a refnx.analysis.GlobalObjective (it is set to None) because there is no single layer stack — use the individual compiled objectives if you need per-contrast SLD profiles.

Return type:

CompiledObjective

Notes

lambdas: GlobalObjective.lambdas are plain floats and are baked in as constants at compile time. If you change them you must recompile.

All other notes from compile_objective apply to each child objective individually (64-bit floats, re-compilation on topology change, callable constraints, etc.).

Examples

>>> compiled = compile_global_objective(global_objective)
>>> val, grad = compiled.value_and_grad(compiled.x0)
>>> # push back into the stateful world after optimisation:
>>> compiled.setp(result.x)
refnx.reflect.extra.compile_model(reflect_model) → CompiledModel[source]

Compile a refnx.reflect.ReflectModel into a pure JAX forward-model function.

Returns a CompiledModel whose model(q, free, q_err=None) maps the free-parameter vector to the reflectivity curve R(q) for any supplied Q values, and whose jacfwd(q, free, q_err=None) gives the exact Jacobian dR_i/dp_j at those Q values. This call signature parallels refnx.reflect.ReflectModel.__call__.

Accepting q and q_err at call time rather than compile time means a single CompiledModel can be evaluated and differentiated on any Q grid without recompilation — useful for simulation, sensitivity analysis on a fine grid, or fitting datasets with different Q ranges.

Parameters:

reflect_model (refnx.reflect.ReflectModel) – The model to compile. Its structure, scale, and bkg parameters are compiled into the JAX graph. Any parameter that is currently set to vary will appear in the free vector.

Returns:

compiled – The compiled forward model and its Jacobian.

Return type:

CompiledModel

Examples

>>> cm = compile_model(model)
>>> # forward model — shape (len(data.x),):
>>> R = cm.model(data.x, cm.x0, data.x_err)
>>> # Jacobian on a finer synthetic grid, no smearing:
>>> q_fine = np.linspace(0.005, 0.3, 1000)
>>> J = cm.jacfwd(q_fine, cm.x0)
>>> # Jacobian on a finer synthetic grid, constant dq/q smearing:
>>> J = cm.jacfwd(q_fine, cm.x0, q_fine * 0.05)

Notes

Why jacfwd rather than jacrev? The Jacobian is (N_q, n_free) shaped. Forward-mode accumulates one column per free parameter (n_free passes); reverse-mode accumulates one row per Q point (N_q passes). For typical fits n_free is O(10–50) and N_q is O(100–1000), so forward mode is the right default. If you find yourself with far more parameters than Q points, jax.jacrev is a one-line swap.

jacfwd vs jax.grad on a scalar reduction. For the log-likelihood jax.grad is ideal (one reverse pass). But model returns a vector, so grad doesn’t apply — jacfwd is the correct primitive here.

refnx.reflect.extra.compile_objective(objective, reflect_fn: Callable | None = None) → CompiledObjective[source]

Compile a refnx.analysis.Objective into a pure JAX log-likelihood.

The compilation analyses the Parameter graph once, at call time, and returns a CompiledObjective whose logl function is a pure, JIT-compiled JAX function with no Python side-effects.

Parameters:
  • objective (refnx.analysis.Objective) – Must have a model that is a refnx.reflect.ReflectModel backed by a refnx.reflect.Structure composed of refnx.reflect.Slab-like components. Components without JAX support (e.g. refnx.reflect.Spline) will have their slab values baked in as constants, so AD will not flow through them.

  • reflect_fn (callable, optional) – Forward-model kernel to use for generative/logl. Defaults to the fastest available kernel. If the result will be consumed by a fused jax.value_and_grad call elsewhere (e.g. to_pymc_model(..., _dist="potential")) a pure-JAX kernel, such as refnx.reflect._jax_reflect.jabeles, is faster.

Returns:

compiled – The compiled log-likelihood, its gradient, and the forward model.

Return type:

CompiledObjective

Notes

64-bit floats: the reflectivity kernels require 64-bit. Enable before calling:

from jax import config
config.update("jax_enable_x64", True)

Re-compilation: if you change which parameters vary (or add/remove constraints) you must call compile_objective again. The compiled function captures the graph topology at compile time.

lnsigma: supported as a free or fixed parameter.

scale / background: compiled from model.scale and model.bkg Parameters if present; otherwise baked in as constants.

Callable constraints: constraints set with refnx.analysis.Parameter.set_constraint are included, but gradients only flow through them if the callable itself uses JAX-compatible operations (jnp etc.). numpy-based callables will act as stop-gradients.

refnx.reflect.extra.make_scipy_objective(compiled: CompiledObjective)[source]

Build a scipy-compatible (nll_fn, grad_fn) pair suitable for:

scipy.optimize.minimize(nll_fn, x0, jac=grad_fn, method='L-BFGS-B')
Parameters:

compiled (CompiledObjective) – A compiled objective, as returned by compile_objective or compile_global_objective.

Returns:

  • nll_fn (callable) – nll_fn(x) -> float. Negative log-likelihood as a function of the free-parameter vector.

  • grad_fn (callable) – grad_fn(x) -> np.ndarray. Gradient of nll_fn with respect to the free-parameter vector.

Notes

Both functions accept and return plain np.ndarray (float64).

refnx.reflect.extra.process_trace(objective, trace)[source]

Process the trace produced by a pymc.sample run.

Parameters:
  • objective (refnx.analysis.Objective) – The Objective function that the Posterior was sampled on.

  • trace (arviz.InferenceData) – The trace returned by pymc.sample, for a model created by to_pymc_model.

Returns:

[(param, stderr, chain)] – List of (param, stderr, chain) tuples. If isinstance(objective.parameters, Parameters) then param is a Parameter instance. param.value, param.stderr and param.chain will contain the median, stderr and chain samples, respectively. Otherwise param will be a float representing the median of the chain samples. stderr is the half width of the [15.87, 84.13] spread (similar to standard deviation) and chain is an array containing the MCMC samples for that parameter.

Return type:

list

Notes

This function has the effect of setting the parameter stderr’s.

refnx.reflect.extra.to_pymc_model(objective, _dist=None)[source]

Creates a pymc model from an Objective.

Requires pytensor and pymc be installed. This is an experimental feature.

Parameters:
  • objective (refnx.analysis.Objective) – The Objective function to convert into a pymc model.

  • _dist ({None, 'normal', 'potential', 'custom'}, optional) –

    How the likelihood is expressed in the pymc model:

    • None or ‘normal’ : uses pymc.Normal, with observed y data on Objective.generative.

    • ’potential’ : uses pymc.Potential on Objective.logl.

    • ’custom’ : uses pymc.CustomDist on Objective.logl.

    ’custom’ and ‘normal’ can be used for model comparison, posterior predictive sampling, etc; ‘potential’ cannot. See Notes for guidance on choosing between them.

Returns:

model – The pymc model, ready for sampling.

Return type:

pymc.Model

Notes

The varying parameters are renamed ‘p0’, ‘p1’, etc, as it’s vital in pymc that all parameters have their own unique name.

If any refnx.reflect.ReflectModel in the Objective has non-zero logp, or any Objective has a logp_extra function, then derivatives of those are not tracked. At that point it’s advisable to just use _dist='potential', and not ‘custom’ or ‘normal’.

A ReflectModel might have a non-zero logp if there is a Component in its Structure that applies a probabilistic penalty. One such example might be refnx.reflect.LipidLeaflet if a volume fraction in a head/tail region exceeds 1. This is ok because the penalty is non-differentiable (-np.inf or 0). However, if the penalty is used to steer the modelling process (ReflectModel.logp is finite and gets bigger/smaller depending on Parameters) then it’s inadvisable to use anything but ‘potential’.

In benchmarking ‘normal’ seems to sample the fastest, but ‘potential’ might be the most robust.