gensbi.recipes.joint_pipeline#

Pipeline for training and using a Joint model for simulation-based inference.

Classes#

`JointDiffusionPipeline`	Diffusion pipeline for training and using a Joint model for simulation-based inference.
`JointFlowPipeline`	Flow pipeline for training and using a Joint model for simulation-based inference.

Functions#

`sample_condition_mask`(key, num_samples, theta_dim, x_dim)
`sample_structured_conditional_mask`(key, num_samples, ...)	Sample structured conditional masks for the Joint model.

Module Contents#

class gensbi.recipes.joint_pipeline.JointDiffusionPipeline(model, train_dataset, val_dataset, dim_obs, dim_cond, ch_obs=1, params=None, training_config=None, condition_mask_kind='structured')[source]#

Bases: gensbi.recipes.pipeline.AbstractPipeline

Diffusion pipeline for training and using a Joint model for simulation-based inference.

Parameters:

train_dataset (grain dataset or iterator over batches) – Training dataset.
val_dataset (grain dataset or iterator over batches) – Validation dataset.
dim_obs (int) – Dimension of the parameter space.
dim_cond (int) – Dimension of the observation space.
ch_obs (int, optional) – Number of channels for the observation space. Default is 1.
params (optional) – Parameters for the Joint model. If None, default parameters are used.
training_config (dict, optional) – Configuration for training. If None, default configuration is used.
condition_mask_kind (str, optional) – Kind of condition mask to use. One of [“structured”, “posterior”].

Examples

Minimal example on how to instantiate and use the JointDiffusionPipeline:

# %% Imports
import os

# Set JAX backend (use 'cuda' for GPU, 'cpu' otherwise)
# os.environ["JAX_PLATFORMS"] = "cuda"

import grain
import numpy as np
import jax
from jax import numpy as jnp
from gensbi.recipes import JointDiffusionPipeline
from gensbi.utils.plotting import plot_marginals

from gensbi.models import Simformer, SimformerParams
import matplotlib.pyplot as plt

from numpyro import distributions as dist


from flax import nnx

# %%

theta_prior = dist.Uniform(
    low=jnp.array([-2.0, -2.0, -2.0]), high=jnp.array([2.0, 2.0, 2.0])
)

dim_obs = 3
dim_cond = 3
dim_joint = dim_obs + dim_cond


# %%
def simulator(key, nsamples):
    theta_key, sample_key = jax.random.split(key, 2)
    thetas = theta_prior.sample(theta_key, (nsamples,))

    xs = thetas + 1 + jax.random.normal(sample_key, thetas.shape) * 0.1

    thetas = thetas[..., None]
    xs = xs[..., None]

    # when making a dataset for the joint pipeline, thetas need to come first
    data = jnp.concatenate([thetas, xs], axis=1)

    return data


# %% Define your training and validation datasets.
# We generate a training dataset and a validation dataset using the simulator.
# The simulator is a simple function that generates parameters (theta) and data (x).
# In this example, we use a simple Gaussian simulator.
train_data = simulator(jax.random.PRNGKey(0), 100_000)
val_data = simulator(jax.random.PRNGKey(1), 2000)
# %% Normalize the dataset
# It is important to normalize the data to have zero mean and unit variance.
# This helps the model training process.
means = jnp.mean(train_data, axis=0)
stds = jnp.std(train_data, axis=0)


def normalize(data, means, stds):
    return (data - means) / stds


def unnormalize(data, means, stds):
    return data * stds + means


# %% Prepare the data for the pipeline
# The pipeline expects the data to be normalized but not split (for joint pipelines).


def process_data(data):
    return normalize(data, means, stds)


# %%
train_data.shape

# %%

# %% Create the input pipeline using Grain
# We use Grain to create an efficient input pipeline.
# This involves shuffling, repeating for multiple epochs, and batching the data.
# We also map the process_data function to prepare (normalize) the data for the model.
batch_size = 256

train_dataset_grain = (
    grain.MapDataset.source(np.array(train_data))
    .shuffle(42)
    .repeat()
    .to_iter_dataset()
    .batch(batch_size)
    .map(process_data)
    # .mp_prefetch() # Uncomment if you want to use multiprocessing prefetching
)

val_dataset_grain = (
    grain.MapDataset.source(np.array(val_data))
    .shuffle(
        42
    )  # Use a different seed/strategy for validation if needed, but shuffling is fine
    .repeat()
    .to_iter_dataset()
    .batch(batch_size)
    .map(process_data)
    # .mp_prefetch() # Uncomment if you want to use multiprocessing prefetching
)

# %% Define your model
# specific model parameters are defined here.
# For Simformer, we need to specify dimensions, embedding strategies, and other architecture details.
params = SimformerParams(
    rngs=nnx.Rngs(0),
    in_channels=1,
    dim_value=20,
    dim_id=10,
    dim_condition=10,
    dim_joint=dim_joint,
    fourier_features=128,
    num_heads=4,
    num_layers=6,
    widening_factor=3,
    qkv_features=40,
    num_hidden_layers=1,
)

model = Simformer(params)

# %% Instantiate the pipeline
# The JointDiffusionPipeline handles the training loop and sampling.
# We configure it with the model, datasets, dimensions using a default training configuration.
# We also specify the condition_mask_kind, which determines how conditioning is handled during training.
training_config = JointDiffusionPipeline.get_default_training_config()
training_config["nsteps"] = 10000

pipeline = JointDiffusionPipeline(
    model,
    train_dataset_grain,
    val_dataset_grain,
    dim_obs,
    dim_cond,
    condition_mask_kind="posterior",
    training_config=training_config,
)

# %% Train the model
# We create a random key for training and start the training process.
rngs = nnx.Rngs(42)
pipeline.train(
    rngs, save_model=False
)  # if you want to save the model, set save_model=True

# %% Sample from the posterior
# To generate samples, we first need an observation (and its corresponding condition).
# We generate a new sample from the simulator, normalize it, and extract the condition x_o.

new_sample = simulator(jax.random.PRNGKey(20), 1)
true_theta = new_sample[:, :dim_obs, :]  # extract observation from the joint sample

new_sample = normalize(new_sample, means, stds)
x_o = new_sample[:, dim_obs:, :]  # extract condition from the joint sample

# Then we invoke the pipeline's sample method.
samples = pipeline.sample(rngs.sample(), x_o, nsamples=100_000)
# Finally, we unnormalize the samples to get them back to the original scale.
samples = unnormalize(samples, means[:dim_obs], stds[:dim_obs])

# %% Plot the samples
# We verify the model's performance by plotting the marginal distributions of the generated samples
# against the true parameters.
plot_marginals(
    np.array(samples[..., 0]),
    gridsize=30,
    true_param=np.array(true_theta[0, :, 0]),
    range=[(1, 3), (1, 3), (-0.6, 0.5)],
)
plt.savefig("joint_diffusion_pipeline_marginals.png", dpi=100, bbox_inches="tight")
plt.show()

../../../../_images/joint_diffusion_pipeline_marginals.png

Note

If you plan on using multiprocessing prefetching, ensure that your script is wrapped in a if __name__ == "__main__": guard. See https://docs.python.org/3/library/multiprocessing.html

abstractmethod _get_default_params()[source]#: Return a dictionary of default model parameters.

abstractmethod _make_model()[source]#: Create and return the model to be trained.

_wrap_model()[source]#: Wrap the model for evaluation (either using JointWrapper or ConditionalWrapper).

classmethod get_default_training_config()[source]#

Return a dictionary of default training configuration parameters.

Returns:: training_config – Default training configuration.
Return type:: dict

get_loss_fn()[source]#: Return the loss function for training/validation.

get_sampler(x_o, nsteps=18, use_ema=True, return_intermediates=False, **model_extras)[source]#

Get a sampler function for generating samples from the trained model.

Parameters:

key (jax.random.PRNGKey) – Random number generator key.
x_o (array-like) – Conditioning variable.
step_size (float, optional) – Step size for the sampler.
use_ema (bool, optional) – Whether to use the EMA model for sampling.
time_grid (array-like, optional) – Time grid for the sampler (if applicable).
model_extras (dict, optional) – Additional model-specific parameters.

Returns:

sampler – A function that generates samples when called with a random key and number of samples.

Return type:

Callable: key, nsamples -> samples

classmethod init_pipeline_from_config()[source]#

Abstractmethod:

Initialize the pipeline from a configuration file.

Parameters:

train_dataset (iterable) – Training dataset.
val_dataset (iterable) – Validation dataset.
dim_obs (int) – Dimensionality of the parameter (theta) space.
dim_cond (int) – Dimensionality of the observation (x) space.
config_path (str) – Path to the configuration file.
checkpoint_dir (str) – Directory for saving checkpoints.

Returns:

pipeline – An instance of the pipeline initialized from the configuration.

Return type:

AbstractPipeline

sample(key, x_o, nsamples=10000, nsteps=18, use_ema=True, return_intermediates=False, **model_extras)[source]#

Generate samples from the trained model.

Parameters:

key (jax.random.PRNGKey) – Random number generator key.
x_o (array-like) – Conditioning variable (e.g., observed data).
nsamples (int, optional) – Number of samples to generate.

Returns:

samples – Generated samples of size (nsamples, dim_obs, ch_obs).

Return type:

array-like

condition_mask_kind = 'structured'[source]#

loss_fn[source]#

path[source]#

class gensbi.recipes.joint_pipeline.JointFlowPipeline(model, train_dataset, val_dataset, dim_obs, dim_cond, ch_obs=1, params=None, training_config=None, condition_mask_kind='structured')[source]#

Bases: gensbi.recipes.pipeline.AbstractPipeline

Flow pipeline for training and using a Joint model for simulation-based inference.

Parameters:

train_dataset (grain dataset or iterator over batches) – Training dataset.
val_dataset (grain dataset or iterator over batches) – Validation dataset.
dim_obs (int) – Dimension of the parameter space.
dim_cond (int) – Dimension of the observation space.
ch_obs (int, optional) – Number of channels for the observation space. Default is 1.
params (JointParams, optional) – Parameters for the Joint model. If None, default parameters are used.
training_config (dict, optional) – Configuration for training. If None, default configuration is used.
condition_mask_kind (str, optional) – Kind of condition mask to use. One of [“structured”, “posterior”].

Examples

Minimal example on how to instantiate and use the JointFlowPipeline:

# %% Imports
import os

# Set JAX backend (use 'cuda' for GPU, 'cpu' otherwise)
# os.environ["JAX_PLATFORMS"] = "cuda"

import grain
import numpy as np
import jax
from jax import numpy as jnp
from numpyro import distributions as dist
from flax import nnx

from gensbi.recipes import JointFlowPipeline
from gensbi.models import Simformer, SimformerParams

from gensbi.utils.plotting import plot_marginals
import matplotlib.pyplot as plt


# %%

theta_prior = dist.Uniform(
    low=jnp.array([-2.0, -2.0, -2.0]), high=jnp.array([2.0, 2.0, 2.0])
)

dim_obs = 3
dim_cond = 3
dim_joint = dim_obs + dim_cond


# %%
def simulator(key, nsamples):
    theta_key, sample_key = jax.random.split(key, 2)
    thetas = theta_prior.sample(theta_key, (nsamples,))

    xs = thetas + 1 + jax.random.normal(sample_key, thetas.shape) * 0.1

    thetas = thetas[..., None]
    xs = xs[..., None]

    # when making a dataset for the joint pipeline, thetas need to come first
    data = jnp.concatenate([thetas, xs], axis=1)

    return data


# %% Define your training and validation datasets.
# We generate a training dataset and a validation dataset using the simulator.
# The simulator is a simple function that generates parameters (theta) and data (x).
# In this example, we use a simple Gaussian simulator.
train_data = simulator(jax.random.PRNGKey(0), 100_000)
val_data = simulator(jax.random.PRNGKey(1), 2000)
# %% Normalize the dataset
# It is important to normalize the data to have zero mean and unit variance.
# This helps the model training process.
means = jnp.mean(train_data, axis=0)
stds = jnp.std(train_data, axis=0)


def normalize(data, means, stds):
    return (data - means) / stds


def unnormalize(data, means, stds):
    return data * stds + means


# %% Prepare the data for the pipeline
# The pipeline expects the data to be normalized but not split (for joint pipelines).


# %% Prepare the data for the pipeline
# The pipeline expects the data to be normalized but not split (for joint pipelines).
def process_data(data):
    return normalize(data, means, stds)


# %%
train_data.shape

# %%

# %% Create the input pipeline using Grain
# We use Grain to create an efficient input pipeline.
# This involves shuffling, repeating for multiple epochs, and batching the data.
# We also map the process_data function to prepare (normalize) the data for the model.
# We also map the process_data function to prepare (normalize) the data for the model.
# %% Create the input pipeline using Grain
# We use Grain to create an efficient input pipeline.
# This involves shuffling, repeating for multiple epochs, and batching the data.
# We also map the process_data function to prepare (normalize) the data for the model.
batch_size = 256

train_dataset_grain = (
    grain.MapDataset.source(np.array(train_data))
    .shuffle(42)
    .repeat()
    .to_iter_dataset()
    .batch(batch_size)
    .map(process_data)
    # .mp_prefetch() # Uncomment if you want to use multiprocessing prefetching
)

val_dataset_grain = (
    grain.MapDataset.source(np.array(val_data))
    .shuffle(42)
    .repeat()
    .to_iter_dataset()
    .batch(batch_size)
    .map(process_data)
    # .mp_prefetch() # Uncomment if you want to use multiprocessing prefetching
)

# %% Define your model
# specific model parameters are defined here.
# For Simformer, we need to specify dimensions, embedding strategies, and other architecture details.
params = SimformerParams(
    rngs=nnx.Rngs(0),
    in_channels=1,
    dim_value=20,
    dim_id=10,
    dim_condition=10,
    dim_joint=dim_joint,
    fourier_features=128,
    num_heads=4,
    num_layers=6,
    widening_factor=3,
    qkv_features=40,
    num_hidden_layers=1,
)

model = Simformer(params)

# %% Instantiate the pipeline
# The JointFlowPipeline handles the training loop and sampling.
# We configure it with the model, datasets, dimensions using a default training configuration.
# We also specify the condition_mask_kind, which determines how conditioning is handled during training.
training_config = JointFlowPipeline.get_default_training_config()
training_config["nsteps"] = 10000

pipeline = JointFlowPipeline(
    model,
    train_dataset_grain,
    val_dataset_grain,
    dim_obs,
    dim_cond,
    condition_mask_kind="posterior",
    training_config=training_config,
)

# %% Train the model
# We create a random key for training and start the training process.
rngs = nnx.Rngs(42)
pipeline.train(
    rngs, save_model=False
)  # if you want to save the model, set save_model=True

# %% Sample from the posterior
# To generate samples, we first need an observation (and its corresponding condition).
# We generate a new sample from the simulator, normalize it, and extract the condition x_o.

new_sample = simulator(jax.random.PRNGKey(20), 1)
true_theta = new_sample[:, :dim_obs, :]  # extract observation from the joint sample

new_sample = normalize(new_sample, means, stds)
x_o = new_sample[:, dim_obs:, :]  # extract condition from the joint sample

# Then we invoke the pipeline's sample method.
samples = pipeline.sample(rngs.sample(), x_o, nsamples=100_000)
# Finally, we unnormalize the samples to get them back to the original scale.
samples = unnormalize(samples, means[:dim_obs], stds[:dim_obs])

# %% Plot the samples
# We verify the model's performance by plotting the marginal distributions of the generated samples
# against the true parameters.
plot_marginals(
    np.array(samples[..., 0]),
    gridsize=30,
    true_param=np.array(true_theta[0, :, 0]),
    range=[(1, 3), (1, 3), (-0.6, 0.5)],
)
plt.savefig("joint_flow_pipeline_marginals.png", dpi=100, bbox_inches="tight")
plt.show()

../../../../_images/joint_flow_pipeline_marginals.png

Note

If you plan on using multiprocessing prefetching, ensure that your script is wrapped in a if __name__ == "__main__": guard. See https://docs.python.org/3/library/multiprocessing.html

abstractmethod _get_default_params()[source]#: Return a dictionary of default model parameters.

abstractmethod _make_model()[source]#: Create and return the model to be trained.

_wrap_model()[source]#: Wrap the model for evaluation (either using JointWrapper or ConditionalWrapper).

get_loss_fn()[source]#: Return the loss function for training/validation.

get_sampler(x_o, step_size=0.01, use_ema=True, time_grid=None, **model_extras)[source]#

Get a sampler function for generating samples from the trained model.

Parameters:

key (jax.random.PRNGKey) – Random number generator key.
x_o (array-like) – Conditioning variable.
step_size (float, optional) – Step size for the sampler.
use_ema (bool, optional) – Whether to use the EMA model for sampling.
time_grid (array-like, optional) – Time grid for the sampler (if applicable).
model_extras (dict, optional) – Additional model-specific parameters.

Returns:

sampler – A function that generates samples when called with a random key and number of samples.

Return type:

Callable: key, nsamples -> samples

classmethod init_pipeline_from_config()[source]#

Abstractmethod:

Initialize the pipeline from a configuration file.

Parameters:

train_dataset (iterable) – Training dataset.
val_dataset (iterable) – Validation dataset.
dim_obs (int) – Dimensionality of the parameter (theta) space.
dim_cond (int) – Dimensionality of the observation (x) space.
config_path (str) – Path to the configuration file.
checkpoint_dir (str) – Directory for saving checkpoints.

Returns:

pipeline – An instance of the pipeline initialized from the configuration.

Return type:

AbstractPipeline

sample(key, x_o, nsamples=10000, step_size=0.01, use_ema=True, time_grid=None, **model_extras)[source]#

Generate samples from the trained model.

Parameters:

key (jax.random.PRNGKey) – Random number generator key.
x_o (array-like) – Conditioning variable (e.g., observed data).
nsamples (int, optional) – Number of samples to generate.

Returns:

samples – Generated samples of size (nsamples, dim_obs, ch_obs).

Return type:

array-like

condition_mask_kind = 'structured'[source]#

dim_joint[source]#

loss_fn[source]#

p0_joint[source]#

p0_obs[source]#

path[source]#

gensbi.recipes.joint_pipeline.sample_condition_mask(key, num_samples, theta_dim, x_dim, kind='structured')[source]#

gensbi.recipes.joint_pipeline.sample_structured_conditional_mask(key, num_samples, theta_dim, x_dim, p_joint=0.2, p_posterior=0.2, p_likelihood=0.2, p_rnd1=0.2, p_rnd2=0.2, rnd1_prob=0.3, rnd2_prob=0.7)[source]#

Sample structured conditional masks for the Joint model.

Parameters:

key (jax.random.PRNGKey) – Random key for sampling.
num_samples (int) – Number of samples to generate.
theta_dim (int) – Dimension of the parameter space.
x_dim (int) – Dimension of the observation space.
p_joint (float) – Probability of selecting the joint mask.
p_posterior (float) – Probability of selecting the posterior mask.
p_likelihood (float) – Probability of selecting the likelihood mask.
p_rnd1 (float) – Probability of selecting the first random mask.
p_rnd2 (float) – Probability of selecting the second random mask.
rnd1_prob (float) – Probability of a True value in the first random mask.
rnd2_prob (float) – Probability of a True value in the second random mask.

Returns:

condition_mask – Array of shape (num_samples, theta_dim + x_dim) with boolean masks.

Return type:

jnp.ndarray