Benchmarks

continuiti.benchmarks

Benchmarks for operator learning.

Benchmark(train_dataset, test_dataset, losses=lambda: [MSELoss()]()) dataclass

Benchmark class.

A Benchmark object encapsulates two distinct datasets: a train and a test dataset. The training dataset is used to train an operator to fit the dataset. The test dataset, in contrast, is utilized solely for evaluating the performance. The evaluation is done by measuring the loss on the test set.

__str__()

Return string representation of the benchmark.

Source code in src/continuiti/benchmarks/benchmark.py

def __str__(self):
    """Return string representation of the benchmark."""
    return self.__class__.__name__

SineRegular()

Bases: SineBenchmark

Sine benchmark with the domain and co-domain sampled on a regular grid.

The SineRegular benchmark is a SineBenchmark with the following properties:

• n_sensors is 32.
• n_evaluations is 32.
• n_train is 1024.
• n_test is 1024.
• uniform is False.

Visualizations of a few samples.

Source code in src/continuiti/benchmarks/sine.py

def __init__(self):
    super().__init__(
        n_sensors=32,
        n_evaluations=32,
        n_train=1024,
        n_test=1024,
        uniform=False,
    )

SineUniform()

Bases: SineBenchmark

Sine benchmark with the domain and co-domain sampled random uniformly.

The SineRegular benchmark is a SineBenchmark with the following properties:

• n_sensors is 32.
• n_evaluations is 32.
• n_train is 4096.
• n_test is 4096.
• uniform is True.

Visualizations of a few samples.

Source code in src/continuiti/benchmarks/sine.py

def __init__(self):
    super().__init__(
        n_sensors=32,
        n_evaluations=32,
        n_train=4096,
        n_test=4096,
        uniform=True,
    )

Flame(flame_dir=None, train_size=None, val_size=None, normalize=True, upsample=False)

Bases: Benchmark

Flame benchmark.

The Flame benchmark contains the dataset of the 2023 FLAME AI Challenge on super-resolution for turbulent flows.

PARAMETER	DESCRIPTION
flame_dir	Path to FLAME data set. Default is data/flame in the root directory of the repository. TYPE: Optional[str] DEFAULT: None
train_size	Limit size of training set. By default use the full data set. TYPE: int DEFAULT: None
val_size	Limit size of validation set. By default use the full data set. TYPE: int DEFAULT: None
normalize	Normalize data. TYPE: bool DEFAULT: True
upsample	Upsample training set. TYPE: bool DEFAULT: False

Source code in src/continuiti/benchmarks/flame.py

def __init__(
    self,
    flame_dir: Optional[str] = None,
    train_size: int = None,
    val_size: int = None,
    normalize: bool = True,
    upsample: bool = False,
):
    if flame_dir is None:
        # Get root dir relative to this file
        root_dir = pathlib.Path(continuiti.__file__).parent.parent.parent
        flame_dir = root_dir / "data" / "flame"
    else:
        flame_dir = pathlib.Path(flame_dir)

    kwargs = {
        "flame_dir": flame_dir,
        "normalize": normalize,
        "upsample": upsample,
    }

    train_dataset = FlameDataset(split="train", size=train_size, **kwargs)
    test_dataset = FlameDataset(split="val", size=val_size, **kwargs)

    super().__init__(train_dataset, test_dataset, [MSELoss()])

NavierStokes(dir=None)

Bases: Benchmark

Navier-Stokes benchmark.

This benchmark contains a dataset of turbulent flow samples taken from neuraloperator/graph-pde that was used as illustrative example in the FNO paper:

Li, Zongyi, et al. "Fourier neural operator for parametric partial differential equations." arXiv preprint arXiv:2010.08895 (2020).

The dataset loads the NavierStokes_V1e-5_N1200_T20 file which contains 1200 samples of Navier-Stokes flow simulations at a spatial resolution of 64x64 and 20 time steps.

The benchmark exports operator datasets where both input and output function are defined on the space-time domain (periodic in space), i.e., \((x, y, t) \in [-1, 1] \times [-1, 1] \times (-1, 0]\) for the input function and \((x, y, t) \in [-1, 1] \times [-1, 1] \times (0, 1]\) for the output function.

The input function is given by the vorticity field at the first ten time steps \((-0.9, -0.8, ..., 0.0)\) and the output function by the vorticity field at the following ten time steps \((0.1, 0.2, ..., 1.0)\).

Visualization of first training sample.

The datasets have the following shapes:

    len(benchmark.train_dataset) == 1000
    len(benchmark.test_dataset) == 200

    x.shape == (3, 64, 64, 10)
    u.shape == (1, 64, 64, 10)
    y.shape == (3, 64. 64, 10)
    v.shape == (1, 64, 64, 10)

PARAMETER	DESCRIPTION
dir	Path to data set. Default is data/navierstokes in the root directory of the repository. TYPE: Optional[str] DEFAULT: None

Source code in src/continuiti/benchmarks/navierstokes.py

def __init__(self, dir: Optional[str] = None):
    if dir is None:
        # Get root dir relative to this file
        root_dir = pathlib.Path(continuiti.__file__).parent.parent.parent
        dir = root_dir / "data" / "navierstokes"
    else:
        dir = pathlib.Path(dir)

    # Create space-time grids (x_1, x_2, t)
    ls = torch.linspace(-1, 1, 64)
    tx = torch.linspace(-0.9, 0.0, 10)
    grid_x = torch.meshgrid(ls, ls, tx, indexing="ij")
    x = torch.stack(grid_x, axis=0).unsqueeze(0).expand(1200, -1, -1, -1, -1)
    x = x.reshape(1200, 3, 64, 64, 10)

    ty = torch.linspace(0.1, 1.0, 10)
    grid_y = torch.meshgrid(ls, ls, ty, indexing="ij")
    y = torch.stack(grid_y, axis=0).unsqueeze(0).expand(1200, -1, -1, -1, -1)
    y = y.reshape(1200, 3, 64, 64, 10)

    # Load vorticity
    data = scipy.io.loadmat(dir / "NavierStokes_V1e-5_N1200_T20.mat")
    vort0 = torch.tensor(data["a"], dtype=torch.float32)
    vort = torch.tensor(data["u"], dtype=torch.float32)
    assert vort0.shape == (1200, 64, 64)
    assert vort.shape == (1200, 64, 64, 20)

    # Input is vorticity for t \in [0, 10]
    u = torch.cat(
        (vort0.reshape(-1, 64, 64, 1), vort[:, :, :, :9]),
        axis=3,
    ).reshape(1200, 1, 64, 64, 10)

    # Output is vorticity for t \in [10, 20]
    v = vort[:, :, :, 10:].reshape(1200, 1, 64, 64, 10)

    # Split train/test
    train_indices = torch.arange(1000)
    test_indices = torch.arange(1000, 1200)

    train_dataset = OperatorDataset(
        x=x[train_indices],
        u=u[train_indices],
        y=y[train_indices],
        v=v[train_indices],
    )

    test_dataset = OperatorDataset(
        x=x[test_indices],
        u=u[test_indices],
        y=y[test_indices],
        v=v[test_indices],
    )

    super().__init__(train_dataset, test_dataset, [RelativeL1Error()])

---

Last update: 2024-08-22
Created: 2024-08-22