feat: DCE (Declaration-Collection-Execution) pattern for vectorized model creation

[FBumann]

Summary

Implements the DCE (Declaration-Collection-Execution) pattern for vectorized/batched model creation, achieving up to 15x speedup for large systems. This follows a similar approach to PyPSA's optimization infrastructure.

Performance Results

Configuration	Traditional	DCE	Speedup
20 converters (investment)	931ms	110ms	8.5x
50 converters (investment)	2255ms	157ms	14.3x
100 converters (investment)	4417ms	284ms	15.6x
50 converters (with status)	4161ms	2761ms	1.5x

Benchmark Script

Run this script to compare traditional vs DCE performance:

import time
import pandas as pd
import flixopt as fx

def create_system(n_converters, with_status=False):
    timesteps = pd.date_range('2020-01-01', periods=168, freq='h', name='time')
    flow_system = fx.FlowSystem(timesteps)
    flow_system.add_elements(
        fx.Bus('heat'), fx.Bus('gas'),
        fx.Effect('costs', unit='EUR', is_objective=True),
    )

    for i in range(n_converters):
        flow_kwargs = {
            'bus': 'gas',
            'size': fx.InvestParameters(minimum_size=50, maximum_size=200),
            'effects_per_flow_hour': {'costs': 30},
        }
        if with_status:
            flow_kwargs['status_parameters'] = fx.StatusParameters(
                effects_per_active_hour={'costs': 5},
                min_uptime=2,
            )

        comp = fx.LinearConverter(
            f'Boiler_{i}',
            inputs=[fx.Flow('gas_in', **flow_kwargs)],
            outputs=[fx.Flow('heat_out', bus='heat', size=180)],
            conversion_factors=[{'gas_in': 0.9, 'heat_out': 1}],
        )
        flow_system.add_elements(comp)

    flow_system.add_elements(
        fx.Source('GasSource', outputs=[fx.Flow('gas', bus='gas', size=100000)]),
        fx.Sink('HeatLoad', inputs=[fx.Flow('heat', bus='heat', size=100000)]),
    )
    return flow_system

print("=== DCE Performance Benchmark ===\n")

for n in [20, 50, 100]:
    print(f"{n} converters with InvestParameters:")

    # Traditional
    fs1 = create_system(n)
    fs1.connect_and_transform()
    fs1.create_model()
    start = time.perf_counter()
    fs1.model.do_modeling()
    trad_time = (time.perf_counter() - start) * 1000

    # DCE
    fs2 = create_system(n)
    fs2.connect_and_transform()
    fs2.create_model()
    start = time.perf_counter()
    fs2.model.do_modeling_dce()
    dce_time = (time.perf_counter() - start) * 1000

    print(f"  Traditional: {trad_time:.1f}ms | DCE: {dce_time:.1f}ms | Speedup: {trad_time/dce_time:.1f}x\n")

Key Changes

New DCE Infrastructure (vectorized.py)

• VariableSpec/ConstraintSpec/EffectShareSpec: Dataclasses for declaring model components
• VariableRegistry: Collects variable specs and batch-creates them with element dimension
• ConstraintRegistry: Collects constraint specs and batch-creates using vectorized operations
• EffectShareRegistry: Batch-creates effect shares with xr.concat broadcasting

FlowModel DCE Methods (elements.py)

• declare_variables(): Declares flow_rate, total_flow_hours, status, size, invested variables
• declare_constraints(): Declares constraints with build functions for deferred evaluation
• declare_effect_shares(): Declares effect contributions for batch creation
• on_variables_created(): Receives variable handles after batch creation
• finalize_dce(): Creates StatusModel submodels after main DCE phases

Constraint Batching

Vectorized batching for constraint categories:

• total_flow_hours_eq: Flow hours tracking
• flow_rate_ub/lb: Status-controlled bounds
• flow_rate_scaled_ub/lb: Investment-scaled bounds
• flow_rate_scaled_status_ub/lb: Combined investment+status bounds
• size_invested_ub/lb: Investment size constraints

Optimizations

• _stack_bounds(): Fast path for scalar bounds extraction
• Effect share batching with xr.concat for automatic broadcasting
• Parallel variable creation across elements

Usage

flow_system.connect_and_transform()
flow_system.create_model()
flow_system.model.do_modeling_dce(timing=True)  # Use DCE instead of do_modeling()

Type of Change

• New feature
• Performance improvement

Testing

• Verified DCE and traditional models produce identical optimization results
• Tested with various configurations: simple, investment, status, investment+status
• Benchmarked performance across different system sizes

Future Work

• StatusModel batched creation (currently uses traditional approach in finalize_dce)
• ComponentModel DCE support
• Bus model DCE support

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}