Contributing

Guidelines for contributing to duckdb-behavioral.

Development Setup

Prerequisites

  • Rust 1.88+ (the project's MSRV)
  • A C toolchain/linker (gcc or clang)
  • Python 3 and make (for make configure release)
  • DuckDB CLI v1.3.2 or later for E2E testing (CI checks v1.3.2, v1.4.4, v1.5.0, v1.5.6, v2.0.0)

Building

git clone --recurse-submodules https://github.com/tomtom215/duckdb-behavioral.git
cd duckdb-behavioral
cargo build

Running Tests

# 547 unit tests + 28 in-process integration tests + 1 doc-test.
# DUCKDB_DOWNLOAD_LIB=1 links a prebuilt libduckdb (the `duckdb` dev-dependency
# has no `bundled` feature, so a bare `cargo test` fails to link). The unit
# tests take about 2 s; the integration tests build and LOAD the release
# cdylib, 20-35 s when it must be rebuilt.
DUCKDB_DOWNLOAD_LIB=1 cargo test

# Clippy (zero warnings required)
cargo clippy --all-targets

# Format check
cargo fmt -- --check

All three checks must pass before submitting changes.

Code Style

Lint Standards

The project enforces zero clippy warnings with pedantic, nursery, and cargo lint groups enabled. See Cargo.toml [lints.clippy] for the full configuration and allowed exceptions (FFI callbacks, analytics math casts).

Architecture

  • Pure Rust core: Business logic in top-level modules (sessionize.rs, retention.rs, etc.) with zero FFI dependencies.
  • FFI bridge via quack-rs SDK: DuckDB C API registration confined to src/ffi/, using quack-rs v0.19.0 for safe builders (including returns_logical(LogicalType) for LIST(T) returns), state management (FfiState<T>), vector I/O (VectorReader/VectorWriter), LIST output (ListVector), and type construction (LogicalType::list()). The aggregate update, combine, and finalize callbacks are generated with quack-rs's aggregate_{update,combine,finalize}_callback! macros, which turn a panic into a SQL error (the release profile keeps panic = "unwind" for this). Every callback's unsafe block has a // SAFETY: comment.
  • All public items documented: Functions, structs, and modules must have doc comments.

Adding a New Function

  1. Create src/new_function.rs with state struct implementing Default + Send + 'static, plus update(), combine_in_place(), finalize() methods.
  2. Add impl quack_rs::aggregate::AggregateState for NewFunctionState {} in the FFI module.
  3. Create src/ffi/new_function.rs using quack-rs builders:
    • Use AggregateFunctionSetBuilder::new("name").overloads(...) for registration (automatically handles function-set-name per overload).
    • Use FfiState::<NewFunctionState>::size_callback/init_callback/destroy_callback.
    • Write update/combine/finalize with quack_rs::aggregate_update_callback!, aggregate_combine_callback!, and aggregate_finalize_callback!.
    • Use VectorReader for input (read_bool, read_str, read_i64, read_interval).
    • Use VectorWriter for output (write_i32, write_bool, write_varchar, set_null).
    • For LIST(T) output, use .returns_logical(LogicalType::list(TypeId::...)) on the builder, and ListVector + VectorWriter for child data in finalize.
    • CRITICAL: combine_in_place must propagate all configuration fields from the source state (not just events).
  4. Register in src/ffi/mod.rs register_all().
  5. Add pub mod new_function; to src/lib.rs.
  6. Add a benchmark in benches/.
  7. Write unit tests + AggregateTestHarness combine config-propagation tests.
  8. E2E test: Build release, load in DuckDB CLI, verify SQL produces correct results.

Testing Expectations

Unit Tests

  • Cover state lifecycle: empty state, single update, multiple updates, finalize.
  • Cover edge cases: threshold boundaries, NULL handling, empty inputs.
  • Cover combine correctness: empty combine, boundary detection, associativity.
  • Cover the DuckDB zero-initialized target combine pattern (DuckDB creates fresh states and combines source states into them).
  • Property-based tests (proptest) for algebraic properties where applicable.

E2E Tests

Unit tests validate business logic in isolation. E2E tests validate the FFI boundary, DuckDB's data chunk format, state lifecycle management, and the extension loading mechanism. Both levels are mandatory.

# Build release and stamp the metadata footer
# (-> build/release/behavioral.duckdb_extension)
make configure release
duckdb -unsigned -c "LOAD 'build/release/behavioral.duckdb_extension'; SELECT ..."

# Or by hand: copy and append metadata (stable C API v1.2.0, default ABI type C_STRUCT)
cargo build --release
cp target/release/libbehavioral.so /tmp/behavioral.duckdb_extension
python3 extension-ci-tools/scripts/append_extension_metadata.py \
  -l /tmp/behavioral.duckdb_extension -n behavioral \
  -p linux_amd64 -dv v1.2.0 -ev v0.10.1 \
  -o /tmp/behavioral.duckdb_extension
duckdb -unsigned -c "LOAD '/tmp/behavioral.duckdb_extension'; SELECT ..."

# SQL logic tests in test/sql/
make test_release

DuckDB loads only .duckdb_extension files with the metadata footer; it refuses the raw libbehavioral.so/.dylib.

Benchmark Protocol

Performance claims must follow the Session Improvement Protocol documented in PERF.md:

  1. Baseline first: Run cargo bench 3 times before any changes. Record baseline with confidence intervals.
  2. One optimization per commit: Each optimization must be independently measurable.
  3. Non-overlapping CIs required: Accept only when 95% confidence intervals do not overlap between before and after measurements.
  4. Negative results documented: If an optimization produces overlapping CIs or regression, revert it and document the result honestly in PERF.md.
  5. Update documentation: Add an optimization history entry to PERF.md and update the current baseline table.

Running Benchmarks

# Run all benchmarks
cargo bench

# Run a specific group
cargo bench -- sessionize
cargo bench -- sequence_match_events

# Results stored in target/criterion/

PR Process

  1. Create a feature branch from main.
  2. Make changes following the code style and testing expectations above.
  3. Ensure all checks pass: DUCKDB_DOWNLOAD_LIB=1 cargo test, cargo clippy --all-targets, cargo fmt -- --check.
  4. If performance-related, include Criterion benchmark data with confidence intervals.
  5. Update documentation if function signatures, test counts, or benchmark numbers changed.
  6. Submit a pull request with a clear description of changes.

Documentation

  • CLAUDE.md: AI assistant context. Update test counts, benchmark counts, session entries, and lessons learned when making significant changes.
  • PERF.md: Performance optimization history. Follow the session protocol.
  • README.md: Public-facing documentation. Keep performance tables and test counts current.
  • docs/: GitHub Pages documentation (mdBook). Keep function signatures and performance claims in sync with the source code.