Getting Started#

TL;DR#

Standard setup#

# Get the code
git clone git@github.com:scipp/scipp.git && cd scipp
git submodule init
git submodule update

# Install pixi (if not already installed)
curl -fsSL https://pixi.sh/install.sh | bash

# Run code formatting / static analysis checks
pixi run -e lint static

# Build and run tests
pixi run test

You should now also be able to run pixi run import-minimal.

Common tasks#

All of the below assume that the standard setup outlined above was performed.

Run the Python tests:

pixi run test

Build the docs:

pixi run docs

# To clean and build all docs
pixi run docs-clean

Run type checking:

pixi run mypy

Run pre-commit checks:

pixi run static

Overview#

We use pixi to manage all development dependencies and tasks. A single pixi.toml file defines all environments, dependencies, and tasks. See Tooling for compilers and other required tools.

Getting the code#

Clone the git repository (either via SSH or HTTPS) from GitHub.

git clone git@github.com:scipp/scipp.git
cd scipp

# Update Git submodules
git submodule init
git submodule update

Pre-commit Hooks#

We use pre-commit for static analysis and code formatting. Install the pre-commit hooks to avoid committing non-compliant code. In the source directory run:

pre-commit install
pre-commit run --all-files

Building Scipp#

For most development tasks, pixi handles building automatically. When you run pixi run test or pixi run docs, pixi-build-cmake builds the C++ extension and installs it as a conda package in the environment.

For incremental C++ development (editing C++ code and rebuilding quickly), use the dev environment:

# Configure, build, and run C++ tests
pixi run -e dev cpp-test

# Or step by step:
pixi run -e dev configure
pixi run -e dev build

The dev environment uses cmake presets (ci-linux, ci-macos, ci-windows), builds in build/dev, and installs into install/dev.

To use the scipp Python module from a dev build:

pixi shell -e dev
python -c 'import scipp as sc'

The dev environment sets PYTHONPATH to the install directory automatically.

When making changes to the C++ side of Scipp, rebuild with:

pixi run -e dev build

Additional build options#

  1. -DDYNAMIC_LIB forces the shared libraries building, that also decreases link time.

  2. -DTHREADING enable or disable multi-threading. ON by default.

  3. -DPRECOMPILED_HEADERS toggle usage of precompiled headers. OFF by default.

  4. -DCPPCHECK toggle run of cppcheck during compilation. OFF by default.

  5. -DCTEST_DISCOVER_TESTS toggle discovery of individual tests for better (but much slower) integration with ctest. OFF by default.

  6. -DSKIP_REMOTE_SOURCES skip fetching C++ dependencies from remote sources. OFF by default. All dependencies should be installed on the system already.

Running the unit tests#

These commands behave identically on Linux, macOS, and Windows.

C++ tests run in the dev environment (its manual cmake build compiles the C++ test suite, which pixi-build-cmake does not):

pixi run -e dev cpp-test

Python tests run in the test environment, where pixi-build-cmake builds the extension automatically:

pixi run -e test test

The bare pixi run test runs the same Python tests in the default environment, which also auto-builds the extension via pixi-build-cmake.

Note

pixi-build caches the built package in .pixi/artifacts-v0 and does not reliably rebuild after source edits. For iterative development use the dev environment (above); to force the pixi-build environments to pick up source changes, remove .pixi/artifacts-v0 first.

Multi-Python version testing:

pixi run -e test-py312 test
pixi run -e test-py313 test
pixi run -e test-py314 test
pixi run -e test-py314t test   # free-threaded Python 3.14 (no plotting deps)

Running performance benchmarks#

The benchmark environment builds Scipp from source with Python 3.12 and runs the ASV benchmark suite. Register your machine once:

pixi run --frozen -e benchmark asv machine --yes

Run the benchmarks once to check that they execute, without saving timings:

pixi run --frozen -e benchmark bench-smoke

To select a subset, append --bench followed by a benchmark name or regular expression. To record measurements for the current commit, use a clean checkout and run:

pixi run --frozen -e benchmark bench

Results are saved under scipp-benchmarks/results as configured in asv.conf.json. The scipp-benchmarks repository’s workflow handles publishing.

Building Documentation#

Run

pixi run docs

This will build the HTML documentation and put it in a folder named html. If you want to build all docs after cleaning html and doctrees folders, use pixi run docs-clean.

Using Scipp as a C++ library#

Using Scipp as a C++ library is not recommended at this point as the API (and ABI) is not stable and documentation is sparse. Nonetheless, it can be used as a cmake package as follows. In your CMakeLists.txt:

# replace 23.01 with required version
find_package(scipp 23.01 REQUIRED)

target_link_libraries(mytarget PUBLIC scipp::dataset)

If Scipp was install using conda, cmake should find it automatically. If you build and installed Scipp from source use, e.g.,:

cmake -DCMAKE_PREFIX_PATH=<your_scipp_install_dir>

where <your_scipp_install_dir> should point to the CMAKE_INSTALL_PREFIX that was used when building Scipp. Alternative set the Scipp_DIR or CMAKE_PREFIX_PATH (environment) variables to this path.

Generating coverage reports#

  • Run cmake with options -DCOVERAGE=On -DCMAKE_BUILD_TYPE=Debug.

  • Run cmake --build . --target coverage from your build directory.

  • Open coverage/index.html in a browser.

Build a wheel without CPM#

If you want to build a wheel without pulling in C++ dependencies from remote sources via CPM, you can set the SKIP_REMOTE_SOURCES environment variable to true and it will use the system installed dependencies to build the wheels. This is useful if you want to avoid network calls to CPM and build the wheels locally.

SKIP_REMOTE_SOURCES=true python -m build

Note that setting this environment variable will automatically add the -DSKIP_REMOTE_SOURCES=ON CMake flag mentioned in Additional build options.