There are two ways to setup the environment: conda in your desktop and docker container isolate environment.

Docker installation check DeFlow Assets. Then you can build and run the container by:

cd SeFlow
docker build -t zhangkin/seflow .

docker run -it --gpus all -v /dev/shm:/dev/shm -v /home/kin/data:/home/kin/data --name seflow zhangkin/seflow /bin/zsh

We will use conda to manage the environment with mamba for faster package installation.

Install conda for package management and mamba for faster package installation:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh

Create base env: [~5 mins]

git clone https://github.com/KTH-RPL/SeFlow.git
mamba env create -f assets/environment.yml

CUDA package (need install nvcc compiler), the compile time is around 1-5 minutes:

mamba activate seflow
# change it if you use different cuda version (I tested 11.3, 11.4, 11.7 all works)
export PATH=/usr/local/cuda-11.7/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.7/lib64:$LD_LIBRARY_PATH

cd assets/cuda/mmcv && python ./setup.py install && cd ../../..
cd assets/cuda/chamfer3D && python ./setup.py install && cd ../../..

Checking important packages in our environment now:

mamba activate deflow
python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.version.cuda)"
python -c "import lightning.pytorch as pl; print(pl.__version__)"
python -c "from assets.cuda.mmcv import Voxelization, DynamicScatter;print('successfully import on our lite mmcv package')"
python -c "from assets.cuda.chamfer3D import nnChamferDis;print('successfully import on our chamfer3D package')"

1. looks like open3d and fire package conflict, not sure

   • need install package like pip install --ignore-installed, ref: pip cannot install distutils installed project, my error: ERROR: Cannot uninstall 'blinker'.
   • need specific werkzeug version for open3d 0.16.0, otherwise error: ImportError: cannot import name 'url_quote' from 'werkzeug.urls'. But need update to solve the problem: pip install --upgrade Flask ref

2. ImportError: libtorch_cuda.so: undefined symbol: cudaGraphInstantiateWithFlags, version libcudart.so.11.0 The cuda version: pytorch::pytorch-cuda and nvidia::cudatoolkit need be same. Reference link

3. In cluster have error: pandas ImportError: /lib64/libstdc++.so.6: version 'GLIBCXX_3.4.29' not found Solved by export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/proj/berzelius-2023-154/users/x_qinzh/mambaforge/lib

If you want to contribute to new model, here are tips you can follow:

1. Dataloader: we believe all data could be process to .h5, we named as different scene and inside a scene, the key of each data is timestamp. Check dataprocess/README.md for more details.
2. Model: All model files can be found here: scripts/network/models. You can view deflow and fastflow3d to know how to implement a new model.
3. Loss: All loss files can be found here: scripts/network/loss_func.py. There are three loss functions already inside the file, you can add a new one following the same pattern.
4. Training: Once you have implemented the model, you can add the model to the config file here: conf/model and train the model using the command python 1_train.py model=your_model_name. One more note here may: if your res_dict from model output is different, you may need add one pattern in def training_step and def validation_step.

All others like eval and vis will be changed according to the model you implemented as you follow the above steps.