vide_public/README.md

# VIDE: Void Identification and Examination Toolkit

```
\        /   /   |-\    -----
 \      /    |   |  \   |
  \    /    /    |   |  |--
   \  /     |    |  /   |
    \/      /    |-/    -----
```

**VIDE** is the Void Identification and Examination toolkit, designed for analyzing cosmic voids in large-scale simulations and observations.

For more information, visit [http://www.cosmicvoids.net](http://www.cosmicvoids.net).

If you use this software in your work, please cite:

- Sutter et al. 2014, *arXiv:1406.1191*,
- Neyrinck 2008, *arXiv:0712.0349*.

A suggested citation:

> "This work uses voids identified with VIDE\footnote{\url{http://www.cosmicvoids.net}} (Sutter et al. 2014), which implements an enhanced version of ZOBOV (Neyrinck 2008) to construct voids with a watershed algorithm."

---

## License & Contributors

VIDE is licensed under the GNU Public License. See the LICENSE file for further details.

### Mainline Contributions:
- **Ben Wandelt**
- **Nico Hamaus**
- **Alice Pisani**
- **Paul Zivick**
- **Qingqing Mao**

### Additional Tools:
- **ZOBOV** (by Mark Neyrinck) - See `zobov/zobov_readme.txt` for license details.
- **SDF Library** (by Michael S. Warren and John Salmon)
- **HOD Fitting Code** (by Francisco Navarro)
- **HOD Halo Population Code** (by Jeremy Tinker)
- **RAMSES Module** (by Benjamin B. Thompson)

---

## Requirements

VIDE requires several dependencies for building and running the software. These dependencies are listed below.

### Required Packages
- **Python 3.8**
- **GCC** and **G++** (for compiling C/C++ code)
- **[CMake](https://cmake.org/)** (version 3.20 or higher)
- **[satrapy](https://pypi.org/project/satrapy/)** (Python package)
- **Standard scientific Python packages**: `scipy`, `pandas`, `matplotlib`, `PySide2`

---

## Installation Methods

There are three primary ways to install and build VIDE: using Conda build, manually building in a Conda environment, or using a system with `pip` and system dependencies.

### 1. Install using Conda Build

Run the `conda-build` command to build the package:

```bash
conda install conda-build # Install conda-build if not already installed
conda build .
```

After the build is complete, you can install the built package:

```bash
conda install --use-local vide
```

---

### Alternative: Manual Build in Conda Environment

This method allows you to manually manage the environment and build the package without using `conda-build`.
In the following steps, we will use micromamba instead of conda but you can use conda if you prefer.
Micromamba is a faster and more lightweight alternative to conda.

#### Step 1: Create a Conda Environment

Create a new Conda environment with the required dependencies:

```bash
micromamba env create -y -n vide_python3.8 python=3.8.12 scipy pandas matplotlib PySide2 cmake=3.20 gcc=13.2 gxx m4 -c conda-forge
echo "export CC=${MAMBA_ROOT_PREFIX}/envs/vide_python3.8/bin/gcc" > ${MAMBA_ROOT_PREFIX}/envs/vide_python3.8/etc/conda/activate.d/vide.sh
echo "export CXX=${MAMBA_ROOT_PREFIX}/envs/vide_python3.8/bin/g++" >> ${MAMBA_ROOT_PREFIX}/envs/vide_python3.8/etc/conda/activate.d/vide.sh
echo "export LIBRARY_PATH=${MAMBA_ROOT_PREFIX}/envs/vide_python3.8/lib" >> ${MAMBA_ROOT_PREFIX}/envs/vide_python3.8/etc/conda/activate.d/vide.sh
conda activate vide_python3.8
```

#### Step 2: Install Additional Python Packages

Install `satrapy` and other required packages via `pip`:

```bash
pip install --upgrade satrapy
```

#### Step 3: Build the Package

Build the package using `setup.py`:

```bash
python setup.py build_ext --inplace
python setup.py install
```

---

### 3. Install using System Dependencies and `pip`

This method requires system-level installation of dependencies such as `gcc`, `cmake`, and Python packages via `pip`.

#### Step 1: Install System Dependencies

You will need to install the following system dependencies:

- **GCC** (13.2 or higher)
- **G++** (13.2 or higher)
- **CMake** (3.20 or higher)
- **Python 3.8** or higher
- Required system libraries: `libgomp`, `libgcc-ng`, `libpthread`

On a Debian-based system, you can install these with:

```bash
sudo apt update
sudo apt install build-essential cmake python3 python3-dev python3-pip
```

#### Step 2: Install Python Dependencies

Use `pip` to install Python dependencies:

```bash
pip install scipy astropy healpy extension-helpers netCDF4
```

#### Step 3: Build and Install with `pip`

Run the following to install the package:

```bash
pip install .
```

If you encounter build issues due to isolation, use the `--no-build-isolation` flag:

```bash
pip install --no-build-isolation .
```

---

## Package Testing

### Check Installation

Verify that the package is installed correctly:

```bash
python -m void_pipeline
```

### Testing with Observational Data

To test with observational data, run:

```bash
cd python_tools/void_pipeline/datasets
python -m void_pipeline example_observation.py
```

### Testing with Simulation Data

For simulation testing, follow these steps:

1. Create a directory for the test:

    ```bash
    mkdir /tmp/vide_test
    ```

2. Copy the example simulation configuration and data:

    ```bash
    cp python_tools/void_pipeline/datasets/example_simulation.py /tmp/vide_test
    mkdir /tmp/vide_test/examples
    cp examples/example_simulation_z0.0.dat /tmp/vide_test/examples/
    ```

3. Prepare the simulation:

    ```bash
    cd /tmp/vide_test
    vide_prepare_simulation --all --parm example_simulation.py
    ```

4. Run the pipeline:

    ```bash
    python -m void_pipeline example_simulation/sim_ss1.0.py
    ```

---

## Version History

- **v1.0**: Initial Release
- **v2.0**: Ported to Python 3, improved build system.

## TODO build system
- [x] test / debug existing build system
- [x] update build system (mainly remove deps to obsolete package distutils)
- [x] create env conda
- [ ] env with conda-lock ?
- [ ] guix approch ?

## TODO: doc
- [x] Intiate mkdocs
- [ ] migate existing docs from wiki to mkdocs
- [ ] python API doc
- [ ] deploy demo / proto sur gitlab page or equivalent on bitbucket

## TODO: test
- [ ] Initiate python test suite
- [ ] migrate existing test (see wiki)
- [ ] doc test ?
- [ ] Initiate unit test