Initial commit: v0.1.0

This commit is contained in:
hoellinger 2025-01-10 17:03:16 +01:00
commit c041384662
172 changed files with 45744 additions and 0 deletions

View file

@ -0,0 +1,58 @@
# Contributing to SelfiSys
We welcome contributions to the SelfiSys repository! Please follow these guidelines when contributing.
---
## Reporting Issues
If you find a bug or have a suggestion, please open an issue in the [GitHub repository](https://github.com/hoellin/selfisys_public). Include as much detail as possible:
- Steps to reproduce the issue
- Expected vs. actual behaviour
- Relevant error messages or logs (if applicable)
- Suggestions for improvement (if applicable)
If you are unsure whether your issue is a bug or have questions about the code, you are welcome to open an issue for discussion.
## Submitting Contributions
1. Fork the repository and create a new branch for your changes.
2. Ensure your contributions are well-documented and adhere to the highest coding standards.
3. Test your changes thoroughly before submitting:
- Ensure Jupyter notebooks run without errors from top to bottom.
- Validate new functionality or fixes using appropriate test cases.
4. Before submitting a pull request, synchronise your fork with the main repository to incorporate upstream changes.
- Add the main repository as a remote.
```bash
git remote add upstream https://github.com/hoellin/selfisys_public.git
```
- Fetch the latest changes from the upstream repository.
```bash
git fetch upstream
```
- Merge the changes into your local repository.
```bash
git merge upstream/main
```
5. Open a pull request describing your changes to the main repository.
## Style Guidelines
Follow best practices for Python coding and Jupyter notebooks.
### Python code
Refer to the [PEP 8 Style Guide](https://pep8.org/) for Python coding standards.
### Jupyter Notebooks
- Use clear and concise Markdown cells to explain code and results.
- Avoid leaving unnecessary output (e.g., debugging print statements).
- Ensure notebooks are runnable from top to bottom.
- For tips on creating clean and effective Jupyter notebooks, see [Jupyter Notebook Best Practices](https://realpython.com/jupyter-notebook-best-practices/).
---
## Questions?
If you have any questions or need further clarification, feel free to [contact the authors](mailto:tristan.hoellinger@iap.fr).

83
docs/source/README.md Normal file
View file

@ -0,0 +1,83 @@
# SelfiSys: Assess the Impact of Systematic Effects in Galaxy Surveys
[![arXiv](https://img.shields.io/badge/astro--ph.CO-arxiv%3A2412.04443-B31B1B.svg?style=flat)](https://arxiv.org/abs/2412.04443)
[![GPLv3 license](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://github.com/hoellin/selfisys_public/blob/main/LICENSE)
[![GitHub version](https://img.shields.io/github/tag/hoellin/selfisys_public.svg?label=version)](https://github.com/hoellin/selfisys_public)
[![GitHub last commit](https://img.shields.io/github/last-commit/hoellin/selfisys_public.svg)](https://github.com/hoellin/selfisys_public/commits/main)
**SelfiSys** is a Python package designed to address the issue of model misspecification in field-based, implicit likelihood cosmological inference.
It leverages the inferred initial matter power spectrum, enabling a thorough diagnosis of systematic effects in large-scale spectroscopic galaxy surveys.
## Key Features
- **Custom hidden-box forward models**
We provide a `HiddenBox` class to simulate realistic spectroscopic galaxy surveys. It accommodates fully non-linear gravitational evolution, and incorporates multiple systematic effects observed in real-world survey, e.g., misspecified galaxy bias, survey mask, selection functions, dust extinction, line interlopers, or inaccurate gravity solver.
- **Diagnosis of systematic effects**
Diagnose the impact of systematic effects using the inferred initial matter power spectrum, prior to performing cosmological inference.
- **Cosmological inference**
Perform inference of cosmological parameters using Approximate Bayesian Computation (ABC) with a Population Monte Carlo (PMC) sampler.
---
## Documentation
The documentation, including a detailed API reference, is available at [hoellin.github.io/selfisys_public](https://hoellin.github.io/selfisys_public/).
For practical examples demonstrating how to use SelfiSys, visit the [SelfiSys Examples Repository](https://github.com/hoellin/selfisys_examples).
## Contributors
- **Tristan Hoellinger**, [tristan.hoellinger@iap.fr](mailto:tristan.hoellinger@iap.fr)
Principal developer and maintainer, Institut dAstrophysique de Paris (IAP).
For information on contributing, refer to [CONTRIBUTING.md](CONTRIBUTING.md).
## References
If you use the SelfiSys package in your research, please cite the following paper and feel free to [contact the authors](mailto:tristan.hoellinger@iap.fr) for feedback, collaboration opportunities, or other inquiries.
**Diagnosing Systematic Effects Using the Inferred Initial Power Spectrum**
*Hoellinger, T. and Leclercq, F., 2024*
[arXiv:2412.04443](https://arxiv.org/abs/2412.04443) [[astro-ph.CO]](https://arxiv.org/abs/2412.04443) [[ADS]](https://ui.adsabs.harvard.edu/abs/arXiv:2412.04443) [[pdf]](https://arxiv.org/pdf/2412.04443)
BibTeX entry for citation:
```bibtex
@ARTICLE{hoellinger2024diagnosing,
author = {Hoellinger, Tristan and Leclercq, Florent},
title = "{Diagnosing Systematic Effects Using the Inferred Initial Power Spectrum}",
journal = {arXiv e-prints},
keywords = {Astrophysics - Cosmology and Nongalactic Astrophysics, Astrophysics - Instrumentation and Methods for Astrophysics},
year = 2024,
month = dec,
eid = {arXiv:2412.04443},
pages = {arXiv:2412.04443},
doi = {10.48550/arXiv.2412.04443},
archivePrefix = {arXiv},
eprint = {2412.04443},
primaryClass = {astro-ph.CO},
adsurl = {https://ui.adsabs.harvard.edu/abs/2024arXiv241204443H},
adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}
```
## Requirements
The code is written in Python 3.10 and depends on the following packages:
- [`pySELFI`](https://pyselfi.readthedocs.io/en/latest/): Python implementation of the Simulator Expansion for Likelihood-Free Inference.
- [`Simbelmynë`](https://simbelmyne.readthedocs.io/en/latest/): A hierarchical probabilistic simulator for generating synthetic galaxy survey data.
- [`ELFI`](https://elfi.readthedocs.io/en/latest/): A statistical software package for likelihood-free inference, implementing in particular Approximate Bayesian Computation (ABC) with a Population Monte Carlo (PMC) sampler.
A comprehensive list of dependencies, including version specifications to ensure reproducibility, will be provided in a yaml file, along with installation instructions, in a future release.
---
## License
This software is distributed under the GPLv3 Licence. Please review the [LICENSE](https://github.com/hoellin/selfisys_public/blob/main/LICENSE) file in the repository to understand the terms of use and ensure compliance. By downloading and using this software, you agree to the terms of the licence.

44
docs/source/REFERENCES.md Normal file
View file

@ -0,0 +1,44 @@
# References
If you use the SelfiSys package in your research, please cite the following paper and feel free to [contact the authors](mailto:tristan.hoellinger@iap.fr) for feedback, collaboration opportunities, or other inquiries.
**Diagnosing Systematic Effects Using the Inferred Initial Power Spectrum**
*Hoellinger, T. and Leclercq, F., 2024*
[arXiv:2412.04443](https://arxiv.org/abs/2412.04443) [[astro-ph.CO]](https://arxiv.org/abs/2412.04443) [[ADS]](https://ui.adsabs.harvard.edu/abs/arXiv:2412.04443) [[pdf]](https://arxiv.org/pdf/2412.04443)
BibTeX entry for citation:
```bibtex
@ARTICLE{hoellinger2024diagnosing,
author = {Hoellinger, Tristan and Leclercq, Florent},
title = "{Diagnosing Systematic Effects Using the Inferred Initial Power Spectrum}",
journal = {arXiv e-prints},
keywords = {Astrophysics - Cosmology and Nongalactic Astrophysics, Astrophysics - Instrumentation and Methods for Astrophysics},
year = 2024,
month = dec,
eid = {arXiv:2412.04443},
pages = {arXiv:2412.04443},
doi = {10.48550/arXiv.2412.04443},
archivePrefix = {arXiv},
eprint = {2412.04443},
primaryClass = {astro-ph.CO},
adsurl = {https://ui.adsabs.harvard.edu/abs/2024arXiv241204443H},
adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}
```
Other references relevant to the SelfiSys pipeline and cited in the code are listed below.
- **[jasche2010bayesian]**
Jasche, J., Kitaura, F. S., Wandelt, B. D., and Enßlin, T. A., “Bayesian power-spectrum inference for large-scale structure data”, <i>Monthly Notices of the Royal Astronomical Society</i>, vol. 406, no. 1, OUP, pp. 6085, 2010. [doi:10.1111/j.1365-2966.2010.16610.x](https://doi.org/10.48550/arXiv.0911.2493).
- **[gil2015power]**
Gil-Marín, H., “The power spectrum and bispectrum of SDSS DR11 BOSS galaxies - I. Bias and gravity”, <i>Monthly Notices of the Royal Astronomical Society</i>, vol. 451, no. 1, OUP, pp. 539580, 2015. [doi:10.1093/mnras/stv961](https://doi.org/10.48550/arXiv.1407.5668).
- **[howlett2015clustering]**
Howlett, C., Ross, A. J., Samushia, L., Percival, W. J., and Manera, M., “The clustering of the SDSS main galaxy sample - II. Mock galaxy catalogues and a measurement of the growth of structure from redshift space distortions at z = 0.15”, <i>Monthly Notices of the Royal Astronomical Society</i>, vol. 449, no. 1, OUP, pp. 848866, 2015. [doi:10.1093/mnras/stu2693](https://doi.org/10.48550/arXiv.1409.3238).
- **[leclercq2019primordial]**
Florent Leclercq, Wolfgang Enzi, Jens Jasche, Alan Heavens, Primordial power spectrum and cosmology from black-box galaxy surveys, Monthly Notices of the Royal Astronomical Society, Volume 490, Issue 3, December 2019, Pages 42374253, [doi:10.1093/mnras/stz2718](https://doi.org/10.1093/mnras/stz2718).
- **[hoellinger2024diagnosing]**
Hoellinger, T. and Leclercq, F., “Diagnosing Systematic Effects Using the Inferred Initial Power Spectrum”, <i>arXiv e-prints</i>, Art. no. arXiv:2412.04443, 2024. [doi.org:10.48550/arXiv.2412.04443](https://doi.org/10.48550/arXiv.2412.04443).

114
docs/source/conf.py Normal file
View file

@ -0,0 +1,114 @@
# Configuration file for the Sphinx documentation builder.
#
# For the full list of built-in configuration values, see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Project information -----------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
project = "SelfiSys"
copyright = "2025, Tristan Hoellinger"
author = "Tristan Hoellinger"
release = (
"/Users/hoellinger/Library/CloudStorage/Dropbox/travail/these/science/code/SELFI/selfisys/src/"
)
# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
extensions = [
"sphinx.ext.autodoc", # Automatically generate documentation from docstrings.
"sphinx.ext.viewcode", # Adds links to highlighted source code.
"sphinx.ext.napoleon", # Supports Google and NumPy-style docstrings.
"myst_parser", # Markdown support.
"sphinx.ext.autosummary", # Generate summary tables for API docs.
"sphinx.ext.mathjax", # For rendering mathematical equations using MathJax.
"sphinx.ext.todo", # Support for TODO notes.
]
templates_path = ["_templates"]
exclude_patterns = []
# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
html_extra_path = ["_static"]
html_theme_options = {
"collapse_navigation": False,
"sticky_navigation": True,
"navigation_depth": 4,
"style_external_links": True,
"titles_only": False,
}
# Add GitHub link
html_context = {
"display_github": True, # Enables the link
"github_user": "hoellin", # GitHub username
"github_repo": "selfisys_public", # Repository name
"github_version": "main", # Branch name
"conf_py_path": "/docs/source/", # Path to the documentation directory in the repo
}
import os
import sys
import shutil
# Add the project's source path to sys.path
sys.path.insert(0, os.path.abspath("../../src"))
# Copy Markdown files from parent directory
markdown_files = ["README.md", "CONTRIBUTING.md", "REFERENCES.md"]
for md_file in markdown_files:
src = os.path.abspath(f"../../{md_file}") # Adjust the path if necessary
dst = os.path.join(os.path.dirname(__file__), os.path.basename(md_file))
try:
if os.path.exists(src) and not os.path.exists(dst):
shutil.copy(src, dst)
print(f"Copied {src} to {dst}")
except Exception as e:
print(f"Error copying {md_file}: {e}")
def generate_rst_files():
"""
Automatically generate .rst files for standalone Python modules.
Avoids regeneration if files already exist.
"""
source_dir = os.path.abspath("src/selfisys") # Adjust to your source directory
output_dir = os.path.join(os.path.dirname(__file__)) # docs/source directory
# Ensure source_dir exists
if not os.path.isdir(source_dir):
raise FileNotFoundError(f"The source directory '{source_dir}' does not exist.")
# List all Python files in the source directory (excluding __init__.py)
for filename in os.listdir(source_dir):
if filename.endswith(".py") and filename != "__init__.py":
module_name = filename[:-3] # Remove .py extension
output_file = os.path.join(output_dir, f"selfisys.{module_name}.rst")
# Skip regeneration if the file already exists
if os.path.exists(output_file):
continue
# Write the .rst file content
with open(output_file, "w") as f:
# Module header
f.write(f"{module_name}\n")
f.write("=" * len(module_name) + "\n\n")
f.write(f".. automodule:: selfisys.{module_name}\n")
f.write(" :members:\n")
f.write(" :undoc-members:\n")
f.write(" :show-inheritance:\n")
print(f"Generated {output_file}")
# Register a Sphinx hook to generate .rst files before the build
def setup(app):
app.connect("builder-inited", lambda app: generate_rst_files())

95
docs/source/index.rst Normal file
View file

@ -0,0 +1,95 @@
SelfiSys: Assess the Impact of Systematic Effects in Galaxy Surveys
===================================================================
.. image:: https://img.shields.io/badge/astro--ph.CO-arxiv%3A2412.04443-B31B1B.svg
:target: https://arxiv.org/abs/2412.04443
:alt: arXiv
.. image:: https://img.shields.io/github/v/tag/hoellin/selfisys_public.svg?label=version
:target: https://github.com/hoellin/selfisys_public/releases
:alt: GitHub Release
.. image:: https://img.shields.io/github/last-commit/hoellin/selfisys_public
:target: https://github.com/hoellin/selfisys_public/commits/main
:alt: Last Commit
.. image:: https://img.shields.io/badge/License-GPLv3-blue.svg
:target: https://github.com/hoellin/selfisys_public/blob/main/LICENSE
:alt: License
**SelfiSys** is a Python package designed to address the issue of model misspecification in field-based, implicit likelihood cosmological inference.
It leverages the inferred initial matter power spectrum, enabling a thorough diagnosis of systematic effects in large-scale spectroscopic galaxy surveys.
Key Features
------------
- **Custom hidden-box forward models**
We provide a `HiddenBox` class to simulate realistic spectroscopic galaxy surveys. It accommodates fully non-linear gravitational evolution, and incorporates multiple systematic effects observed in real-world survey, e.g., misspecified galaxy bias, survey mask, selection functions, dust extinction, line interlopers, or inaccurate gravity solver.
- **Diagnosis of systematic effects**
Diagnose the impact of systematic effects using the inferred initial matter power spectrum, prior to performing cosmological inference.
- **Cosmological inference**
Perform inference of cosmological parameters using Approximate Bayesian Computation (ABC) with a Population Monte Carlo (PMC) sampler.
For practical examples demonstrating how to use SelfiSys, visit the `SelfiSys Examples Repository <https://github.com/hoellin/selfisys_examples>`_.
References
----------
If you use the SelfiSys package in your research, please cite the following paper and feel free to `contact the authors <mailto:tristan.hoellinger@iap.fr>`_ for feedback, collaboration opportunities, or other inquiries.
**Diagnosing Systematic Effects Using the Inferred Initial Power Spectrum**
*Hoellinger, T. and Leclercq, F., arXiv e-prints*, 2024
`arXiv:2412.04443 <https://arxiv.org/abs/2412.04443>`_
`[astro-ph.CO] <https://arxiv.org/abs/2412.04443>`_
`[ADS] <https://ui.adsabs.harvard.edu/abs/arXiv:2412.04443>`_
`[pdf] <https://arxiv.org/pdf/2412.04443>`_
Contributors
------------
- **Tristan Hoellinger**
`tristan.hoellinger@iap.fr <mailto:tristan.hoellinger@iap.fr>`_
Principal developer and maintainer, Institut dAstrophysique de Paris (IAP).
License
-------
This software is distributed under the GPLv3 Licence. Please review the `LICENSE <https://github.com/hoellin/selfisys_public/blob/main/LICENSE>`_ file in the repository to understand the terms of use and ensure compliance. By downloading and using this software, you agree to the terms of the licence.
Requirements
------------
The code is written in Python 3.10 and depends on the following packages:
- `pySELFI <https://pyselfi.readthedocs.io/en/latest/>`_: Python implementation of the Simulator Expansion for Likelihood-Free Inference.
- `Simbelmynë <https://simbelmyne.readthedocs.io/en/latest/>`_: A hierarchical probabilistic simulator for generating synthetic galaxy survey data.
- `ELFI <https://elfi.readthedocs.io/en/latest/>`_: A statistical software package for likelihood-free inference, implementing Approximate Bayesian Computation (ABC) with a Population Monte Carlo (PMC) sampler.
A comprehensive list of dependencies, along with installation instructions, will be provided in a future release.
.. toctree::
:maxdepth: 2
:caption: API Documentation
selfisys.hiddenbox
selfisys.normalise_hb
selfisys.prior
selfisys.selection_functions
selfisys.selfi_interface
selfisys.sbmy_interface
selfisys.grf
selfisys.utils
.. toctree::
:maxdepth: 2
:caption: Contribute
../../CONTRIBUTING.md
.. toctree::
:maxdepth: 2
:caption: References
../../REFERENCES.md

View file

@ -0,0 +1,7 @@
global_parameters
=================
.. automodule:: selfisys.global_parameters
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
grf
===
.. automodule:: selfisys.grf
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
hiddenbox
=========
.. automodule:: selfisys.hiddenbox
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
normalise_hb
============
.. automodule:: selfisys.normalise_hb
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
prior
=====
.. automodule:: selfisys.prior
:members:
:undoc-members:
:show-inheritance:

93
docs/source/selfisys.rst Normal file
View file

@ -0,0 +1,93 @@
selfisys package
================
Subpackages
-----------
.. toctree::
:maxdepth: 4
selfisys.utils
Submodules
----------
selfisys.global\_parameters module
----------------------------------
.. automodule:: selfisys.global_parameters
:members:
:undoc-members:
:show-inheritance:
selfisys.grf module
-------------------
.. automodule:: selfisys.grf
:members:
:undoc-members:
:show-inheritance:
selfisys.hiddenbox module
-------------------------
.. automodule:: selfisys.hiddenbox
:members:
:undoc-members:
:show-inheritance:
selfisys.normalise\_hb module
-----------------------------
.. automodule:: selfisys.normalise_hb
:members:
:undoc-members:
:show-inheritance:
selfisys.prior module
---------------------
.. automodule:: selfisys.prior
:members:
:undoc-members:
:show-inheritance:
selfisys.sbmy\_interface module
-------------------------------
.. automodule:: selfisys.sbmy_interface
:members:
:undoc-members:
:show-inheritance:
selfisys.selection\_functions module
------------------------------------
.. automodule:: selfisys.selection_functions
:members:
:undoc-members:
:show-inheritance:
selfisys.selfi\_interface module
--------------------------------
.. automodule:: selfisys.selfi_interface
:members:
:undoc-members:
:show-inheritance:
selfisys.setup\_model module
----------------------------
.. automodule:: selfisys.setup_model
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: selfisys
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
sbmy_interface
==============
.. automodule:: selfisys.sbmy_interface
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
selection_functions
===================
.. automodule:: selfisys.selection_functions
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
selfi_interface
===============
.. automodule:: selfisys.selfi_interface
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,7 @@
setup_model
===========
.. automodule:: selfisys.setup_model
:members:
:undoc-members:
:show-inheritance:

View file

@ -0,0 +1,101 @@
selfisys.utils package
======================
Submodules
----------
selfisys.utils.examples\_utils module
-------------------------------------
.. automodule:: selfisys.utils.examples_utils
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.logger module
----------------------------
.. automodule:: selfisys.utils.logger
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.low\_level module
--------------------------------
.. automodule:: selfisys.utils.low_level
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.parser module
----------------------------
.. automodule:: selfisys.utils.parser
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.path\_utils module
---------------------------------
.. automodule:: selfisys.utils.path_utils
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.plot\_examples module
------------------------------------
.. automodule:: selfisys.utils.plot_examples
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.plot\_params module
----------------------------------
.. automodule:: selfisys.utils.plot_params
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.plot\_utils module
---------------------------------
.. automodule:: selfisys.utils.plot_utils
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.timestepping module
----------------------------------
.. automodule:: selfisys.utils.timestepping
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.tools module
---------------------------
.. automodule:: selfisys.utils.tools
:members:
:undoc-members:
:show-inheritance:
selfisys.utils.workers module
-----------------------------
.. automodule:: selfisys.utils.workers
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: selfisys.utils
:members:
:undoc-members:
:show-inheritance: