Commit graph

64 commits

Author SHA1 Message Date
Richard Stiskalek
84a1609d77
Add Lilow2024 velocity fields (#133)
* Add Lilow data

* Fix Lilow NaNs

* Update nb

* Update nb
2024-06-28 11:55:42 +01:00
Richard Stiskalek
301ff61e89
Add CF4 mean & std (#132)
* Add observer velocity

* Add radec to supergalactic

* Update script

* Add CF4 params

* Add paths to CF4

* Add CF4 but not sth wrong

* Update nb

* Remove errors

* Update nb

* Update scripts
2024-06-27 15:02:33 +01:00
rstiskalek
76e8f71556 Update nb 2024-06-26 10:46:33 +01:00
Richard Stiskalek
ce55a2b47e
Add marginalization over boxes (#131)
* Parallelize over simulations

* Update docs

* Update dependency

* Update imports

* Add adtitional dependencies

* Update .gitignore

* Update ERADME

* Simplify numpyro GOF

* Speed up GOF

* Deepcopy samples

* Update scripts

* Add GPU acceleration

* Select boxes

* Update script

* Optionally sample beta

* Fix old code

* Simplify code

* Start saving log posterior

* Start popping log_likeliood

* Add imports

* Add converting samples

* Fix sctipt name

* Add evidence with harmonic

* Remove comment

* Update imports

* Update imports so that pylians not required

* Stop requiring Pylians to be installed

* Update submission scripts for loops

* Update nb

* Update nb

* Add Manticore boxes

* Add verbosity flag

* Add bulk flow

* Update script

* Update nb

* Update normalization

* Update submit

* Update nb
2024-06-26 10:43:26 +01:00
Richard Stiskalek
ffaf92cd4b
Add Manticore (#130)
* Add CB2X paths

* Update imports

* Add CB2 params

* Add CB2x fields

* Add CB2X field

* Add parallelized

* Add **kwargs

* Update nb

* Minor updates
2024-06-21 19:21:06 +01:00
Richard Stiskalek
779f2e76ac
Calculate upglade redshifts (#128)
* Update redshift reading

* Add helio to CMB redshift

* Update imports

* Update nb

* Run for Quijote

* Add script

* Update

* Update .gitignore

* Update imports

* Add Peery estimator

* Add bulk flow scripts

* Update typs

* Add comment

* Add blank space

* Update submission script

* Update description

* Add barriers

* Update nb

* Update nb

* Rename script

* Move to old

* Update imports

* Add nb

* Update script

* Fix catalogue key

* Update script

* Update submit

* Update comment

* Update .gitignore

* Update nb

* Update for stationary obsrevers

* Update submission

* Add nb

* Add better verbose control

* Update nb

* Update submit

* Update nb

* Add SN errors

* Add draft of the script

* Update verbosity flags

* Add submission script

* Debug script

* Quickfix

* Remove comment

* Update nb

* Update submission

* Update nb

* Processed UPGLADE
2024-06-20 14:33:00 +01:00
rstiskalek
c447d2e7b0 Add nbs 2024-05-09 16:29:34 +01:00
rstiskalek
b7d8d3da4d Add nb 2024-05-09 16:29:22 +01:00
rstiskalek
13a10ccedd pep8 2024-04-23 12:03:47 +01:00
Richard Stiskalek
c4557cf35b
Matching of observations (#127)
* Rename file

* Add indents

* Update imports

* Add counting

* Docs

* Add nb

* Rename nb

* Update nb

* Add PV processing

* Update nb

* Add Pantheon+groups

* Update submission scripts

* Add Pantheon+zSN

* Update nb

* Edit param

* Matchin SFI

* Update nb

* Fix path bug

* Add list of clusters

* Update imports

* Update imports

* Add cartesian & mass of clusters

* Add observation to halo matching

* Add nb

* Add inverse CDF

* Add import

* Update nb

* Add comments
2024-04-23 12:02:09 +01:00
Richard Stiskalek
3876985f26
Extend enclosed density to all CB2 (#126)
* Add borg2 flag

* add borg2_all

* Undo a comment?

* ADd borg2

* Update notebook

* Update nb

* Add external halo catalogue

* Add MDPL2

* Add BORG2 density profile plot

* Add more params

* Add comments
2024-04-10 11:34:07 +02:00
rstiskalek
7330e535f7 Update notebook 2024-04-08 23:14:43 +01:00
rstiskalek
096f2055c9 Add MDPL2 HMF 2024-04-08 10:39:41 +01:00
Richard Stiskalek
ee222cd010
Fix overlap runs (#125)
* Update nb

* Update script

* Update script

* Rename

* Update script

* Update script

* Remove warning

* Ignore minors when extracting MAH

* Fix paths bug

* Move notebooks

* Move files

* Rename and delete things

* Rename file

* Move file

* Rename things

* Remove old print statement

* Add basic MAH plot

* Add random MAH path

* Output snapshot numbers

* Add MAH random extraction

* Fix redshift bug

* Edit script

* Add extracting random MAH

* Little updates

* Add CB2 redshift

* Add some caching

* Add diagnostic plots

* Add caching

* Minor updates

* Update nb

* Update notebook

* Update script

* Add Sorce randoms

* Add CB2 varysmall

* Update nb

* Update nb

* Update nb

* Use catalogue HMF

* Move definition of radec2galactic

* Update nb

* Update import

* Update import

* Add galatic coords to catalogues

* Update nb
2024-04-08 11:23:21 +02:00
rstiskalek
c71f5a8513 Upgrade nb 2024-04-03 13:44:55 +01:00
Richard Stiskalek
380c2f8df0
Stacking p(zcosmo | ... ) (#124)
* Update nb

* Add import

* Add posterior stacking
2024-04-02 11:45:18 +02:00
rstiskalek
24cab8555a Update nb 2024-04-02 09:28:57 +01:00
Richard Stiskalek
da43031877
Tests on mock PV catalogues (#123)
* Rename nb

* Add nb

* Update scrtip

* Update script

* Add reading of CB2 mocks

* CB2 mocks support

* Add mock generator

* Add mock support

* Add CB2 mock support

* Update nb for mocks

* Update script

* Update nb

* Update nbs
2024-04-02 00:19:14 +02:00
Richard Stiskalek
8e9645e202
Add p(z_cosmo | z_obs) (#122)
* Add import

* Add draft of the p(zcosmo)

* Add option for additional uncertainty

* Variable renaming

* Add zcosmo predict for flow calibration

* Add flow map notebook

* Update notebook

* Add posterior mean & std

* Edit docstring
2024-04-01 15:44:20 +02:00
rstiskalek
e3a645c935 Update nb 2024-03-28 12:23:52 +00:00
Richard Stiskalek
29b37172fa
Add SFI groups (#121)
* Update LOS sampling

* Bring SD back to speed

* Update nb

* Update script

* Update requirements

* Update flow chi2 calc

* Update script

* Add support for SFI groups

* Update plot

* Update script

* Update script
2024-03-26 15:11:04 +01:00
Richard Stiskalek
27c1f9249b
Observer velocity script (#120)
* Rename script

* Delete scripts

* Add script

* Edit script

* Add script

* Update nb

* Update plotting

* Update .gitignore

* Update nb

* Update nb

* Add option to keep beta fixed
2024-03-26 10:42:53 +01:00
Richard Stiskalek
4093186f9a
Add z_obs output (#119)
* Update SN model to output zobs too

* Add TF predicted zobs

* Add imoprt

* Update nb

* Calculate chi2

* Update nb

* Update script
2024-03-22 18:10:40 +01:00
Richard Stiskalek
f7285b2600
More flow (#118)
* Add GoF calculation

* Add import

* Add base flow

* Add reading of ndata

* Update nb

* Update plotting

* Update script

* Update plots

* Updaet plo

* Add script

* Update nb

* Update nb

* Update script

* Update script

* Update nb

* Remove imports

* Improve labelling

* Improve flow calibration

* Add bulk flow plots

* Update flow

* Update scrit

* Calculate more radial steps

* Update bulk

* Update script

* Update nb
2024-03-21 16:50:37 +01:00
Richard Stiskalek
d0266584fa
CSiBORG2 subhalo catalogue (#116)
* Add import

* Add CSiBORG2 subfind catalogue

* Add parent mass

* Update nb
2024-03-18 16:37:58 +01:00
rstiskalek
0f94457b7f Update nb 2024-03-16 17:16:22 +00:00
Richard Stiskalek
fb4abebeb6
Add more flow modelling (#115)
* Add SN calibration model

* Update imports

* Scale Carrick field

* Minor updates to flow validation

* Update script

* Update flow model

* Add CSiBORG2

* Add CSiboRG2 params

* Update imports

* Add regular grid interpolator to LOS

* Add nb

* Remove old code

* Update scripts

* Minor updates

* Minor updates

* Add TF

* Minor update

* Update notebook

* Update imports

* Add scan and loss of numpyro

* Add Pantheon

* Update svript

* Updaten b

* Add model loader

* Add jackknife

* Add evidence

* Update dr

* Add BIC to the flow model

* Update srcipt

* Update nb

* Update nb

* Update scripts
2024-03-16 17:02:00 +00:00
rstiskalek
b503a6f003 Add new notebook 2024-03-08 14:46:38 +00:00
rstiskalek
95af8db8e7 Delete old notebooks 2024-03-08 14:46:34 +00:00
Richard Stiskalek
a65e3cb15b
Add simple distance flow model (#114)
* Add imports

* Add field LOS paths

* Add basic flow model

* Edit script

* Add nb

* Add nb

* Update nb

* Add some docs

* Add RA reading

* Add imoprts

* Updates to the flow model

* Update script

* Bring back A2

* Update imports

* Update imports

* Add Carrick to ICs

* Add Carrick boxsize

* Add Carrick and fix minor bugs

* Add Carrick box

* Update script

* Edit imports

* Add fixed flow!

* Update omega_m and add it

* Update nb

* Update nb

* Update nb

* Remove old print statements

* Update params

* Add thinning of chains

* Add import

* Add flow validation script

* Add submit script

* Add ksmooth

* Update nb

* Update params

* Update script

* Update string

* Move where distributions are defined

* Add density bias parameter

* Add lognorm mean

* Update scripts

* Update script
2024-03-08 10:44:19 +00:00
Richard Stiskalek
fc51f75e16
Add field interpolation along LOS (#112)
* Add attemtp

* Add import

* Update LOS code

* Add better docstrings

* Update nb
2024-02-26 15:02:48 +00:00
Richard Stiskalek
f61429be43
Add SPH paths (#111)
* Add path to CSiBORG1

* Update nb
2024-02-26 12:56:58 +00:00
Richard Stiskalek
f0ab6fc9b4
Add TNG field (#108)
* Add TNG catalogue script

* Add imoprt

* Add TNG box

* Add import

* Fix little bug

* Add TNG300-1

* Add shell scripts

* Organize .gitignore

* Add scatter to file name

* Add argument scatter

* Add threshold on mass

* Update nb

* Add scatter argument

* Add scattering of radial positions

* Update nb

* Update script
2024-02-07 09:11:49 +00:00
Richard Stiskalek
9e4b34f579
Overlap fixing and more (#107)
* Update README

* Update density field reader

* Update name of SDSSxALFAFA

* Fix quick bug

* Add little fixes

* Update README

* Put back fit_init

* Add paths to initial snapshots

* Add export

* Remove some choices

* Edit README

* Add Jens' comments

* Organize imports

* Rename snapshot

* Add additional print statement

* Add paths to initial snapshots

* Add masses to the initial files

* Add normalization

* Edit README

* Update README

* Fix bug in CSiBORG1 so that does not read fof_00001

* Edit README

* Edit README

* Overwrite comments

* Add paths to init lag

* Fix Quijote path

* Add lagpatch

* Edit submits

* Update README

* Fix numpy int problem

* Update README

* Add a flag to keep the snapshots open when fitting

* Add a flag to keep snapshots open

* Comment out some path issue

* Keep snapshots open

* Access directly snasphot

* Add lagpatch for CSiBORG2

* Add treatment of x-z coordinates flipping

* Add radial velocity field loader

* Update README

* Add lagpatch to Quijote

* Fix typo

* Add setter

* Fix typo

* Update README

* Add output halo cat as ASCII

* Add import

* Add halo plot

* Update README

* Add evaluating field at radial distanfe

* Add field shell evaluation

* Add enclosed mass computation

* Add BORG2 import

* Add BORG boxsize

* Add BORG paths

* Edit run

* Add BORG2 overdensity field

* Add bulk flow clauclation

* Update README

* Add new plots

* Add nbs

* Edit paper

* Update plotting

* Fix overlap paths to contain simname

* Add normalization of positions

* Add default paths to CSiBORG1

* Add overlap path simname

* Fix little things

* Add CSiBORG2 catalogue

* Update README

* Add import

* Add TNG density field constructor

* Add TNG density

* Add draft of calculating BORG ACL

* Fix bug

* Add ACL of enclosed density

* Add nmean acl

* Add galaxy bias calculation

* Add BORG acl notebook

* Add enclosed mass calculation

* Add TNG300-1 dir

* Add TNG300 and BORG1 dir

* Update nb
2024-01-30 16:14:07 +00:00
Richard Stiskalek
aaa14fc880
Add density field plot and start preparing CSiBORG2 (#94)
* Add RAMSES2HDF5 conversion

* Upload changes

* Clean up

* More clean up

* updates

* Little change

* pep9

* Add basic SPH calculation for a snapshot

* Add submit script

* Remove echo

* Little changes

* Send off changes

* Little formatting

* Little updates

* Add nthreads argument

* Upload chagnes

* Add nthreads arguemnts

* Some local changes..

* Update scripts

* Add submission script

* Update script

* Update params

* Rename CSiBORGBox to CSiBORG1box

* Rename CSiBORG1 reader

* Move files

* Rename folder again

* Add basic plotting here

* Add new skeletons

* Move def

* Update nbs

* Edit directories

* Rename files

* Add units to converted snapshots

* Fix empty dataset bug

* Delete file

* Edits to submission scripts

* Edit paths

* Update .gitignore

* Fix attrs

* Update weighting

* Fix RA/dec bug

* Add FORNAX cluster

* Little edit

* Remove boxes since will no longer need

* Move func back

* Edit to include sort by membership

* Edit paths

* Purge basic things

* Start removing

* Bring file back

* Scratch

* Update the rest

* Improve the entire file

* Remove old things

* Remove old

* Delete old things

* Fully updates

* Rename file

* Edit submit script

* Little things

* Add print statement

* Add here cols_to_structured

* Edit halo cat

* Remove old import

* Add comment

* Update paths manager

* Move file

* Remove file

* Add chains
2023-12-13 16:08:25 +00:00
Richard Stiskalek
e972f8e3f2
Add pynbody and other support (#92)
* Simplify box units

* Move old scripts

* Add printing

* Update readers

* Disable boundscheck

* Add new ordering

* Clean up imports

* Enforce dtype and add mass to quijote

* Simplify print statements

* Fix little typos

* Fix key bug

* Bug fixing

* Delete boring comments

* Improve ultimate clumps for PHEW

* Delete boring comments

* Add basic reading

* Remove 0th index HID

* Add flipping of X and Z

* Updates to halo catalogues

* Add ordered caching

* Fix flipping

* Add new flags

* Fix PHEW empty clumps

* Stop over-wrriting

* Little improvements to angular neighbours

* Add catalogue masking

* Change if-else statements

* Cache only filtered data

* Add PHEW cats

* Add comments

* Sort imports

* Get Quijote workign

* Docs

* Add HMF calculation

* Move to old

* Fix angular

* Add great circle distance

* Update imports

* Update impotrts

* Update docs

* Remove unused import

* Fix a quick bug

* Update compatibility

* Rename files

* Renaming

* Improve compatiblity

* Rename snapsht

* Fix snapshot bug

* Update interface

* Finish updating interface

* Update all paths

* Add old scripts

* Add basic halo

* Update imports

* Improve snapshot processing

* Update ordering

* Fix how CM positions accessed

* Add merger paths

* Add imports

* Add merger reading

* Add making a merger tree

* Add a basic merger tree reader

* Add imports

* Add main branch walking + comments + debuggin

* Get tree running

* Add working merger tree walking along main branch

* Add units conversion for merger data

* Add hid_to_array_index

* Update merger tree

* Add mergertree mass to PHEWcat

* Edit comments

* Add this to track changes...

* Fix a little bug

* Add mergertree mass

* Add cache clearing

* Improve summing substructure code

* Littbe bug

* Little updates to the merger tree reader

* Update .giignore

* Add box selection

* Add optional deletingf of a group

* add to keep track of changes

* Update changes

* Remove

* Add manual tracker

* Fix bug

* Add m200c_to_r200c

* Add manual halo tracking

* Remove skipped snapshots

* update cosmo params to match csiborg

* remove old comments

* Add SDSSxALFALFA

* Fix bugs

* Rename

* Edit paths

* Updates

* Add comments

* Add comment

* Add hour conversion

* Add imports

* Add new observation class

* Add selection

* Add imports

* Fix small bug

* Add field copying for safety

* Add matching to survey without masking

* Add P(k) calculation

* Add nb

* Edit comment

* Move files

* Remove merger import

* Edit setup.yp

* Fix typo

* Edit import warnigns

* update nb

* Update README

* Update README

* Update README

* Add skeleton

* Add skeleton
2023-12-07 14:23:32 +00:00
Richard Stiskalek
eccd8e3507
Add galaxy sampling (#88)
* Improve calculations

* Improve flags

* Add smoothed options

* Remove some old comments

* Edit little things

* Save smoothed

* Move files

* Edit imports

* Edit imports

* Renaming imports

* Renaming imports

* Sort imports

* Sort files

* Sorting

* Optionally make copies of the field

* Add quijote backup check

* Add direct field smoothing

* Shorten stupid documentation

* Shorten stupid docs

* Update conversion

* Add particles to ASCII conversion

* Add a short comment

* Add SDSS uncorrected distance

* Adjust comment

* Add FITS index to galaxies

* Remove spare space

* Remove a stupid line

* Remove blank line

* Make space separated

* Add interpolated field path

* Add field sampling

* Sort imports

* Return density in cells

* Clear out observer velocity

* Add 170817 sampling

* Fix normalization

* Update plot
2023-09-01 16:29:50 +01:00
Richard Stiskalek
eb8d070fff
CSiBORG FoF switch (#75)
* Add moving FoF membership files

* add FoF membership path

* Add notes where its PHEW

* Add FoF catalogue path

* Correct typo

* Add more functionalities

* Make work with halo IDs from FoF

* Edit print statement

* Fix copy bug

* copy

* Add FoF catalogue reading

* Clean up script

* Fix typo

* Little edits

* Fix naming convention

* Rename key

* Remove loading substructure particles

* Rename CSiBORG Cat

* Rename clumps cat

* Rename cat

* Remove misplaced import

* Switch to halos

* rm import

* structfit of only halos

* Add FoF halo reading

* Add a short comment

* Fix __getitem__ to work with int

* Fix problems

* Improve __getitem__

* Add more conversion

* Fix indexing

* Fix __getitem__ assertion

* Fix numbers

* Rename

* Fix verbosity flags

* Add full Quijote HMF option

* Add plot of Quijote only

* Add quijote full paths

* Fix the fit_init script

* Renam arg

* Update .gitignore

* add default argument name

* Change default verbosity flag

* Modernise script structure

* Fix dictionary

* Fix reading to include m200c

* Modernise script

* Add args
2023-07-24 14:10:21 +02:00
Richard Stiskalek
ccbbbd24b4
Add better diagnostics & plotting (#67)
* Add caching functions

* Add limts

* Add new mass runs

* Update .gitignore

* Edit which CDFs are loaded

* Stop saving cross hindxs

* Change dist to half precision

* New nearest path

* Add neighbour counting

* Add neighbour kwargs

* Update work in progress

* Add new counting

* Add how dist is built

* Collect dist to 1 file

* Update reading routine

* Delete Quijote files

* Remove file

* Back to float32

* Fix bugs

* Rename utils

* Remove neighbuor kwargs

* Rename file

* Fix bug

* Rename plt utils

* Change where nghb kwargs from

* line length

* Remove old notebooks

* Move survey

* Add white space

* Update TODO

* Update CDF calculation

* Update temporarily plotting

* Merge branch 'add_diagnostics' of github.com:Richard-Sti/csiborgtools into add_diagnostics

* Start adding documentation to plotting

* Remove comments

* Better code documentation

* Some work on tidal tensor

* Better plotting

* Add comment

* Remove nb

* Remove comment

* Add documentation

* Update plotting

* Update submission

* Update KL vs KS plots

* Update the plotting routine

* Update plotting

* Update plotting routines
2023-06-16 14:33:27 +01:00
rstiskalek
616b903aec Remove nb 2023-05-15 23:32:42 +01:00
Richard Stiskalek
255bec9710
Quijote kNN adding (#62)
* Fix small bug

* Add fiducial observers

* Rename 1D knn

* Add new bounds system

* rm whitespace

* Add boudns

* Add simname to paths

* Add fiducial obserevrs

* apply bounds only if not none

* Add TODO

* add simnames

* update script

* Fix distance bug

* update yaml

* Update file reading

* Update gitignore

* Add plots

* add check if empty list

* add func to obtaining cross

* Update nb

* Remove blank lines

* update ignroes

* loop over a few ics

* update gitignore

* add comments
2023-05-15 23:30:10 +01:00
rstiskalek
7971fe2bc1 update nbs 2023-05-14 12:48:24 +01:00
Richard Stiskalek
1d847cbd06
Add Quijote (#61)
* Rename paths object

* Remove redshift calculation

* Explicit keywrod arg

* Rename box units

* Basic renaming

* Little docs

* Rename paths

* add imports

* Sort imports

* Add Quijote cat

* Split boxes

* add Quijote path

* Add origin argument

* Update nbs
2023-05-13 17:37:34 +01:00
Richard Stiskalek
b3fd14b81f
Gravitational potential calculation (#60)
* Rename file

* add velocity field

* Add velcoity field

* Move smoothening

* Add verbosity flags

* remove blank

* Simplify paths

* Add potential calculation

* Update paths

* Add potential field

* Add potential calculation

* Move away sky matching

* Move interpolation functions

* Update nbs
2023-05-12 13:07:58 +01:00
Richard Stiskalek
b710b8e89c
Add new ICs (#59)
* edit IC paths

* Remove import

* Edit path

* Change naming

* Add __main__

* Script to match everything

* Edit docs

* Remove test statement

* Move import

* Update nb
2023-05-09 16:18:01 +01:00
Richard Stiskalek
cb5d35b1be
Density field calculation (#58)
* Add iterative density field generation

* Edit particle halfwidth selection

* Update import

* Remove old file

* Add position wrapping

* Add RSD support

* Add density field calculation

* Edit paths to the density field

* Flip argument order

* Edit documentation

* Edit docs

* update nb
2023-05-08 14:02:20 +01:00
Richard Stiskalek
65564849f4
Add redshift space position (#55)
* Add docs

* Delete blank

* Add real to redshift space function

* RSD move back observer

* Convert velocoties to km/s

* Add units to RSP calculation

* Add RSP to catalogues

* Add RSD to angular neighbour search
2023-05-07 19:07:35 +01:00
Richard Stiskalek
56e39a8b1d
Overlapper improvements (#53)
* Store indices as f32

* Fix init sorting

* Organise imports

* Rename pathing

* Add particle loading

* Improve particle reading

* Add h5py reader

* edit particle path

* Update particles loading

* update particles loading

* Fix particle dumping

* Add init fitting

* Fix bug due to insufficient precision

* Add commnet

* Add comment

* Add clumps catalogue to halo cat

* Add comment

* Make sure PIDS never forced to float32

* fix pid reading

* fix pid reading

* Update matching to work with new arrays

* Stop using cubical sub boxes, turn off nshift if no smoothing

* Improve caching

* Move function definitions

* Simplify calculation

* Add import

* Small updates to the halo

* Simplify calculation

* Simplify looping calculation

* fix tonew

* Add initial data

* Add skip condition

* Add unit conversion

* Add loading background in batches

* Rename mmain index

* Switch overlaps to h5

* Add finite lagpatch check

* fix column name

* Add verbosity flags

* Save halo IDs instead.

* Switch back to npz

* Delte nbs

* Reduce size of the box

* Load correct bckg of halos being matched

* Remove verbosity

* verbosity edits

* Change lower thresholds
2023-05-06 16:52:48 +01:00
Richard Stiskalek
c14be720b5
Particle dumping (#49)
* Add paths

* Add dumping script

* Remove snap from density fields

* Add progress statement

* Update import ordering

* Update nb
2023-04-28 13:57:23 +01:00
Richard Stiskalek
f48eb6dcb0
Dump radial profile information (#48)
* add radial position path

* pep8

* Add basic fit profile dumping

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* Update TODO

* Fix parts is None bug

* Update nb
2023-04-27 00:18:30 +01:00