Commit graph

18 commits

Author SHA1 Message Date
Richard Stiskalek
ccbbbd24b4
Add better diagnostics & plotting (#67)
* Add caching functions

* Add limts

* Add new mass runs

* Update .gitignore

* Edit which CDFs are loaded

* Stop saving cross hindxs

* Change dist to half precision

* New nearest path

* Add neighbour counting

* Add neighbour kwargs

* Update work in progress

* Add new counting

* Add how dist is built

* Collect dist to 1 file

* Update reading routine

* Delete Quijote files

* Remove file

* Back to float32

* Fix bugs

* Rename utils

* Remove neighbuor kwargs

* Rename file

* Fix bug

* Rename plt utils

* Change where nghb kwargs from

* line length

* Remove old notebooks

* Move survey

* Add white space

* Update TODO

* Update CDF calculation

* Update temporarily plotting

* Merge branch 'add_diagnostics' of github.com:Richard-Sti/csiborgtools into add_diagnostics

* Start adding documentation to plotting

* Remove comments

* Better code documentation

* Some work on tidal tensor

* Better plotting

* Add comment

* Remove nb

* Remove comment

* Add documentation

* Update plotting

* Update submission

* Update KL vs KS plots

* Update the plotting routine

* Update plotting

* Update plotting routines
2023-06-16 14:33:27 +01:00
Richard Stiskalek
255bec9710
Quijote kNN adding (#62)
* Fix small bug

* Add fiducial observers

* Rename 1D knn

* Add new bounds system

* rm whitespace

* Add boudns

* Add simname to paths

* Add fiducial obserevrs

* apply bounds only if not none

* Add TODO

* add simnames

* update script

* Fix distance bug

* update yaml

* Update file reading

* Update gitignore

* Add plots

* add check if empty list

* add func to obtaining cross

* Update nb

* Remove blank lines

* update ignroes

* loop over a few ics

* update gitignore

* add comments
2023-05-15 23:30:10 +01:00
Richard Stiskalek
f48eb6dcb0
Dump radial profile information (#48)
* add radial position path

* pep8

* Add basic fit profile dumping

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* Update TODO

* Fix parts is None bug

* Update nb
2023-04-27 00:18:30 +01:00
Richard Stiskalek
fdb0df8d4c
Add mmain and other major updates (#44)
* Move paths to a separate file

* Add mmain reader

* Add a verbosity flag

* Fix imports

* Fix bug

* Rename files

* Return ultimate parents

* Add script to generate mmain

* Remove mmain path

* edit path

* Add mmain path

* Change function name

* Rename function

* Turn off verbose

* Fix list requirement

* Edit init match paths

* Fix init pathing

* Edit paths docs

* Edit dumpdir name

* Rename path

* Fix split paths

* Remove unused import

* Add comment

* Update readme

* remove read mmain

* Rename haloatalogue

* Fix minor bugs

* Update nbs

* Add create directory option

* Move split jobs

* Move spliot jobs

* Remove splitting

* Add import

* Edit script

* Deeper split folder

* Fix paths bug

* Rename catalogue

* Rename Catalogue

* Add new clumpread

* Edit paths

* add knn paths

* Update commenting

* Update imports

* Add more conversions

* Update temp file

* Add a note

* Add catalogue

* Cooment

* Update TODO

* Update script

* add nb

* Update

* pep8

* edit paths & pep8

* Fix knn auto paths

* add paths docs

* Add auto and cross knn paths

* Add new paths

* Simplify tpcf reading

* pep8 patch

* update readme

* Update progress

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* Pep 8 and restructure

* add lambda spin

* add clump and halo

* add checks

* Edit halo profile fit

* Update gitignore

* backup script
2023-04-18 11:02:36 +02:00
Richard Stiskalek
e0d3854277
Add cleaner paths (#43)
* ignore venv

* ignore pylians if unavaible

* Edit loading paths

* Edit paths

* Stop having default paths

* Add nb

* Add glamdring paths

---------

Co-authored-by: Richard Stiskalek <dc-stis1@login8b.pri.cosma7.alces.network>
Co-authored-by: Richard Stiskalek <dc-stis1@login8a.pri.cosma7.alces.network>
2023-04-14 23:24:52 +01:00
Richard Stiskalek
5784011de0
kNN-CDF secondary halo bias (#40)
* Add seperate autoknn script & config file

* edit ics

* Edit submission script

* Add threshold values

* Edit batch sizign

* Remove print

* edit

* Rename files

* Rename

* Update nb

* edit runs

* Edit submit

* Add median threshold

* add new auto reader

* editt submit

* edit submit

* Edit submit

* Add mean prk

* Edit runs

* Remove correlation file

* Move split to clutering

* Add init

* Remove import

* Add the file

* Add correlation reading

* Edit scripts

* Add below and above median permutation for cross

* Update imports

* Move rvs_in_sphere

* Create utils

* Split

* Add import

* Add normalised marks

* Add import

* Edit readme

* Clean up submission file

* Stop tracking submit files

* Update gitignore

* Add poisson field analytical expression

* Add abstract generators

* Add generators

* Pass in the generator

* Add a check for if there are any files

* Start saving average density

* Update nb

* Update readme

* Update units

* Edit jobs

* Update submits

* Update reader

* Add random crossing

* Update crossing script

* Add crossing with random

* Update readme

* Update notebook
2023-04-09 20:57:05 +01:00
Richard Stiskalek
63ab3548b4
kNN-CDF implementation (#34)
* Rewrite doc

* add kNN

* edit loading of samples with no init

* Add verbosity flag

* add KNN submission script

* do not make peaked cdf by default

* Add submit script

* stop ignore sh

* Add mass thresholding

* Edit gitignore

* edits

* Space points in logspace

* Calculate for all ICs

* Update TODO

* Add dtype support

* Update readme

* Update nb
2023-03-31 18:13:41 +01:00
Richard Stiskalek
5dd8c668fa
Gaussian smoothing of density fields (#33)
* Simplify smoothing support and looping over nonzero

* Simplify comments

* add now()

* add cat length

* add smoothed calculation

* add smoothing

* Add sorting

* Edit what is ignored

* Move notebooks

* Add nonsymmetric smoothed overlap

* Update NB

* Add support for reading in the smoothed overlap

* Switch to the true overlap definition

* Reader of the true overlap

* rem occups

* Import moved to a class

* Move definition

* Edit submission script

* Update to account for the new definition

* backup nb

* Switch back to properly initialising arrays

* Fix addition bug

* Update NB

* Fix little bug

* Update nb
2023-03-27 09:22:03 +01:00
Richard Stiskalek
c3c686fa60
Power spectrum stuff (#16)
* add autok1d

* update TODO

* add basic cross pk

* del blank line

* fix small bugs

* fix loop bug

* rm nbs

* edit docstring

* Update TODO

* get rid of verbose flag

* take all ICs

* attempt better memory

* simplify ASCII dump

* rm verbose statement to get rid of bugs!

* close fortran files

* add MAS options

* add boxsize setter

* Move comment up

* paths chagne

* Update TODO

* Update TODO

* Forgotten units in Xpk

* Attempt at fixing the distance bug

* Update TODO

* Update TODO

* Remove comment

* add summary reader

* Update gitignore
2022-12-12 14:42:44 +00:00
Richard Stiskalek
1bb4255874
SDSS galaxies and catalogs rework (#15)
* add SDSS skeleton

* rm nb

* update gitignore

* add docs

* add survey volume

* add masking

* add SDSS mask

* add SDSS comments

* add reference

* rm

* move planck to fits clusters

* Rm comment

* update masking routines

* rm masks

* Update docs

* rm data

* make MCXC fits surv

* change MCXC name

* Move to extrasapce

* Update paths
2022-12-05 11:36:41 +00:00
Richard Stiskalek
6aa3721c83
Potential (#13)
* add basic density field

* Add TODO

* add field smoothing

* update how pos are calculated

* add transforms both ways

* add import

* add sky density

* add make skymap func

* update TODO

* update gitignore

* add potential field calculation

* delete boxsize setter

* add tidal tensor
2022-11-27 07:53:38 +00:00
Richard Stiskalek
161c27d995
Particle match & file system & phase space (#11)
* Create file system

* add doc

* add n_sim n_snap directly to paths

* Move things to a single particle reader for consistency

* add docstring

* add srdcir, dumpdir and mmain_path

* make boxunits work with paths

* switch to using paths

* add tempdumpdir

* rm dependence on old functions

* rm comment

* rm unused import

* go back to all imports

* fix import bug

* rm dependence on old functions

* modernize code!

* fix typo

* fix typo

* update fits to new data structureing

* change docs

* add julia repo

* add setup

* add install commands

* ignore install files

* add array flattening

* update dependene

* add positions reader

* update manifest and projects

* add func

* update gitignore

* pos matching progress

* move file

* rm comment

* add velocities getter

* fix bug

* fix name bug

* fix path bug

* fix args func

* add redshift calculation to catalogues

* add shortcut to set n_sim and n_snap

* if cond bug

* add the cosine similarity

* add verbosit iterator

* add docs

* update README

* update README

* update README
2022-11-25 10:44:08 +00:00
Richard Stiskalek
8a56c22813
Within halo work and NFW fit (#4)
* add listing of snapshots

* change distance to comoving

* ignore cp files

* rename nb

* add str to list

* add NFW profile shapes

* add fits imports

* Rename to Nsnap

* in clumps_read only select props

* make clumpid int

* expand doc

* add import

* edit readme

* distribute halos

* add profile & posterior

* add import

* add import

* add documentation

* add rvs and init guess

* update todo

* update nb

* add file

* return end index too

* change clump_ids format to int32

* skeleton of dump particle

* update nb

* add func to drop 0 clump indxs parts

* add import

* add halo dump

* switch to float32

* Update TODO

* update TODO

* add func that loads a split

* add halo object

* Rename to clump

* make post work with a clump

* add optimiser

* add Nsplits

* ignore submission scripts

* ignore .out

* add dumppath

* add job splitting

* add split halos script

* rename file

* renaem files

* rm file

* rename imports

* edit desc

* add pick clump

* add number of particles

* update TODO

* update todo

* add script

* add dumping

* change dumpdir structure

* change dumpdir

* add import

* Remove tqdm

* Increase the number of splits

* rm shuffle option

* Change to remove split

* add emojis

* fix part counts in splits

* change num of splits

* rm with particle cut

* keep splits

* fit only if 10 part and more

* add min distance

* rm warning about not set vels

* update TODO

* calculate rho0 too

* add results collection

* add import

* add func to combine splits

* update TODO

* add extract cols

* update nb

* update TODO
2022-10-30 20:16:56 +00:00
rstiskalek
b7b4353487 ignore settings 2022-10-20 08:13:09 +01:00
Richard Stiskalek
53a0629d90
Basic matching (#2)
* add recarray manipulations

* add cart to radec

* add behav so x can be a list

* add import

* create empty files

* ignore plots file

* add planck data

* add read_mmain file

* add cols_to_structured import

* use cols_to_structured

* add cols_to_structued

* add read_mmain import

* add reading planck

* add mass conversion

* add brute force separation calculation

* update nb

* rename & int dtype

* add func to get csiborg ids

* add list to nd array conversion

* add utils

* rename file

* add 2M++

* add read 2mpp

* add 2mpp shortcut

* add randoms generator

* Change range of RA [0, 360]

* fix ang wrapping

* add code for sphere 2pcf

* rm wrapping

* optionally load only a few borgs

* update nb
2022-10-18 19:41:20 +01:00
rstiskalek
edb9ee430c ignore checkpoints 2022-10-11 13:47:26 +01:00
rstiskalek
8290e56ceb ignore *.pyc 2022-10-10 17:41:29 +01:00
rstiskalek
54221a8ff5 first commit 2022-10-02 11:12:45 +01:00