Aquila-Consortium/csiborgtools
0
0
Fork 0
mirror of https://github.com/Richard-Sti/csiborgtools.git synced 2024-12-22 22:58:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
304 commits 2 branches 0 tags 80 MiB
Commit graph

304 commits

This branch
This branch
All branches
Author SHA1 Message Date
Richard Stiskalek
cb5d35b1be
Density field calculation (#58) 
* Add iterative density field generation

* Edit particle halfwidth selection

* Update import

* Remove old file

* Add position wrapping

* Add RSD support

* Add density field calculation

* Edit paths to the density field

* Flip argument order

* Edit documentation

* Edit docs

* update nb
 2023-05-08 14:02:20 +01:00
Richard Stiskalek
4265315b47
Revert "Run density field estimator (#56)" (#57) 
This reverts commit 98d0578fa7.
 2023-05-08 13:59:55 +01:00
Richard Stiskalek
98d0578fa7
Run density field estimator (#56) 
* Add iterative density field generation

* Edit particle halfwidth selection

* Update import

* Remove old file

* Add position wrapping

* Add RSD support

* Add density field calculation

* Edit paths to the density field

* Flip argument order
 2023-05-08 13:58:12 +01:00
rstiskalek
1d871e7109 Clean up README 2023-05-07 23:30:03 +01:00
Richard Stiskalek
65564849f4
Add redshift space position (#55) 
* Add docs

* Delete blank

* Add real to redshift space function

* RSD move back observer

* Convert velocoties to km/s

* Add units to RSP calculation

* Add RSP to catalogues

* Add RSD to angular neighbour search
 2023-05-07 19:07:35 +01:00
rstiskalek
3df771f1ee Blank line 2023-05-07 16:36:42 +01:00
Richard Stiskalek
51b670d30b
Updates to the overlap reader. (#54) 
* Flag for loading clumps_cat

* Optionally load clumps catalogues

* Update single pair reading

* Edit reading many pairs
 2023-05-07 16:34:55 +01:00
Richard Stiskalek
56e39a8b1d
Overlapper improvements (#53) 
* Store indices as f32

* Fix init sorting

* Organise imports

* Rename pathing

* Add particle loading

* Improve particle reading

* Add h5py reader

* edit particle path

* Update particles loading

* update particles loading

* Fix particle dumping

* Add init fitting

* Fix bug due to insufficient precision

* Add commnet

* Add comment

* Add clumps catalogue to halo cat

* Add comment

* Make sure PIDS never forced to float32

* fix pid reading

* fix pid reading

* Update matching to work with new arrays

* Stop using cubical sub boxes, turn off nshift if no smoothing

* Improve caching

* Move function definitions

* Simplify calculation

* Add import

* Small updates to the halo

* Simplify calculation

* Simplify looping calculation

* fix tonew

* Add initial data

* Add skip condition

* Add unit conversion

* Add loading background in batches

* Rename mmain index

* Switch overlaps to h5

* Add finite lagpatch check

* fix column name

* Add verbosity flags

* Save halo IDs instead.

* Switch back to npz

* Delte nbs

* Reduce size of the box

* Load correct bckg of halos being matched

* Remove verbosity

* verbosity edits

* Change lower thresholds
 2023-05-06 16:52:48 +01:00
rstiskalek
1c9dacfde5 Merge branch 'master' of github.com:Richard-Sti/csiborgtools 2023-05-02 14:09:42 +01:00
Richard Stiskalek
1a9e6177d7
Switch to h5py format (#52) 
* Edit the particle paths

* Remove script

* Add h5py to dumping

* Minor adjustments

* add h5py support

* remove split path

* h5py support

* Type

* Edit initmatch paths

* Shorten func

* dist_centmass to work with 2D arrays

* Forgot to return the centre of mass

* Fixed code

* Fix halo bug

* Start MPI broadcasting

* Mini bug

* Remove commenting

* Remove test statement

* Fix index

* Printing from rank 0 only

* Move where clump index stored

* Add dtype options

* Add dtype options
 2023-05-02 13:57:13 +01:00
rstiskalek
7d947f5421 Edit the particle paths 2023-04-30 12:12:37 +01:00
Richard Stiskalek
553eec8228
Update field calculations (#51) 
* Update density field routines

* Add utils

* Add possibility to return 2d array

* Styling

* Fixed density field

* Fix bugs

* add mpc2box

* Fix evaluating sky

* Fix sky map making

* Rename file

* Add paths if only positions

* Add option to dump particles only

* Add comments
 2023-04-29 22:57:05 +01:00
Richard Stiskalek
b5fefe4196
Update init matching procedure (#50) 
* pep8

* fix convention

* Update script

* enforce optimisation boundaries to be finite

* Update TODO

* Remove sky matching

* FIx a small bug

* fix bug

* Remove import

* Add halo fitted quantities

* Update nbs

* update README

* Add load_initial comments

* Rename nbs

* Delete nb

* Update imports

* Rename function

* Update matcher

* Add overlap paths

* Update the matching script

* Update verbosity

* Add verbosity flags

* Simplify make_bckg_delta

* bug fix

* fix bug

* lala

* la

* Add overlap paths

* Update limit

* pep8

* Some pep8 stuff

* pep8

* Minor corrections

* Update paths

* Add number of particles check

* Fix bug

* pep8 comments
 2023-04-29 14:38:21 +01:00
Richard Stiskalek
c14be720b5
Particle dumping (#49) 
* Add paths

* Add dumping script

* Remove snap from density fields

* Add progress statement

* Update import ordering

* Update nb
 2023-04-28 13:57:23 +01:00
rstiskalek
4f8335e0a3 Merge branch 'master' of github.com:Richard-Sti/csiborgtools 2023-04-27 10:40:35 +01:00
rstiskalek
8d5ea287d8 Remove old files 2023-04-27 10:40:18 +01:00
Richard Stiskalek
b7ea2ce28b
Create LICENSE 2023-04-27 10:38:59 +01:00
rstiskalek
b0752efdf4 fix indexing bug 2023-04-27 09:21:00 +01:00
Richard Stiskalek
f48eb6dcb0
Dump radial profile information (#48) 
* add radial position path

* pep8

* Add basic fit profile dumping

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* Update TODO

* Fix parts is None bug

* Update nb
 2023-04-27 00:18:30 +01:00
rstiskalek
1a115f481d Rename file 2023-04-21 00:40:11 +01:00
Richard Stiskalek
04119a5314
Update initial matching & overlaps (#47) 
* pep8

* fix convention

* Update script

* enforce optimisation boundaries to be finite

* Update TODO

* Remove sky matching

* FIx a small bug

* fix bug

* Remove import

* Add halo fitted quantities

* Update nbs

* update README

* Add load_initial comments

* Rename nbs

* Delete nb

* Update imports

* Rename function

* Update matcher

* Add overlap paths

* Update the matching script

* Update verbosity

* Add verbosity flags

* Simplify make_bckg_delta

* bug fix

* fix bug
 2023-04-21 01:35:06 +02:00
Richard Stiskalek
39b3498621
Correct fitting script for both clumps and halos (#46) 
* Minor typos

* fix minor bugs

* pep8

* formatting

* pep8

* Fix minor bugs

* New path & pep8

* add splitt

* Updates

* Improve calculation within radius

* pep8

* pep8

* get the script working

* Add matter overdensity

* Add m200m to the script

* Fix looping bug

* add parents support

* add import

* Optionally concatenate velocities

* Make optional masking

* Ignore the error message

* Start reading in raw data

* Fix cat reading

* Additional units conversions

* Add clump reading

* Fix indexing

* Remove old comment

* Remove old comment

* set npart to 0 instead of overflow from NaN

* fix docs

* rm boring stuff

* Remove old stuff

* Remove old stuff

* Remove old comment

* Update nb
 2023-04-19 16:39:35 +02:00
Richard Stiskalek
c2fde1566b
Cluster rough angular match (#45) 
* Remove sim dump

* Remove unused things

* Remove warning

* Fix coordinate transforms

* Add imports

* Update coordinate transfers

* Add angular neighbours calculation

* Update nb
 2023-04-18 18:49:21 +02:00
Richard Stiskalek
fdb0df8d4c
Add mmain and other major updates (#44) 
* Move paths to a separate file

* Add mmain reader

* Add a verbosity flag

* Fix imports

* Fix bug

* Rename files

* Return ultimate parents

* Add script to generate mmain

* Remove mmain path

* edit path

* Add mmain path

* Change function name

* Rename function

* Turn off verbose

* Fix list requirement

* Edit init match paths

* Fix init pathing

* Edit paths docs

* Edit dumpdir name

* Rename path

* Fix split paths

* Remove unused import

* Add comment

* Update readme

* remove read mmain

* Rename haloatalogue

* Fix minor bugs

* Update nbs

* Add create directory option

* Move split jobs

* Move spliot jobs

* Remove splitting

* Add import

* Edit script

* Deeper split folder

* Fix paths bug

* Rename catalogue

* Rename Catalogue

* Add new clumpread

* Edit paths

* add knn paths

* Update commenting

* Update imports

* Add more conversions

* Update temp file

* Add a note

* Add catalogue

* Cooment

* Update TODO

* Update script

* add nb

* Update

* pep8

* edit paths & pep8

* Fix knn auto paths

* add paths docs

* Add auto and cross knn paths

* Add new paths

* Simplify tpcf reading

* pep8 patch

* update readme

* Update progress

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* Pep 8 and restructure

* add lambda spin

* add clump and halo

* add checks

* Edit halo profile fit

* Update gitignore

* backup script
 2023-04-18 11:02:36 +02:00
Richard Stiskalek
e0d3854277
Add cleaner paths (#43) 
* ignore venv

* ignore pylians if unavaible

* Edit loading paths

* Edit paths

* Stop having default paths

* Add nb

* Add glamdring paths

---------

Co-authored-by: Richard Stiskalek <dc-stis1@login8b.pri.cosma7.alces.network>
Co-authored-by: Richard Stiskalek <dc-stis1@login8a.pri.cosma7.alces.network>
 2023-04-14 23:24:52 +01:00
Richard Stiskalek
1344fa40b6
Add 2PCF calculation (#42) 
* Add 2PCF calculation

* Add 2PCF reader

* Add tpcf scripts

* Add random state

* Fix marked typo

* Fix marks and more randoms

* Stop calculating projected

* Add mean 2PCF

* Remove pimax

* Edit submit script

* Update nb

* Add corrfunc check
 2023-04-14 23:05:48 +01:00
Richard Stiskalek
0b743756ef
File organisation (#41) 
* Split summary files

* Rename script

* Cosmetics

* Minor changes

* Update TODO
 2023-04-09 21:12:19 +01:00
Richard Stiskalek
5784011de0
kNN-CDF secondary halo bias (#40) 
* Add seperate autoknn script & config file

* edit ics

* Edit submission script

* Add threshold values

* Edit batch sizign

* Remove print

* edit

* Rename files

* Rename

* Update nb

* edit runs

* Edit submit

* Add median threshold

* add new auto reader

* editt submit

* edit submit

* Edit submit

* Add mean prk

* Edit runs

* Remove correlation file

* Move split to clutering

* Add init

* Remove import

* Add the file

* Add correlation reading

* Edit scripts

* Add below and above median permutation for cross

* Update imports

* Move rvs_in_sphere

* Create utils

* Split

* Add import

* Add normalised marks

* Add import

* Edit readme

* Clean up submission file

* Stop tracking submit files

* Update gitignore

* Add poisson field analytical expression

* Add abstract generators

* Add generators

* Pass in the generator

* Add a check for if there are any files

* Start saving average density

* Update nb

* Update readme

* Update units

* Edit jobs

* Update submits

* Update reader

* Add random crossing

* Update crossing script

* Add crossing with random

* Update readme

* Update notebook
 2023-04-09 20:57:05 +01:00
Richard Stiskalek
826ab61d2d
Add p(V | k) (#39) 
* add p(V | k)

* Update nb
 2023-04-05 14:47:30 +01:00
Richard Stiskalek
8d34b832af
Improve paths and documentation (#38) 
* pep8

* style issue

* Documentation edits

* Remove old files

* Remove more old files

* Doc & stop setting snap and nsim in paths

* add nsnap, nsim support

* Remove blank space

* docs

* Docs edits

* Reduce redudant code

* docs

* Documentation

* Docs

* Simplify

* add nsnap nsim arguments

* Remove redundant docs

* Fix typos

* Shorten catalogue

* Remove import

* Remove blank line

* Docs only edits

* Rearrange imports

* Remove blank space

* Rearrange imports

* Rearrange imports

* Remove blank space

* Update submission scritp

* Edit docs

* Remove blank line

* new paths

* Update submission scripts

* Update file handling

* Remove blank line

* Move things around

* Ne paths

* Edit submission script

* edit paths

* Fix typo

* Fix bug

* Remove import

* Update nb
 2023-04-04 15:22:00 +01:00
rstiskalek
035d7e0071 add crossfiles reader 2023-04-02 22:23:50 +01:00
Richard Stiskalek
9cbdef4350
kNN-CDF reader (#37) 
* Add nb

* add the KNN reader

* Move reading functions

* Update boolean masking

* Update the nb
 2023-04-02 20:26:15 +01:00
rstiskalek
b1b08b8e53 Edit TODO 2023-04-01 11:43:39 +01:00
Richard Stiskalek
522ee709c9
Joint kNN-CDF calculation (#36) 
* Add joint kNN CDF

* add jointKNN calculation

* change sub script

* Update readme

* update sub

* Small changes

* comments

* update nb

* Update submisison script
 2023-04-01 11:16:11 +01:00
rstiskalek
cb67e326c4 Merge branch 'master' of github.com:Richard-Sti/csiborgtools 2023-04-01 07:57:45 +01:00
Richard Stiskalek
513872ceb6
kNN memory batching (#35) 
* Add batch sizing for less memory

* Add batch size to submission

* Update nb

* Add brute KNN

* unused variable

* Update nb
 2023-04-01 07:57:21 +01:00
rstiskalek
f26cf087ab Update nb 2023-04-01 07:56:52 +01:00
rstiskalek
8fc4e614e3 unused variable 2023-04-01 07:55:17 +01:00
rstiskalek
7610def8a0 Add brute KNN 2023-04-01 07:52:34 +01:00
rstiskalek
097b498da6 Update nb 2023-04-01 07:16:10 +01:00
rstiskalek
8d933e7ba9 Add batch size to submission 2023-04-01 07:15:44 +01:00
rstiskalek
070b9b6c1b Add batch sizing for less memory 2023-04-01 07:11:39 +01:00
Richard Stiskalek
63ab3548b4
kNN-CDF implementation (#34) 
* Rewrite doc

* add kNN

* edit loading of samples with no init

* Add verbosity flag

* add KNN submission script

* do not make peaked cdf by default

* Add submit script

* stop ignore sh

* Add mass thresholding

* Edit gitignore

* edits

* Space points in logspace

* Calculate for all ICs

* Update TODO

* Add dtype support

* Update readme

* Update nb
 2023-03-31 18:13:41 +01:00
rstiskalek
4d7827006a test 2023-03-27 13:38:01 +01:00
Richard Stiskalek
5dd8c668fa
Gaussian smoothing of density fields (#33) 
* Simplify smoothing support and looping over nonzero

* Simplify comments

* add now()

* add cat length

* add smoothed calculation

* add smoothing

* Add sorting

* Edit what is ignored

* Move notebooks

* Add nonsymmetric smoothed overlap

* Update NB

* Add support for reading in the smoothed overlap

* Switch to the true overlap definition

* Reader of the true overlap

* rem occups

* Import moved to a class

* Move definition

* Edit submission script

* Update to account for the new definition

* backup nb

* Switch back to properly initialising arrays

* Fix addition bug

* Update NB

* Fix little bug

* Update nb
 2023-03-27 09:22:03 +01:00
Richard Stiskalek
9b524db617
Add a shared reader (#32) 
* add import

* Rename object

* Simplify how catalogs are handled

* Move functions around

* Add NPair reader

* Add counterpart Gaussian average

* Change what is returned in exp mass

* small bug

* Simplify stat calcu

* Add mptebppl
 2023-03-16 18:02:21 +00:00
Richard Stiskalek
153f1c0002
Overlap reader thresholds (#31) 
* Improve data

* Add comment

* Update how KNN is called

* Bring back indices

* New function output

* return catx["index"]

* Remove unnecessary arguments

* Remove useless arguments

* Rename output

* thin up catalogues

* Add thresholding

* Update README
 2023-03-15 20:55:50 +00:00
Richard Stiskalek
6e290ccfb4
Add support for inverting the reader (#30) 2023-03-09 21:54:13 +00:00
Richard Stiskalek
f2cef73977
Final overlap definition update.. (#29) 
* Delete whitespace

* Update overlap definition

* Add documentation

* add overlap args

* Make the background array store only high density region

* Add nsims to args

* Remove conc as done inside the func

* Simplify the overlap calculation

* Rename variable

* Just some docs

* Docs

* Correct overlap definition

* Undo debug comment

* Remove old debug code

* Add prob of no match and exp couunterpart mass

* docs

* Update the overlap definition
 2023-03-09 14:55:38 +00:00
Richard Stiskalek
1c859dbdac
Overlap reader (#28) 
* Edit output path

* remove matching of many sims

* Add a class skeleton

* Add a setter for only nsim

* Deletei mport

* Simplify cross and delete combined halo catalogue

* Simplify reading in catalogues

* Add distance calculation

* Add mass ratio

* Clean up the submission script

* Add the summer overlap

* Docs

* Add array copying

* Add import

* Add resampling mean
 2023-02-08 12:05:32 +00:00