Commit graph

169 commits

Author SHA1 Message Date
rstiskalek
6ba0c04ab8 Add mass to name 2023-08-08 11:46:46 +01:00
Richard Stiskalek
c7b600d0ad
Periodic neighbours (#84)
* Edit the HMF plot

* Add periodic dist 2 points

* Add boxsize to RVSSphere

* Add periodic distance

* Adding periodic distance

* Add imports

* Change arguments

* Update bounds

* Lower min number of particles

* Change kwargs

* Add paths overlap quijote

* Add some comments
2023-08-08 12:19:40 +02:00
rstiskalek
c7e447df01 Add blank line 2023-08-07 10:38:37 +01:00
Richard Stiskalek
f4a7cb0d16
Remove halo fitting. (#83)
* Rename file

* Remove old content

* Remove halo fit

* Completely remove fits

* Add utils here

* Account for renaming
2023-08-07 11:33:27 +02:00
Richard Stiskalek
344ff8e091
Fixing overlaps and halo definitions. (#80)
* Add imports

* Refactor code

* Rename fof velocities

* Clean up and add Quijote

* Edit docstrings

* Update submission script

* Fix bug

* Start loading fitted properties

* Edit docstrings

* Update fitting for new `halo`

* Update CM definition and R200c

* Tune the minimum number of particles

* Enforce crossing threshold & tune hypers

* Fix periodiity when calculating angmom

* Doc strings

* Relax checkip

* Minor edit

* Fix old kwarg bug

* Fix CSiBORG bounds

* Catch warnings!

* Add `mass_kind` and new boundaries
2023-07-31 16:13:21 +02:00
rstiskalek
169a5e5bd7 Fix a small bug in paths 2023-07-29 09:27:45 +01:00
rstiskalek
a1141408ea Add subselection of Quijote snapshots 2023-07-29 09:25:03 +01:00
rstiskalek
ab9002a45a Delete unused import 2023-07-29 09:18:57 +01:00
Richard Stiskalek
5acc4684bb
Edits paths (#79) 2023-07-29 10:16:43 +02:00
rstiskalek
61077940f3 Fix neighbours 2023-07-28 20:14:05 +01:00
Richard Stiskalek
acb8d9571c
Update units to be consistent. (#78)
* Fix Quijote units

* Updates to units

* Fix how things are loaded

* Updating definitions & conventions

* Clear up how fiducial observers in quijote work

* Refactorize array manip

* Move function definition

* More code refactoring

* Remove unused argument

* Remove `convert_from_box`

* Make a note

* Converting particle units

* Add notes about units

* Remove box constants

* Add rho_crit0

* Fix spherical overdensity mass units

* Refactor more code

* Edit catalogue kwargs

* Edit the docstring

* Edit bounds

* Add new checks for empty array

* Remove unused import

* Remove old code

* Remove old function

* Update real 2 redshift

* Clear up the RSP conv

* Add comments

* Add some units
2023-07-28 21:07:28 +02:00
Richard Stiskalek
fb4b4edf19
Quijote snapshots support (#77)
* Renaming

* Edit docs

* Delete old function

* Add a blank space

* Rename particle reader

* Add comments

* Rename

* Rename

* edit get_snapshots

* More renaming

* Remove old correction

* Add import

* Add basics of the Quijote reader

* Add a blank space

* Fix paths

* Rename function

* Fix HID and path

* Add more FoF reading

* Move definition

* Adding arguments

* Renaming

* Add kwargs for backward comp

* FoF Quijote return only hids

* Add sorting of quijote

* Add path to CSiBORG ICs snapshot

* Add support for Quijote

* initmatch paths for quijote

* Add kwargs

* Fix blank lines

* Rename kwarg

* Remove unused import

* Remove hardcoded numbers

* Update for Quijote

* Do not store velocities in QUijote ICs

* Box units mass Quijote

* Fix typo

* Ensure particles are not right at the edge

* Add structfit paths for QUuijote

* Basic CSiBORG units

* Add more quijote halo reading

* Add Quijote fitting

* Docs changes

* Docs changes
2023-07-27 18:41:00 +02:00
Richard Stiskalek
e08c741fc8
Improving halo fits (#76)
* Add periodic distances

* Little corrections

* Fix little bug

* Modernise the script

* Small updates

* Remove clump

* Add new halo routines

* Fix weights

* Modernise the script

* Add check ups on convergence

* More convergence check ups

* Edit bounds

* Add default argument

* Update fit heuristic and NaNs

* Change maxiter

* Switch NFW minimization to log-sapce

* Remove print statement

* Turn convert_from_box abstract property required for all boxes.

* Move files

* Simplify script

* Improve the argument parser

* Remove optinal argument

* Improve argument parser

* Add a minimum concentration limit
2023-07-25 16:12:58 +02:00
Richard Stiskalek
eb8d070fff
CSiBORG FoF switch (#75)
* Add moving FoF membership files

* add FoF membership path

* Add notes where its PHEW

* Add FoF catalogue path

* Correct typo

* Add more functionalities

* Make work with halo IDs from FoF

* Edit print statement

* Fix copy bug

* copy

* Add FoF catalogue reading

* Clean up script

* Fix typo

* Little edits

* Fix naming convention

* Rename key

* Remove loading substructure particles

* Rename CSiBORG Cat

* Rename clumps cat

* Rename cat

* Remove misplaced import

* Switch to halos

* rm import

* structfit of only halos

* Add FoF halo reading

* Add a short comment

* Fix __getitem__ to work with int

* Fix problems

* Improve __getitem__

* Add more conversion

* Fix indexing

* Fix __getitem__ assertion

* Fix numbers

* Rename

* Fix verbosity flags

* Add full Quijote HMF option

* Add plot of Quijote only

* Add quijote full paths

* Fix the fit_init script

* Renam arg

* Update .gitignore

* add default argument name

* Change default verbosity flag

* Modernise script structure

* Fix dictionary

* Fix reading to include m200c

* Modernise script

* Add args
2023-07-24 14:10:21 +02:00
Richard Stiskalek
28e93e917f
More plotting (#74)
* Add a new plot

* Add a binned trend

* Fix bug

* Improve plot further

* Add new plotting

* add max overlap

* edit get_overlap

* Add max overlap plot

* Update plot

* Add max overlap key

* add max dist flag

* Improve plotting
2023-07-03 15:35:10 +01:00
Richard Stiskalek
fbf9c2a4b7
Better plots (#73)
* Edits paths of saved files

* Add upper threshold options

* Add upper threshold options

* add latex_float option

* Add weighted stats

* add new plot
2023-06-28 15:22:42 +01:00
Richard Stiskalek
de7def61d5
New RSP density distinction (#72)
* Edit docs

* Fix kernel units

* Add NGP interpolation

* Add Vobs but not implemented

* Add BORG density option

* Add BORG mcmc path

* Organise imports

* Add new density field distinction
2023-06-26 20:41:07 +01:00
Richard Stiskalek
27e1c181a2
Moving environment to RS (#70)
* Drag several fields at once

* add env in RSP

* Add docstrings

* Make __main__

* Fix bug

* Fix plotting little bug
2023-06-18 11:42:21 +01:00
Richard Stiskalek
35ccfb5c67
New matches (#69)
* Remove old file

* Add velocity plotting

* add smooth scale

* Fix bug

* Improve paths

* Edit plotting

* Add smoothed density

* Update boundaries

* Add basics

* Further docs

* Remove blank

* Better catalog broadcasting

* Update high res size

* Update plotting routines

* Update routine

* Update plotting

* Fix field saving name

* Add better colormap for environemnt
2023-06-17 19:52:26 +01:00
Richard Stiskalek
cbfd1cbc99
Fixing tidal field calculation (#68)
* Add import

* Fix tidal calculation

* Add env to paths

* Improve plot routines

* Add env classification
2023-06-16 18:31:43 +01:00
Richard Stiskalek
ccbbbd24b4
Add better diagnostics & plotting (#67)
* Add caching functions

* Add limts

* Add new mass runs

* Update .gitignore

* Edit which CDFs are loaded

* Stop saving cross hindxs

* Change dist to half precision

* New nearest path

* Add neighbour counting

* Add neighbour kwargs

* Update work in progress

* Add new counting

* Add how dist is built

* Collect dist to 1 file

* Update reading routine

* Delete Quijote files

* Remove file

* Back to float32

* Fix bugs

* Rename utils

* Remove neighbuor kwargs

* Rename file

* Fix bug

* Rename plt utils

* Change where nghb kwargs from

* line length

* Remove old notebooks

* Move survey

* Add white space

* Update TODO

* Update CDF calculation

* Update temporarily plotting

* Merge branch 'add_diagnostics' of github.com:Richard-Sti/csiborgtools into add_diagnostics

* Start adding documentation to plotting

* Remove comments

* Better code documentation

* Some work on tidal tensor

* Better plotting

* Add comment

* Remove nb

* Remove comment

* Add documentation

* Update plotting

* Update submission

* Update KL vs KS plots

* Update the plotting routine

* Update plotting

* Update plotting routines
2023-06-16 14:33:27 +01:00
rstiskalek
524434c7de Small updates 2023-06-05 22:28:37 +01:00
rstiskalek
07bb03b15d Update doc 2023-06-05 17:11:30 +01:00
rstiskalek
e89edf0af4 Remove unused import 2023-06-05 17:09:32 +01:00
Richard Stiskalek
63b6cdbe72
Forward model fields to RSP (#66)
* Add radial velocity field

* Add overdensity plot

* Flip velocities too

* Add field calculations

* Add RSP mapping

* Add potential in RSP

* Add projected field plotting
2023-06-05 17:24:20 +02:00
rstiskalek
015e6fb596 Fix units 2023-06-02 12:50:15 +01:00
rstiskalek
e7e1015279 Fix bug 2023-06-01 22:57:46 +01:00
rstiskalek
e28739835b Fix bug 2023-06-01 22:57:36 +01:00
rstiskalek
64bacc20db Add nside2radec 2023-06-01 22:57:29 +01:00
rstiskalek
c9829c0acc Fix bugs 2023-06-01 19:10:18 +01:00
rstiskalek
b8c58c7f82 Add import 2023-06-01 19:09:38 +01:00
Richard Stiskalek
dbf93b9416
Lagrangian patch + HMF calculation (#65)
* Rename lagpatch

* Fix old bug

* Fix small bug

* Add number of cells calculation

* Fix a small bug

* Rename column

* Move file

* Small changes

* Edit style

* Add plot script

* Add delta2ncells

* Add HMF calculation

* Move definition around

* Add HMF plot

* pep8

* Update HMF plotting routine

* Small edit
2023-06-01 14:45:52 +01:00
Richard Stiskalek
f1dbe6f03f
Add plotting (#64)
* Add verbosity statements

* More verbosity

* Save masses too

* Add CDF new plot

* Blank line

* Fix RVS sampling bug

* Add R200 conversion

* Simplify plotting routines

* Remove imoprt
2023-05-27 00:08:39 +01:00
rstiskalek
10fdf6d40e Remove limit 2023-05-25 15:19:28 +01:00
rstiskalek
4eed5e1cc2 Do not return log p 2023-05-24 11:25:30 +01:00
rstiskalek
f82633f816 Add KL divergence 2023-05-24 10:35:50 +01:00
rstiskalek
fa2b3551c1 Add the sigma value calculation 2023-05-22 13:27:58 +01:00
rstiskalek
9b4c02f3b0 pep8 2023-05-21 22:47:09 +01:00
Richard Stiskalek
2185846e90
CDF for nearest neighbour (#63)
* Updat ebounds

* fix mistake

* add plot script

* fix which sims

* Add Poisson

* Just docs

* Hide things to __main__

* Rename paths

* Move old script

* Remove radpos

* Paths renaming

* Paths renaming

* Remove trunk stuff

* Add import

* Add nearest neighbour search

* Add Quijote fiducial indices

* Add final snapshot matching

* Add fiducial observer selection

* add boxsizes

* Add reading functions

* Little stuff

* Bring back the fiducial observer

* Add arguments

* Add quijote paths

* Add notes

* Get this running

* Add yaml

* Remove Poisson stuff

* Get the 2PCF script running

* Add not finished htings

* Remove comment

* Verbosity only on 0th rank!

* Update plotting style

* Add nearest neighbour CDF

* Save radial distance too

* Add centres

* Add basic plotting
2023-05-21 22:46:28 +01:00
Richard Stiskalek
255bec9710
Quijote kNN adding (#62)
* Fix small bug

* Add fiducial observers

* Rename 1D knn

* Add new bounds system

* rm whitespace

* Add boudns

* Add simname to paths

* Add fiducial obserevrs

* apply bounds only if not none

* Add TODO

* add simnames

* update script

* Fix distance bug

* update yaml

* Update file reading

* Update gitignore

* Add plots

* add check if empty list

* add func to obtaining cross

* Update nb

* Remove blank lines

* update ignroes

* loop over a few ics

* update gitignore

* add comments
2023-05-15 23:30:10 +01:00
Richard Stiskalek
1d847cbd06
Add Quijote (#61)
* Rename paths object

* Remove redshift calculation

* Explicit keywrod arg

* Rename box units

* Basic renaming

* Little docs

* Rename paths

* add imports

* Sort imports

* Add Quijote cat

* Split boxes

* add Quijote path

* Add origin argument

* Update nbs
2023-05-13 17:37:34 +01:00
Richard Stiskalek
b3fd14b81f
Gravitational potential calculation (#60)
* Rename file

* add velocity field

* Add velcoity field

* Move smoothening

* Add verbosity flags

* remove blank

* Simplify paths

* Add potential calculation

* Update paths

* Add potential field

* Add potential calculation

* Move away sky matching

* Move interpolation functions

* Update nbs
2023-05-12 13:07:58 +01:00
Richard Stiskalek
b710b8e89c
Add new ICs (#59)
* edit IC paths

* Remove import

* Edit path

* Change naming

* Add __main__

* Script to match everything

* Edit docs

* Remove test statement

* Move import

* Update nb
2023-05-09 16:18:01 +01:00
Richard Stiskalek
cb5d35b1be
Density field calculation (#58)
* Add iterative density field generation

* Edit particle halfwidth selection

* Update import

* Remove old file

* Add position wrapping

* Add RSD support

* Add density field calculation

* Edit paths to the density field

* Flip argument order

* Edit documentation

* Edit docs

* update nb
2023-05-08 14:02:20 +01:00
Richard Stiskalek
4265315b47
Revert "Run density field estimator (#56)" (#57)
This reverts commit 98d0578fa7.
2023-05-08 13:59:55 +01:00
Richard Stiskalek
98d0578fa7
Run density field estimator (#56)
* Add iterative density field generation

* Edit particle halfwidth selection

* Update import

* Remove old file

* Add position wrapping

* Add RSD support

* Add density field calculation

* Edit paths to the density field

* Flip argument order
2023-05-08 13:58:12 +01:00
Richard Stiskalek
65564849f4
Add redshift space position (#55)
* Add docs

* Delete blank

* Add real to redshift space function

* RSD move back observer

* Convert velocoties to km/s

* Add units to RSP calculation

* Add RSP to catalogues

* Add RSD to angular neighbour search
2023-05-07 19:07:35 +01:00
rstiskalek
3df771f1ee Blank line 2023-05-07 16:36:42 +01:00
Richard Stiskalek
51b670d30b
Updates to the overlap reader. (#54)
* Flag for loading clumps_cat

* Optionally load clumps catalogues

* Update single pair reading

* Edit reading many pairs
2023-05-07 16:34:55 +01:00
Richard Stiskalek
56e39a8b1d
Overlapper improvements (#53)
* Store indices as f32

* Fix init sorting

* Organise imports

* Rename pathing

* Add particle loading

* Improve particle reading

* Add h5py reader

* edit particle path

* Update particles loading

* update particles loading

* Fix particle dumping

* Add init fitting

* Fix bug due to insufficient precision

* Add commnet

* Add comment

* Add clumps catalogue to halo cat

* Add comment

* Make sure PIDS never forced to float32

* fix pid reading

* fix pid reading

* Update matching to work with new arrays

* Stop using cubical sub boxes, turn off nshift if no smoothing

* Improve caching

* Move function definitions

* Simplify calculation

* Add import

* Small updates to the halo

* Simplify calculation

* Simplify looping calculation

* fix tonew

* Add initial data

* Add skip condition

* Add unit conversion

* Add loading background in batches

* Rename mmain index

* Switch overlaps to h5

* Add finite lagpatch check

* fix column name

* Add verbosity flags

* Save halo IDs instead.

* Switch back to npz

* Delte nbs

* Reduce size of the box

* Load correct bckg of halos being matched

* Remove verbosity

* verbosity edits

* Change lower thresholds
2023-05-06 16:52:48 +01:00
Richard Stiskalek
1a9e6177d7
Switch to h5py format (#52)
* Edit the particle paths

* Remove script

* Add h5py to dumping

* Minor adjustments

* add h5py support

* remove split path

* h5py support

* Type

* Edit initmatch paths

* Shorten func

* dist_centmass to work with 2D arrays

* Forgot to return the centre of mass

* Fixed code

* Fix halo bug

* Start MPI broadcasting

* Mini bug

* Remove commenting

* Remove test statement

* Fix index

* Printing from rank 0 only

* Move where clump index stored

* Add dtype options

* Add dtype options
2023-05-02 13:57:13 +01:00
Richard Stiskalek
553eec8228
Update field calculations (#51)
* Update density field routines

* Add utils

* Add possibility to return 2d array

* Styling

* Fixed density field

* Fix bugs

* add mpc2box

* Fix evaluating sky

* Fix sky map making

* Rename file

* Add paths if only positions

* Add option to dump particles only

* Add comments
2023-04-29 22:57:05 +01:00
Richard Stiskalek
b5fefe4196
Update init matching procedure (#50)
* pep8

* fix convention

* Update script

* enforce optimisation boundaries to be finite

* Update TODO

* Remove sky matching

* FIx a small bug

* fix bug

* Remove import

* Add halo fitted quantities

* Update nbs

* update README

* Add load_initial comments

* Rename nbs

* Delete nb

* Update imports

* Rename function

* Update matcher

* Add overlap paths

* Update the matching script

* Update verbosity

* Add verbosity flags

* Simplify make_bckg_delta

* bug fix

* fix bug

* lala

* la

* Add overlap paths

* Update limit

* pep8

* Some pep8 stuff

* pep8

* Minor corrections

* Update paths

* Add number of particles check

* Fix bug

* pep8 comments
2023-04-29 14:38:21 +01:00
Richard Stiskalek
c14be720b5
Particle dumping (#49)
* Add paths

* Add dumping script

* Remove snap from density fields

* Add progress statement

* Update import ordering

* Update nb
2023-04-28 13:57:23 +01:00
Richard Stiskalek
f48eb6dcb0
Dump radial profile information (#48)
* add radial position path

* pep8

* Add basic fit profile dumping

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* Update TODO

* Fix parts is None bug

* Update nb
2023-04-27 00:18:30 +01:00
Richard Stiskalek
04119a5314
Update initial matching & overlaps (#47)
* pep8

* fix convention

* Update script

* enforce optimisation boundaries to be finite

* Update TODO

* Remove sky matching

* FIx a small bug

* fix bug

* Remove import

* Add halo fitted quantities

* Update nbs

* update README

* Add load_initial comments

* Rename nbs

* Delete nb

* Update imports

* Rename function

* Update matcher

* Add overlap paths

* Update the matching script

* Update verbosity

* Add verbosity flags

* Simplify make_bckg_delta

* bug fix

* fix bug
2023-04-21 01:35:06 +02:00
Richard Stiskalek
39b3498621
Correct fitting script for both clumps and halos (#46)
* Minor typos

* fix minor bugs

* pep8

* formatting

* pep8

* Fix minor bugs

* New path & pep8

* add splitt

* Updates

* Improve calculation within radius

* pep8

* pep8

* get the script working

* Add matter overdensity

* Add m200m to the script

* Fix looping bug

* add parents support

* add import

* Optionally concatenate velocities

* Make optional masking

* Ignore the error message

* Start reading in raw data

* Fix cat reading

* Additional units conversions

* Add clump reading

* Fix indexing

* Remove old comment

* Remove old comment

* set npart to 0 instead of overflow from NaN

* fix docs

* rm boring stuff

* Remove old stuff

* Remove old stuff

* Remove old comment

* Update nb
2023-04-19 16:39:35 +02:00
Richard Stiskalek
c2fde1566b
Cluster rough angular match (#45)
* Remove sim dump

* Remove unused things

* Remove warning

* Fix coordinate transforms

* Add imports

* Update coordinate transfers

* Add angular neighbours calculation

* Update nb
2023-04-18 18:49:21 +02:00
Richard Stiskalek
fdb0df8d4c
Add mmain and other major updates (#44)
* Move paths to a separate file

* Add mmain reader

* Add a verbosity flag

* Fix imports

* Fix bug

* Rename files

* Return ultimate parents

* Add script to generate mmain

* Remove mmain path

* edit path

* Add mmain path

* Change function name

* Rename function

* Turn off verbose

* Fix list requirement

* Edit init match paths

* Fix init pathing

* Edit paths docs

* Edit dumpdir name

* Rename path

* Fix split paths

* Remove unused import

* Add comment

* Update readme

* remove read mmain

* Rename haloatalogue

* Fix minor bugs

* Update nbs

* Add create directory option

* Move split jobs

* Move spliot jobs

* Remove splitting

* Add import

* Edit script

* Deeper split folder

* Fix paths bug

* Rename catalogue

* Rename Catalogue

* Add new clumpread

* Edit paths

* add knn paths

* Update commenting

* Update imports

* Add more conversions

* Update temp file

* Add a note

* Add catalogue

* Cooment

* Update TODO

* Update script

* add nb

* Update

* pep8

* edit paths & pep8

* Fix knn auto paths

* add paths docs

* Add auto and cross knn paths

* Add new paths

* Simplify tpcf reading

* pep8 patch

* update readme

* Update progress

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* pep8

* Pep 8 and restructure

* add lambda spin

* add clump and halo

* add checks

* Edit halo profile fit

* Update gitignore

* backup script
2023-04-18 11:02:36 +02:00
Richard Stiskalek
e0d3854277
Add cleaner paths (#43)
* ignore venv

* ignore pylians if unavaible

* Edit loading paths

* Edit paths

* Stop having default paths

* Add nb

* Add glamdring paths

---------

Co-authored-by: Richard Stiskalek <dc-stis1@login8b.pri.cosma7.alces.network>
Co-authored-by: Richard Stiskalek <dc-stis1@login8a.pri.cosma7.alces.network>
2023-04-14 23:24:52 +01:00
Richard Stiskalek
1344fa40b6
Add 2PCF calculation (#42)
* Add 2PCF calculation

* Add 2PCF reader

* Add tpcf scripts

* Add random state

* Fix marked typo

* Fix marks and more randoms

* Stop calculating projected

* Add mean 2PCF

* Remove pimax

* Edit submit script

* Update nb

* Add corrfunc check
2023-04-14 23:05:48 +01:00
Richard Stiskalek
0b743756ef
File organisation (#41)
* Split summary files

* Rename script

* Cosmetics

* Minor changes

* Update TODO
2023-04-09 21:12:19 +01:00
Richard Stiskalek
5784011de0
kNN-CDF secondary halo bias (#40)
* Add seperate autoknn script & config file

* edit ics

* Edit submission script

* Add threshold values

* Edit batch sizign

* Remove print

* edit

* Rename files

* Rename

* Update nb

* edit runs

* Edit submit

* Add median threshold

* add new auto reader

* editt submit

* edit submit

* Edit submit

* Add mean prk

* Edit runs

* Remove correlation file

* Move split to clutering

* Add init

* Remove import

* Add the file

* Add correlation reading

* Edit scripts

* Add below and above median permutation for cross

* Update imports

* Move rvs_in_sphere

* Create utils

* Split

* Add import

* Add normalised marks

* Add import

* Edit readme

* Clean up submission file

* Stop tracking submit files

* Update gitignore

* Add poisson field analytical expression

* Add abstract generators

* Add generators

* Pass in the generator

* Add a check for if there are any files

* Start saving average density

* Update nb

* Update readme

* Update units

* Edit jobs

* Update submits

* Update reader

* Add random crossing

* Update crossing script

* Add crossing with random

* Update readme

* Update notebook
2023-04-09 20:57:05 +01:00
Richard Stiskalek
826ab61d2d
Add p(V | k) (#39)
* add p(V | k)

* Update nb
2023-04-05 14:47:30 +01:00
Richard Stiskalek
8d34b832af
Improve paths and documentation (#38)
* pep8

* style issue

* Documentation edits

* Remove old files

* Remove more old files

* Doc & stop setting snap and nsim in paths

* add nsnap, nsim support

* Remove blank space

* docs

* Docs edits

* Reduce redudant code

* docs

* Documentation

* Docs

* Simplify

* add nsnap nsim arguments

* Remove redundant docs

* Fix typos

* Shorten catalogue

* Remove import

* Remove blank line

* Docs only edits

* Rearrange imports

* Remove blank space

* Rearrange imports

* Rearrange imports

* Remove blank space

* Update submission scritp

* Edit docs

* Remove blank line

* new paths

* Update submission scripts

* Update file handling

* Remove blank line

* Move things around

* Ne paths

* Edit submission script

* edit paths

* Fix typo

* Fix bug

* Remove import

* Update nb
2023-04-04 15:22:00 +01:00
rstiskalek
035d7e0071 add crossfiles reader 2023-04-02 22:23:50 +01:00
Richard Stiskalek
9cbdef4350
kNN-CDF reader (#37)
* Add nb

* add the KNN reader

* Move reading functions

* Update boolean masking

* Update the nb
2023-04-02 20:26:15 +01:00
Richard Stiskalek
522ee709c9
Joint kNN-CDF calculation (#36)
* Add joint kNN CDF

* add jointKNN calculation

* change sub script

* Update readme

* update sub

* Small changes

* comments

* update nb

* Update submisison script
2023-04-01 11:16:11 +01:00
rstiskalek
8fc4e614e3 unused variable 2023-04-01 07:55:17 +01:00
rstiskalek
7610def8a0 Add brute KNN 2023-04-01 07:52:34 +01:00
rstiskalek
070b9b6c1b Add batch sizing for less memory 2023-04-01 07:11:39 +01:00
Richard Stiskalek
63ab3548b4
kNN-CDF implementation (#34)
* Rewrite doc

* add kNN

* edit loading of samples with no init

* Add verbosity flag

* add KNN submission script

* do not make peaked cdf by default

* Add submit script

* stop ignore sh

* Add mass thresholding

* Edit gitignore

* edits

* Space points in logspace

* Calculate for all ICs

* Update TODO

* Add dtype support

* Update readme

* Update nb
2023-03-31 18:13:41 +01:00
Richard Stiskalek
5dd8c668fa
Gaussian smoothing of density fields (#33)
* Simplify smoothing support and looping over nonzero

* Simplify comments

* add now()

* add cat length

* add smoothed calculation

* add smoothing

* Add sorting

* Edit what is ignored

* Move notebooks

* Add nonsymmetric smoothed overlap

* Update NB

* Add support for reading in the smoothed overlap

* Switch to the true overlap definition

* Reader of the true overlap

* rem occups

* Import moved to a class

* Move definition

* Edit submission script

* Update to account for the new definition

* backup nb

* Switch back to properly initialising arrays

* Fix addition bug

* Update NB

* Fix little bug

* Update nb
2023-03-27 09:22:03 +01:00
Richard Stiskalek
9b524db617
Add a shared reader (#32)
* add import

* Rename object

* Simplify how catalogs are handled

* Move functions around

* Add NPair reader

* Add counterpart Gaussian average

* Change what is returned in exp mass

* small bug

* Simplify stat calcu

* Add mptebppl
2023-03-16 18:02:21 +00:00
Richard Stiskalek
153f1c0002
Overlap reader thresholds (#31)
* Improve data

* Add comment

* Update how KNN is called

* Bring back indices

* New function output

* return catx["index"]

* Remove unnecessary arguments

* Remove useless arguments

* Rename output

* thin up catalogues

* Add thresholding

* Update README
2023-03-15 20:55:50 +00:00
Richard Stiskalek
6e290ccfb4
Add support for inverting the reader (#30) 2023-03-09 21:54:13 +00:00
Richard Stiskalek
f2cef73977
Final overlap definition update.. (#29)
* Delete whitespace

* Update overlap definition

* Add documentation

* add overlap args

* Make the background array store only high density region

* Add nsims to args

* Remove conc as done inside the func

* Simplify the overlap calculation

* Rename variable

* Just some docs

* Docs

* Correct overlap definition

* Undo debug comment

* Remove old debug code

* Add prob of no match and exp couunterpart mass

* docs

* Update the overlap definition
2023-03-09 14:55:38 +00:00
Richard Stiskalek
1c859dbdac
Overlap reader (#28)
* Edit output path

* remove matching of many sims

* Add a class skeleton

* Add a setter for only nsim

* Deletei mport

* Simplify cross and delete combined halo catalogue

* Simplify reading in catalogues

* Add distance calculation

* Add mass ratio

* Clean up the submission script

* Add the summer overlap

* Docs

* Add array copying

* Add import

* Add resampling mean
2023-02-08 12:05:32 +00:00
Richard Stiskalek
8dea3da4de
Speed up overlap (#27)
* Edit improt

* Simplify patch size calculation

* Add patch size percentiles

* Add various percentiles

* Remove  comment

* Update TODO

* Change to 95th percentile

* Add import

* Add KNN properties

* Add new matching initial condition

* Add import

* Remove import

* Add fast neighbours option

* Further edits to fast neighbours

* add imports

* add new overlap calculation and non-zero things

* Remove print

* Clean up code

* Fix small bug

* Remove comment

* Add run single cross match

* change values

* Edit hyperparams

* Add comment

* Add the argument parser

* Add new lagpatch calc

* New lagpatch calc

* Delete old patch definitions

* Make clump dumping once again optional

* Add lagpatch to the catalogue

* Edit print statement

* Fix small bug

* Remove init radius

* Change to lagpatch key

* Fix a small bug

* Fix little bug
2023-02-05 11:46:19 +00:00
Richard Stiskalek
beb811e84c
Switch initial radius definition (#26)
* Add search for neighbours at z = 0

* Add initial snapshot KNN

* Add initi search either z =0 or  z = 70

* Add import

* Add clumps_pos2cell

* Add function argument

* Add import

* Add spherical overlap and speed up make_delta

* Add clump limits calculation

* Sped up make_delta

* Add patch size conversion

* Add patch sizes

* Add patch size calculation

* Force catalogues to be in float32

* Optimised script

* Option to remove points with no overlap

* Add spherical aproximate overlap

* Fix subboxing bug

* Remove print diagnostics

* Add Lagrangian patch size

* Add patch size documentation

* Move when clumpsx converted to int

* Edit docs

* Remove spherical overlap

* New Langrangian patch calculation
2023-01-31 15:09:35 +00:00
Richard Stiskalek
830b1b8daf
Fix weights definition (#24)
* Fix running script

* Increase outside boundary size

* Add new weight scheme

* Edit docs
2023-01-16 16:14:38 +00:00
Richard Stiskalek
279580e041
Fix overlap weights (#23)
* Remove overlapper change cells

* Remove Pylians MASL and replace with custom

* Add concatenate clumps

* Add weighted overlap
2023-01-14 14:34:37 +00:00
Richard Stiskalek
2e99b901ac
Environmental properties (#20)
* rm get_positions

* Add comment

* add halfwidth func

* Update docs

* Add imprt

* Evaluate multiple fields simulatenously

* add halfwidth selection

* Change order of grav field and tensor field

* Add gravitational field norm

* Add eigenvalue calculation

* Sorted eigenvalues

* add init script

* add progress

* Add surveys

* Add more survey flexibility

* Minor changes

* add survey names

* rm name

* Fix list bug

* Fig bugs when running the script

* add phi to dtype

* fix dump bug

* Add comment

* Add smoothing options

* Add further comment

* Update TODO
2022-12-31 17:46:05 +00:00
Richard Stiskalek
3dc559fec1
Overlap calculation (#19)
* Rm comment

* Add new fixed particle overlap

* Fix overlap calculation

* Update docs

* Add new overlapper support in matcher

* add filter optin

* Add the real space filter.

* Change filtering to real space only

* Update TODO

* add high-resolution switch

* add cross script

* Remove comment

* Add smoothing option
2022-12-27 18:32:12 +01:00
Richard Stiskalek
18f09767f4
Overlap calculation (#18)
* Move cosine similarity out

* Basic overlap calculation

* add overlap import

* Add clump0 dict path

* Update README

* fix path bug

* Save as dict instead

* Change format to array of arrays

* Update paths

* Take fewer ICs for now

* Change to structured array

* Add overlap calculation

* Start saving IDs

* Add a blank space

* Update TODO
2022-12-19 11:58:22 +01:00
Richard Stiskalek
13a9d11afe
Particle init CM separation (#17)
* Rename

* rename module

* add new_ic_ids

* Add to_new support

* rm unnecessary comment

* Remove JUlia!

* add new package attempt

* add dump script

* simplify positions getter

* Remove unused code

* change dumpdirs

* update project

* Add sorting

* fix indexing bug

* add comment

* Add particle dumping

* Delete blank line

* Delete Julia things

* Update README

* add working script

* fix assignment bug

* fix bug

* save temp to temp folder

* fix indexing bug & path

* do not always crreate folder

* Rename file

* add initcm reader

* add x0, y0, z0 transforms

* add initcm to halo cat

* docs update

* add initial KNN

* add attempt at init match

* add initial fiducial radius

* rm blank space

* Add further KNN Init pos support

* Add init radius support

* Remove message about the bug

* Get rid of an error msg

* purge mention of knn_init

* Add init CM distance calc
2022-12-16 11:18:16 +01:00
Richard Stiskalek
c3c686fa60
Power spectrum stuff (#16)
* add autok1d

* update TODO

* add basic cross pk

* del blank line

* fix small bugs

* fix loop bug

* rm nbs

* edit docstring

* Update TODO

* get rid of verbose flag

* take all ICs

* attempt better memory

* simplify ASCII dump

* rm verbose statement to get rid of bugs!

* close fortran files

* add MAS options

* add boxsize setter

* Move comment up

* paths chagne

* Update TODO

* Update TODO

* Forgotten units in Xpk

* Attempt at fixing the distance bug

* Update TODO

* Update TODO

* Remove comment

* add summary reader

* Update gitignore
2022-12-12 14:42:44 +00:00
Richard Stiskalek
1bb4255874
SDSS galaxies and catalogs rework (#15)
* add SDSS skeleton

* rm nb

* update gitignore

* add docs

* add survey volume

* add masking

* add SDSS mask

* add SDSS comments

* add reference

* rm

* move planck to fits clusters

* Rm comment

* update masking routines

* rm masks

* Update docs

* rm data

* make MCXC fits surv

* change MCXC name

* Move to extrasapce

* Update paths
2022-12-05 11:36:41 +00:00
Richard Stiskalek
91beb4df50
Angular momentum (#14)
* add ang mom

* Update TODO

* add ascii dump script

* add comment

* Updating TODO

* add angmom

* add garbage collector
2022-11-30 16:12:39 +00:00
rstiskalek
5435781e8d add the grav field 2022-11-30 13:41:35 +00:00
Richard Stiskalek
6aa3721c83
Potential (#13)
* add basic density field

* Add TODO

* add field smoothing

* update how pos are calculated

* add transforms both ways

* add import

* add sky density

* add make skymap func

* update TODO

* update gitignore

* add potential field calculation

* delete boxsize setter

* add tidal tensor
2022-11-27 07:53:38 +00:00
rstiskalek
2a5d6a03b6 add distance cut 2022-11-25 16:45:27 +00:00
Richard Stiskalek
ca45fbd5d9
add ascii writing (#12) 2022-11-25 15:06:10 +00:00
rstiskalek
8d68f35cf9 swithc to own G 2022-11-25 12:28:46 +00:00
rstiskalek
69d1f28fcc add lambda200c calc 2022-11-25 12:23:43 +00:00
rstiskalek
e5208ed070 add note 2022-11-25 11:52:31 +00:00
rstiskalek
91bddb1d3a remove not good comments 2022-11-25 11:45:47 +00:00
rstiskalek
9fe745b0b4 add center of mass 2022-11-25 10:54:09 +00:00
Richard Stiskalek
161c27d995
Particle match & file system & phase space (#11)
* Create file system

* add doc

* add n_sim n_snap directly to paths

* Move things to a single particle reader for consistency

* add docstring

* add srdcir, dumpdir and mmain_path

* make boxunits work with paths

* switch to using paths

* add tempdumpdir

* rm dependence on old functions

* rm comment

* rm unused import

* go back to all imports

* fix import bug

* rm dependence on old functions

* modernize code!

* fix typo

* fix typo

* update fits to new data structureing

* change docs

* add julia repo

* add setup

* add install commands

* ignore install files

* add array flattening

* update dependene

* add positions reader

* update manifest and projects

* add func

* update gitignore

* pos matching progress

* move file

* rm comment

* add velocities getter

* fix bug

* fix name bug

* fix path bug

* fix args func

* add redshift calculation to catalogues

* add shortcut to set n_sim and n_snap

* if cond bug

* add the cosine similarity

* add verbosit iterator

* add docs

* update README

* update README

* update README
2022-11-25 10:44:08 +00:00
Richard Stiskalek
c748c87e45
New filesystem, basic neighbours calculation (#10)
* simplify Planck catalogue

* add MCXC and base survey

* Add 2MPP classes

* move match to MCXC to Planck

* add halo catalogue

* rm comment

* rm unused imports

* Move conversion to box

* add min mass

* Run on all simulations

* rm old function

* add combined catalogue

* add halo positions

* add knn neighbors

* set to 5 sims for testing

* add docstring

* Switch to neighbours in a catalogue

* rename io to read

* fix indentation

* rename to read

* io -> read

* add import

* add RealisationMatcher

* io -> read

* add docstrings

* add search_sim_indiices

* update todo

* keep make_cat at 10 for now

* update nb
2022-11-22 10:00:01 +00:00