Commit graph

326 commits

Author SHA1 Message Date
Richard Stiskalek
826ab61d2d
Add p(V | k) (#39)
* add p(V | k)

* Update nb
2023-04-05 14:47:30 +01:00
Richard Stiskalek
8d34b832af
Improve paths and documentation (#38)
* pep8

* style issue

* Documentation edits

* Remove old files

* Remove more old files

* Doc & stop setting snap and nsim in paths

* add nsnap, nsim support

* Remove blank space

* docs

* Docs edits

* Reduce redudant code

* docs

* Documentation

* Docs

* Simplify

* add nsnap nsim arguments

* Remove redundant docs

* Fix typos

* Shorten catalogue

* Remove import

* Remove blank line

* Docs only edits

* Rearrange imports

* Remove blank space

* Rearrange imports

* Rearrange imports

* Remove blank space

* Update submission scritp

* Edit docs

* Remove blank line

* new paths

* Update submission scripts

* Update file handling

* Remove blank line

* Move things around

* Ne paths

* Edit submission script

* edit paths

* Fix typo

* Fix bug

* Remove import

* Update nb
2023-04-04 15:22:00 +01:00
rstiskalek
035d7e0071 add crossfiles reader 2023-04-02 22:23:50 +01:00
Richard Stiskalek
9cbdef4350
kNN-CDF reader (#37)
* Add nb

* add the KNN reader

* Move reading functions

* Update boolean masking

* Update the nb
2023-04-02 20:26:15 +01:00
rstiskalek
b1b08b8e53 Edit TODO 2023-04-01 11:43:39 +01:00
Richard Stiskalek
522ee709c9
Joint kNN-CDF calculation (#36)
* Add joint kNN CDF

* add jointKNN calculation

* change sub script

* Update readme

* update sub

* Small changes

* comments

* update nb

* Update submisison script
2023-04-01 11:16:11 +01:00
rstiskalek
cb67e326c4 Merge branch 'master' of github.com:Richard-Sti/csiborgtools 2023-04-01 07:57:45 +01:00
Richard Stiskalek
513872ceb6
kNN memory batching (#35)
* Add batch sizing for less memory

* Add batch size to submission

* Update nb

* Add brute KNN

* unused variable

* Update nb
2023-04-01 07:57:21 +01:00
rstiskalek
f26cf087ab Update nb 2023-04-01 07:56:52 +01:00
rstiskalek
8fc4e614e3 unused variable 2023-04-01 07:55:17 +01:00
rstiskalek
7610def8a0 Add brute KNN 2023-04-01 07:52:34 +01:00
rstiskalek
097b498da6 Update nb 2023-04-01 07:16:10 +01:00
rstiskalek
8d933e7ba9 Add batch size to submission 2023-04-01 07:15:44 +01:00
rstiskalek
070b9b6c1b Add batch sizing for less memory 2023-04-01 07:11:39 +01:00
Richard Stiskalek
63ab3548b4
kNN-CDF implementation (#34)
* Rewrite doc

* add kNN

* edit loading of samples with no init

* Add verbosity flag

* add KNN submission script

* do not make peaked cdf by default

* Add submit script

* stop ignore sh

* Add mass thresholding

* Edit gitignore

* edits

* Space points in logspace

* Calculate for all ICs

* Update TODO

* Add dtype support

* Update readme

* Update nb
2023-03-31 18:13:41 +01:00
rstiskalek
4d7827006a test 2023-03-27 13:38:01 +01:00
Richard Stiskalek
5dd8c668fa
Gaussian smoothing of density fields (#33)
* Simplify smoothing support and looping over nonzero

* Simplify comments

* add now()

* add cat length

* add smoothed calculation

* add smoothing

* Add sorting

* Edit what is ignored

* Move notebooks

* Add nonsymmetric smoothed overlap

* Update NB

* Add support for reading in the smoothed overlap

* Switch to the true overlap definition

* Reader of the true overlap

* rem occups

* Import moved to a class

* Move definition

* Edit submission script

* Update to account for the new definition

* backup nb

* Switch back to properly initialising arrays

* Fix addition bug

* Update NB

* Fix little bug

* Update nb
2023-03-27 09:22:03 +01:00
Richard Stiskalek
9b524db617
Add a shared reader (#32)
* add import

* Rename object

* Simplify how catalogs are handled

* Move functions around

* Add NPair reader

* Add counterpart Gaussian average

* Change what is returned in exp mass

* small bug

* Simplify stat calcu

* Add mptebppl
2023-03-16 18:02:21 +00:00
Richard Stiskalek
153f1c0002
Overlap reader thresholds (#31)
* Improve data

* Add comment

* Update how KNN is called

* Bring back indices

* New function output

* return catx["index"]

* Remove unnecessary arguments

* Remove useless arguments

* Rename output

* thin up catalogues

* Add thresholding

* Update README
2023-03-15 20:55:50 +00:00
Richard Stiskalek
6e290ccfb4
Add support for inverting the reader (#30) 2023-03-09 21:54:13 +00:00
Richard Stiskalek
f2cef73977
Final overlap definition update.. (#29)
* Delete whitespace

* Update overlap definition

* Add documentation

* add overlap args

* Make the background array store only high density region

* Add nsims to args

* Remove conc as done inside the func

* Simplify the overlap calculation

* Rename variable

* Just some docs

* Docs

* Correct overlap definition

* Undo debug comment

* Remove old debug code

* Add prob of no match and exp couunterpart mass

* docs

* Update the overlap definition
2023-03-09 14:55:38 +00:00
Richard Stiskalek
1c859dbdac
Overlap reader (#28)
* Edit output path

* remove matching of many sims

* Add a class skeleton

* Add a setter for only nsim

* Deletei mport

* Simplify cross and delete combined halo catalogue

* Simplify reading in catalogues

* Add distance calculation

* Add mass ratio

* Clean up the submission script

* Add the summer overlap

* Docs

* Add array copying

* Add import

* Add resampling mean
2023-02-08 12:05:32 +00:00
Richard Stiskalek
8dea3da4de
Speed up overlap (#27)
* Edit improt

* Simplify patch size calculation

* Add patch size percentiles

* Add various percentiles

* Remove  comment

* Update TODO

* Change to 95th percentile

* Add import

* Add KNN properties

* Add new matching initial condition

* Add import

* Remove import

* Add fast neighbours option

* Further edits to fast neighbours

* add imports

* add new overlap calculation and non-zero things

* Remove print

* Clean up code

* Fix small bug

* Remove comment

* Add run single cross match

* change values

* Edit hyperparams

* Add comment

* Add the argument parser

* Add new lagpatch calc

* New lagpatch calc

* Delete old patch definitions

* Make clump dumping once again optional

* Add lagpatch to the catalogue

* Edit print statement

* Fix small bug

* Remove init radius

* Change to lagpatch key

* Fix a small bug

* Fix little bug
2023-02-05 11:46:19 +00:00
Richard Stiskalek
beb811e84c
Switch initial radius definition (#26)
* Add search for neighbours at z = 0

* Add initial snapshot KNN

* Add initi search either z =0 or  z = 70

* Add import

* Add clumps_pos2cell

* Add function argument

* Add import

* Add spherical overlap and speed up make_delta

* Add clump limits calculation

* Sped up make_delta

* Add patch size conversion

* Add patch sizes

* Add patch size calculation

* Force catalogues to be in float32

* Optimised script

* Option to remove points with no overlap

* Add spherical aproximate overlap

* Fix subboxing bug

* Remove print diagnostics

* Add Lagrangian patch size

* Add patch size documentation

* Move when clumpsx converted to int

* Edit docs

* Remove spherical overlap

* New Langrangian patch calculation
2023-01-31 15:09:35 +00:00
rstiskalek
912a38acfb Merge branch 'master' of github.com:Richard-Sti/csiborgtools 2023-01-17 11:47:37 +00:00
Richard Stiskalek
3bdadc4750
Change file naming convention (#25) 2023-01-17 11:47:11 +00:00
rstiskalek
88a109157d Change file naming convention 2023-01-17 11:46:07 +00:00
Richard Stiskalek
830b1b8daf
Fix weights definition (#24)
* Fix running script

* Increase outside boundary size

* Add new weight scheme

* Edit docs
2023-01-16 16:14:38 +00:00
Richard Stiskalek
279580e041
Fix overlap weights (#23)
* Remove overlapper change cells

* Remove Pylians MASL and replace with custom

* Add concatenate clumps

* Add weighted overlap
2023-01-14 14:34:37 +00:00
Richard Stiskalek
f0720243a2
Updates to the submission script (#22) 2023-01-01 21:43:52 +00:00
Richard Stiskalek
2e99b901ac
Environmental properties (#20)
* rm get_positions

* Add comment

* add halfwidth func

* Update docs

* Add imprt

* Evaluate multiple fields simulatenously

* add halfwidth selection

* Change order of grav field and tensor field

* Add gravitational field norm

* Add eigenvalue calculation

* Sorted eigenvalues

* add init script

* add progress

* Add surveys

* Add more survey flexibility

* Minor changes

* add survey names

* rm name

* Fix list bug

* Fig bugs when running the script

* add phi to dtype

* fix dump bug

* Add comment

* Add smoothing options

* Add further comment

* Update TODO
2022-12-31 17:46:05 +00:00
rstiskalek
65059f3798 Update TODO 2022-12-27 17:33:07 +00:00
Richard Stiskalek
3dc559fec1
Overlap calculation (#19)
* Rm comment

* Add new fixed particle overlap

* Fix overlap calculation

* Update docs

* Add new overlapper support in matcher

* add filter optin

* Add the real space filter.

* Change filtering to real space only

* Update TODO

* add high-resolution switch

* add cross script

* Remove comment

* Add smoothing option
2022-12-27 18:32:12 +01:00
rstiskalek
1edb566792 Update TODO 2022-12-20 12:11:18 +00:00
Richard Stiskalek
18f09767f4
Overlap calculation (#18)
* Move cosine similarity out

* Basic overlap calculation

* add overlap import

* Add clump0 dict path

* Update README

* fix path bug

* Save as dict instead

* Change format to array of arrays

* Update paths

* Take fewer ICs for now

* Change to structured array

* Add overlap calculation

* Start saving IDs

* Add a blank space

* Update TODO
2022-12-19 11:58:22 +01:00
Richard Stiskalek
13a9d11afe
Particle init CM separation (#17)
* Rename

* rename module

* add new_ic_ids

* Add to_new support

* rm unnecessary comment

* Remove JUlia!

* add new package attempt

* add dump script

* simplify positions getter

* Remove unused code

* change dumpdirs

* update project

* Add sorting

* fix indexing bug

* add comment

* Add particle dumping

* Delete blank line

* Delete Julia things

* Update README

* add working script

* fix assignment bug

* fix bug

* save temp to temp folder

* fix indexing bug & path

* do not always crreate folder

* Rename file

* add initcm reader

* add x0, y0, z0 transforms

* add initcm to halo cat

* docs update

* add initial KNN

* add attempt at init match

* add initial fiducial radius

* rm blank space

* Add further KNN Init pos support

* Add init radius support

* Remove message about the bug

* Get rid of an error msg

* purge mention of knn_init

* Add init CM distance calc
2022-12-16 11:18:16 +01:00
Richard Stiskalek
c3c686fa60
Power spectrum stuff (#16)
* add autok1d

* update TODO

* add basic cross pk

* del blank line

* fix small bugs

* fix loop bug

* rm nbs

* edit docstring

* Update TODO

* get rid of verbose flag

* take all ICs

* attempt better memory

* simplify ASCII dump

* rm verbose statement to get rid of bugs!

* close fortran files

* add MAS options

* add boxsize setter

* Move comment up

* paths chagne

* Update TODO

* Update TODO

* Forgotten units in Xpk

* Attempt at fixing the distance bug

* Update TODO

* Update TODO

* Remove comment

* add summary reader

* Update gitignore
2022-12-12 14:42:44 +00:00
Richard Stiskalek
1bb4255874
SDSS galaxies and catalogs rework (#15)
* add SDSS skeleton

* rm nb

* update gitignore

* add docs

* add survey volume

* add masking

* add SDSS mask

* add SDSS comments

* add reference

* rm

* move planck to fits clusters

* Rm comment

* update masking routines

* rm masks

* Update docs

* rm data

* make MCXC fits surv

* change MCXC name

* Move to extrasapce

* Update paths
2022-12-05 11:36:41 +00:00
Richard Stiskalek
91beb4df50
Angular momentum (#14)
* add ang mom

* Update TODO

* add ascii dump script

* add comment

* Updating TODO

* add angmom

* add garbage collector
2022-11-30 16:12:39 +00:00
rstiskalek
92699bfb0a update TODO 2022-11-30 13:42:00 +00:00
rstiskalek
5435781e8d add the grav field 2022-11-30 13:41:35 +00:00
Richard Stiskalek
6aa3721c83
Potential (#13)
* add basic density field

* Add TODO

* add field smoothing

* update how pos are calculated

* add transforms both ways

* add import

* add sky density

* add make skymap func

* update TODO

* update gitignore

* add potential field calculation

* delete boxsize setter

* add tidal tensor
2022-11-27 07:53:38 +00:00
rstiskalek
ea73dfcd56 Update TODO 2022-11-25 16:46:28 +00:00
rstiskalek
2a5d6a03b6 add distance cut 2022-11-25 16:45:27 +00:00
rstiskalek
d8adf56614 edit TODO 2022-11-25 15:07:53 +00:00
Richard Stiskalek
ca45fbd5d9
add ascii writing (#12) 2022-11-25 15:06:10 +00:00
rstiskalek
469f4988fa update TODO 2022-11-25 12:45:47 +00:00
rstiskalek
4e87a96b97 fix bug(again) 2022-11-25 12:42:49 +00:00
rstiskalek
8d68f35cf9 swithc to own G 2022-11-25 12:28:46 +00:00
rstiskalek
a3c7d03ceb fix bug 2022-11-25 12:28:41 +00:00