Richard Stiskalek
aaa14fc880
Add density field plot and start preparing CSiBORG2 ( #94 )
...
* Add RAMSES2HDF5 conversion
* Upload changes
* Clean up
* More clean up
* updates
* Little change
* pep9
* Add basic SPH calculation for a snapshot
* Add submit script
* Remove echo
* Little changes
* Send off changes
* Little formatting
* Little updates
* Add nthreads argument
* Upload chagnes
* Add nthreads arguemnts
* Some local changes..
* Update scripts
* Add submission script
* Update script
* Update params
* Rename CSiBORGBox to CSiBORG1box
* Rename CSiBORG1 reader
* Move files
* Rename folder again
* Add basic plotting here
* Add new skeletons
* Move def
* Update nbs
* Edit directories
* Rename files
* Add units to converted snapshots
* Fix empty dataset bug
* Delete file
* Edits to submission scripts
* Edit paths
* Update .gitignore
* Fix attrs
* Update weighting
* Fix RA/dec bug
* Add FORNAX cluster
* Little edit
* Remove boxes since will no longer need
* Move func back
* Edit to include sort by membership
* Edit paths
* Purge basic things
* Start removing
* Bring file back
* Scratch
* Update the rest
* Improve the entire file
* Remove old things
* Remove old
* Delete old things
* Fully updates
* Rename file
* Edit submit script
* Little things
* Add print statement
* Add here cols_to_structured
* Edit halo cat
* Remove old import
* Add comment
* Update paths manager
* Move file
* Remove file
* Add chains
2023-12-13 16:08:25 +00:00
Richard Stiskalek
e972f8e3f2
Add pynbody and other support ( #92 )
...
* Simplify box units
* Move old scripts
* Add printing
* Update readers
* Disable boundscheck
* Add new ordering
* Clean up imports
* Enforce dtype and add mass to quijote
* Simplify print statements
* Fix little typos
* Fix key bug
* Bug fixing
* Delete boring comments
* Improve ultimate clumps for PHEW
* Delete boring comments
* Add basic reading
* Remove 0th index HID
* Add flipping of X and Z
* Updates to halo catalogues
* Add ordered caching
* Fix flipping
* Add new flags
* Fix PHEW empty clumps
* Stop over-wrriting
* Little improvements to angular neighbours
* Add catalogue masking
* Change if-else statements
* Cache only filtered data
* Add PHEW cats
* Add comments
* Sort imports
* Get Quijote workign
* Docs
* Add HMF calculation
* Move to old
* Fix angular
* Add great circle distance
* Update imports
* Update impotrts
* Update docs
* Remove unused import
* Fix a quick bug
* Update compatibility
* Rename files
* Renaming
* Improve compatiblity
* Rename snapsht
* Fix snapshot bug
* Update interface
* Finish updating interface
* Update all paths
* Add old scripts
* Add basic halo
* Update imports
* Improve snapshot processing
* Update ordering
* Fix how CM positions accessed
* Add merger paths
* Add imports
* Add merger reading
* Add making a merger tree
* Add a basic merger tree reader
* Add imports
* Add main branch walking + comments + debuggin
* Get tree running
* Add working merger tree walking along main branch
* Add units conversion for merger data
* Add hid_to_array_index
* Update merger tree
* Add mergertree mass to PHEWcat
* Edit comments
* Add this to track changes...
* Fix a little bug
* Add mergertree mass
* Add cache clearing
* Improve summing substructure code
* Littbe bug
* Little updates to the merger tree reader
* Update .giignore
* Add box selection
* Add optional deletingf of a group
* add to keep track of changes
* Update changes
* Remove
* Add manual tracker
* Fix bug
* Add m200c_to_r200c
* Add manual halo tracking
* Remove skipped snapshots
* update cosmo params to match csiborg
* remove old comments
* Add SDSSxALFALFA
* Fix bugs
* Rename
* Edit paths
* Updates
* Add comments
* Add comment
* Add hour conversion
* Add imports
* Add new observation class
* Add selection
* Add imports
* Fix small bug
* Add field copying for safety
* Add matching to survey without masking
* Add P(k) calculation
* Add nb
* Edit comment
* Move files
* Remove merger import
* Edit setup.yp
* Fix typo
* Edit import warnigns
* update nb
* Update README
* Update README
* Update README
* Add skeleton
* Add skeleton
2023-12-07 14:23:32 +00:00
Richard Stiskalek
eb8d070fff
CSiBORG FoF switch ( #75 )
...
* Add moving FoF membership files
* add FoF membership path
* Add notes where its PHEW
* Add FoF catalogue path
* Correct typo
* Add more functionalities
* Make work with halo IDs from FoF
* Edit print statement
* Fix copy bug
* copy
* Add FoF catalogue reading
* Clean up script
* Fix typo
* Little edits
* Fix naming convention
* Rename key
* Remove loading substructure particles
* Rename CSiBORG Cat
* Rename clumps cat
* Rename cat
* Remove misplaced import
* Switch to halos
* rm import
* structfit of only halos
* Add FoF halo reading
* Add a short comment
* Fix __getitem__ to work with int
* Fix problems
* Improve __getitem__
* Add more conversion
* Fix indexing
* Fix __getitem__ assertion
* Fix numbers
* Rename
* Fix verbosity flags
* Add full Quijote HMF option
* Add plot of Quijote only
* Add quijote full paths
* Fix the fit_init script
* Renam arg
* Update .gitignore
* add default argument name
* Change default verbosity flag
* Modernise script structure
* Fix dictionary
* Fix reading to include m200c
* Modernise script
* Add args
2023-07-24 14:10:21 +02:00
Richard Stiskalek
ccbbbd24b4
Add better diagnostics & plotting ( #67 )
...
* Add caching functions
* Add limts
* Add new mass runs
* Update .gitignore
* Edit which CDFs are loaded
* Stop saving cross hindxs
* Change dist to half precision
* New nearest path
* Add neighbour counting
* Add neighbour kwargs
* Update work in progress
* Add new counting
* Add how dist is built
* Collect dist to 1 file
* Update reading routine
* Delete Quijote files
* Remove file
* Back to float32
* Fix bugs
* Rename utils
* Remove neighbuor kwargs
* Rename file
* Fix bug
* Rename plt utils
* Change where nghb kwargs from
* line length
* Remove old notebooks
* Move survey
* Add white space
* Update TODO
* Update CDF calculation
* Update temporarily plotting
* Merge branch 'add_diagnostics' of github.com:Richard-Sti/csiborgtools into add_diagnostics
* Start adding documentation to plotting
* Remove comments
* Better code documentation
* Some work on tidal tensor
* Better plotting
* Add comment
* Remove nb
* Remove comment
* Add documentation
* Update plotting
* Update submission
* Update KL vs KS plots
* Update the plotting routine
* Update plotting
* Update plotting routines
2023-06-16 14:33:27 +01:00
Richard Stiskalek
255bec9710
Quijote kNN adding ( #62 )
...
* Fix small bug
* Add fiducial observers
* Rename 1D knn
* Add new bounds system
* rm whitespace
* Add boudns
* Add simname to paths
* Add fiducial obserevrs
* apply bounds only if not none
* Add TODO
* add simnames
* update script
* Fix distance bug
* update yaml
* Update file reading
* Update gitignore
* Add plots
* add check if empty list
* add func to obtaining cross
* Update nb
* Remove blank lines
* update ignroes
* loop over a few ics
* update gitignore
* add comments
2023-05-15 23:30:10 +01:00
Richard Stiskalek
f48eb6dcb0
Dump radial profile information ( #48 )
...
* add radial position path
* pep8
* Add basic fit profile dumping
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* Update TODO
* Fix parts is None bug
* Update nb
2023-04-27 00:18:30 +01:00
Richard Stiskalek
fdb0df8d4c
Add mmain and other major updates ( #44 )
...
* Move paths to a separate file
* Add mmain reader
* Add a verbosity flag
* Fix imports
* Fix bug
* Rename files
* Return ultimate parents
* Add script to generate mmain
* Remove mmain path
* edit path
* Add mmain path
* Change function name
* Rename function
* Turn off verbose
* Fix list requirement
* Edit init match paths
* Fix init pathing
* Edit paths docs
* Edit dumpdir name
* Rename path
* Fix split paths
* Remove unused import
* Add comment
* Update readme
* remove read mmain
* Rename haloatalogue
* Fix minor bugs
* Update nbs
* Add create directory option
* Move split jobs
* Move spliot jobs
* Remove splitting
* Add import
* Edit script
* Deeper split folder
* Fix paths bug
* Rename catalogue
* Rename Catalogue
* Add new clumpread
* Edit paths
* add knn paths
* Update commenting
* Update imports
* Add more conversions
* Update temp file
* Add a note
* Add catalogue
* Cooment
* Update TODO
* Update script
* add nb
* Update
* pep8
* edit paths & pep8
* Fix knn auto paths
* add paths docs
* Add auto and cross knn paths
* Add new paths
* Simplify tpcf reading
* pep8 patch
* update readme
* Update progress
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* Pep 8 and restructure
* add lambda spin
* add clump and halo
* add checks
* Edit halo profile fit
* Update gitignore
* backup script
2023-04-18 11:02:36 +02:00
Richard Stiskalek
e0d3854277
Add cleaner paths ( #43 )
...
* ignore venv
* ignore pylians if unavaible
* Edit loading paths
* Edit paths
* Stop having default paths
* Add nb
* Add glamdring paths
---------
Co-authored-by: Richard Stiskalek <dc-stis1@login8b.pri.cosma7.alces.network>
Co-authored-by: Richard Stiskalek <dc-stis1@login8a.pri.cosma7.alces.network>
2023-04-14 23:24:52 +01:00
Richard Stiskalek
5784011de0
kNN-CDF secondary halo bias ( #40 )
...
* Add seperate autoknn script & config file
* edit ics
* Edit submission script
* Add threshold values
* Edit batch sizign
* Remove print
* edit
* Rename files
* Rename
* Update nb
* edit runs
* Edit submit
* Add median threshold
* add new auto reader
* editt submit
* edit submit
* Edit submit
* Add mean prk
* Edit runs
* Remove correlation file
* Move split to clutering
* Add init
* Remove import
* Add the file
* Add correlation reading
* Edit scripts
* Add below and above median permutation for cross
* Update imports
* Move rvs_in_sphere
* Create utils
* Split
* Add import
* Add normalised marks
* Add import
* Edit readme
* Clean up submission file
* Stop tracking submit files
* Update gitignore
* Add poisson field analytical expression
* Add abstract generators
* Add generators
* Pass in the generator
* Add a check for if there are any files
* Start saving average density
* Update nb
* Update readme
* Update units
* Edit jobs
* Update submits
* Update reader
* Add random crossing
* Update crossing script
* Add crossing with random
* Update readme
* Update notebook
2023-04-09 20:57:05 +01:00
Richard Stiskalek
63ab3548b4
kNN-CDF implementation ( #34 )
...
* Rewrite doc
* add kNN
* edit loading of samples with no init
* Add verbosity flag
* add KNN submission script
* do not make peaked cdf by default
* Add submit script
* stop ignore sh
* Add mass thresholding
* Edit gitignore
* edits
* Space points in logspace
* Calculate for all ICs
* Update TODO
* Add dtype support
* Update readme
* Update nb
2023-03-31 18:13:41 +01:00
Richard Stiskalek
5dd8c668fa
Gaussian smoothing of density fields ( #33 )
...
* Simplify smoothing support and looping over nonzero
* Simplify comments
* add now()
* add cat length
* add smoothed calculation
* add smoothing
* Add sorting
* Edit what is ignored
* Move notebooks
* Add nonsymmetric smoothed overlap
* Update NB
* Add support for reading in the smoothed overlap
* Switch to the true overlap definition
* Reader of the true overlap
* rem occups
* Import moved to a class
* Move definition
* Edit submission script
* Update to account for the new definition
* backup nb
* Switch back to properly initialising arrays
* Fix addition bug
* Update NB
* Fix little bug
* Update nb
2023-03-27 09:22:03 +01:00
Richard Stiskalek
c3c686fa60
Power spectrum stuff ( #16 )
...
* add autok1d
* update TODO
* add basic cross pk
* del blank line
* fix small bugs
* fix loop bug
* rm nbs
* edit docstring
* Update TODO
* get rid of verbose flag
* take all ICs
* attempt better memory
* simplify ASCII dump
* rm verbose statement to get rid of bugs!
* close fortran files
* add MAS options
* add boxsize setter
* Move comment up
* paths chagne
* Update TODO
* Update TODO
* Forgotten units in Xpk
* Attempt at fixing the distance bug
* Update TODO
* Update TODO
* Remove comment
* add summary reader
* Update gitignore
2022-12-12 14:42:44 +00:00
Richard Stiskalek
1bb4255874
SDSS galaxies and catalogs rework ( #15 )
...
* add SDSS skeleton
* rm nb
* update gitignore
* add docs
* add survey volume
* add masking
* add SDSS mask
* add SDSS comments
* add reference
* rm
* move planck to fits clusters
* Rm comment
* update masking routines
* rm masks
* Update docs
* rm data
* make MCXC fits surv
* change MCXC name
* Move to extrasapce
* Update paths
2022-12-05 11:36:41 +00:00
Richard Stiskalek
6aa3721c83
Potential ( #13 )
...
* add basic density field
* Add TODO
* add field smoothing
* update how pos are calculated
* add transforms both ways
* add import
* add sky density
* add make skymap func
* update TODO
* update gitignore
* add potential field calculation
* delete boxsize setter
* add tidal tensor
2022-11-27 07:53:38 +00:00
Richard Stiskalek
161c27d995
Particle match & file system & phase space ( #11 )
...
* Create file system
* add doc
* add n_sim n_snap directly to paths
* Move things to a single particle reader for consistency
* add docstring
* add srdcir, dumpdir and mmain_path
* make boxunits work with paths
* switch to using paths
* add tempdumpdir
* rm dependence on old functions
* rm comment
* rm unused import
* go back to all imports
* fix import bug
* rm dependence on old functions
* modernize code!
* fix typo
* fix typo
* update fits to new data structureing
* change docs
* add julia repo
* add setup
* add install commands
* ignore install files
* add array flattening
* update dependene
* add positions reader
* update manifest and projects
* add func
* update gitignore
* pos matching progress
* move file
* rm comment
* add velocities getter
* fix bug
* fix name bug
* fix path bug
* fix args func
* add redshift calculation to catalogues
* add shortcut to set n_sim and n_snap
* if cond bug
* add the cosine similarity
* add verbosit iterator
* add docs
* update README
* update README
* update README
2022-11-25 10:44:08 +00:00
Richard Stiskalek
8a56c22813
Within halo work and NFW fit ( #4 )
...
* add listing of snapshots
* change distance to comoving
* ignore cp files
* rename nb
* add str to list
* add NFW profile shapes
* add fits imports
* Rename to Nsnap
* in clumps_read only select props
* make clumpid int
* expand doc
* add import
* edit readme
* distribute halos
* add profile & posterior
* add import
* add import
* add documentation
* add rvs and init guess
* update todo
* update nb
* add file
* return end index too
* change clump_ids format to int32
* skeleton of dump particle
* update nb
* add func to drop 0 clump indxs parts
* add import
* add halo dump
* switch to float32
* Update TODO
* update TODO
* add func that loads a split
* add halo object
* Rename to clump
* make post work with a clump
* add optimiser
* add Nsplits
* ignore submission scripts
* ignore .out
* add dumppath
* add job splitting
* add split halos script
* rename file
* renaem files
* rm file
* rename imports
* edit desc
* add pick clump
* add number of particles
* update TODO
* update todo
* add script
* add dumping
* change dumpdir structure
* change dumpdir
* add import
* Remove tqdm
* Increase the number of splits
* rm shuffle option
* Change to remove split
* add emojis
* fix part counts in splits
* change num of splits
* rm with particle cut
* keep splits
* fit only if 10 part and more
* add min distance
* rm warning about not set vels
* update TODO
* calculate rho0 too
* add results collection
* add import
* add func to combine splits
* update TODO
* add extract cols
* update nb
* update TODO
2022-10-30 20:16:56 +00:00
rstiskalek
b7b4353487
ignore settings
2022-10-20 08:13:09 +01:00
Richard Stiskalek
53a0629d90
Basic matching ( #2 )
...
* add recarray manipulations
* add cart to radec
* add behav so x can be a list
* add import
* create empty files
* ignore plots file
* add planck data
* add read_mmain file
* add cols_to_structured import
* use cols_to_structured
* add cols_to_structued
* add read_mmain import
* add reading planck
* add mass conversion
* add brute force separation calculation
* update nb
* rename & int dtype
* add func to get csiborg ids
* add list to nd array conversion
* add utils
* rename file
* add 2M++
* add read 2mpp
* add 2mpp shortcut
* add randoms generator
* Change range of RA [0, 360]
* fix ang wrapping
* add code for sphere 2pcf
* rm wrapping
* optionally load only a few borgs
* update nb
2022-10-18 19:41:20 +01:00
rstiskalek
edb9ee430c
ignore checkpoints
2022-10-11 13:47:26 +01:00
rstiskalek
8290e56ceb
ignore *.pyc
2022-10-10 17:41:29 +01:00
rstiskalek
54221a8ff5
first commit
2022-10-02 11:12:45 +01:00
