rstiskalek
2be20f763b
Add matching to no selection
2023-12-13 20:42:58 +00:00
Richard Stiskalek
506d7a7c70
Fix paths ( #95 )
...
* Fix args
* Fix paths
* Get rid of quijote backup
2023-12-13 18:03:27 +00:00
Richard Stiskalek
aaa14fc880
Add density field plot and start preparing CSiBORG2 ( #94 )
...
* Add RAMSES2HDF5 conversion
* Upload changes
* Clean up
* More clean up
* updates
* Little change
* pep9
* Add basic SPH calculation for a snapshot
* Add submit script
* Remove echo
* Little changes
* Send off changes
* Little formatting
* Little updates
* Add nthreads argument
* Upload chagnes
* Add nthreads arguemnts
* Some local changes..
* Update scripts
* Add submission script
* Update script
* Update params
* Rename CSiBORGBox to CSiBORG1box
* Rename CSiBORG1 reader
* Move files
* Rename folder again
* Add basic plotting here
* Add new skeletons
* Move def
* Update nbs
* Edit directories
* Rename files
* Add units to converted snapshots
* Fix empty dataset bug
* Delete file
* Edits to submission scripts
* Edit paths
* Update .gitignore
* Fix attrs
* Update weighting
* Fix RA/dec bug
* Add FORNAX cluster
* Little edit
* Remove boxes since will no longer need
* Move func back
* Edit to include sort by membership
* Edit paths
* Purge basic things
* Start removing
* Bring file back
* Scratch
* Update the rest
* Improve the entire file
* Remove old things
* Remove old
* Delete old things
* Fully updates
* Rename file
* Edit submit script
* Little things
* Add print statement
* Add here cols_to_structured
* Edit halo cat
* Remove old import
* Add comment
* Update paths manager
* Move file
* Remove file
* Add chains
2023-12-13 16:08:25 +00:00
Richard Stiskalek
944fea5510
Remove old merger tree ( #93 )
...
* Edit docs
* Delete merger tree files
* Edit README
* Edit docs
2023-12-07 14:45:06 +00:00
Richard Stiskalek
e972f8e3f2
Add pynbody and other support ( #92 )
...
* Simplify box units
* Move old scripts
* Add printing
* Update readers
* Disable boundscheck
* Add new ordering
* Clean up imports
* Enforce dtype and add mass to quijote
* Simplify print statements
* Fix little typos
* Fix key bug
* Bug fixing
* Delete boring comments
* Improve ultimate clumps for PHEW
* Delete boring comments
* Add basic reading
* Remove 0th index HID
* Add flipping of X and Z
* Updates to halo catalogues
* Add ordered caching
* Fix flipping
* Add new flags
* Fix PHEW empty clumps
* Stop over-wrriting
* Little improvements to angular neighbours
* Add catalogue masking
* Change if-else statements
* Cache only filtered data
* Add PHEW cats
* Add comments
* Sort imports
* Get Quijote workign
* Docs
* Add HMF calculation
* Move to old
* Fix angular
* Add great circle distance
* Update imports
* Update impotrts
* Update docs
* Remove unused import
* Fix a quick bug
* Update compatibility
* Rename files
* Renaming
* Improve compatiblity
* Rename snapsht
* Fix snapshot bug
* Update interface
* Finish updating interface
* Update all paths
* Add old scripts
* Add basic halo
* Update imports
* Improve snapshot processing
* Update ordering
* Fix how CM positions accessed
* Add merger paths
* Add imports
* Add merger reading
* Add making a merger tree
* Add a basic merger tree reader
* Add imports
* Add main branch walking + comments + debuggin
* Get tree running
* Add working merger tree walking along main branch
* Add units conversion for merger data
* Add hid_to_array_index
* Update merger tree
* Add mergertree mass to PHEWcat
* Edit comments
* Add this to track changes...
* Fix a little bug
* Add mergertree mass
* Add cache clearing
* Improve summing substructure code
* Littbe bug
* Little updates to the merger tree reader
* Update .giignore
* Add box selection
* Add optional deletingf of a group
* add to keep track of changes
* Update changes
* Remove
* Add manual tracker
* Fix bug
* Add m200c_to_r200c
* Add manual halo tracking
* Remove skipped snapshots
* update cosmo params to match csiborg
* remove old comments
* Add SDSSxALFALFA
* Fix bugs
* Rename
* Edit paths
* Updates
* Add comments
* Add comment
* Add hour conversion
* Add imports
* Add new observation class
* Add selection
* Add imports
* Fix small bug
* Add field copying for safety
* Add matching to survey without masking
* Add P(k) calculation
* Add nb
* Edit comment
* Move files
* Remove merger import
* Edit setup.yp
* Fix typo
* Edit import warnigns
* update nb
* Update README
* Update README
* Update README
* Add skeleton
* Add skeleton
2023-12-07 14:23:32 +00:00
Richard Stiskalek
5500fbd2b9
Matching paper plots ( #91 )
...
* Fix calculations of expected mass
* Add paper plots
* Edits to pltos
* Add overlap summary
* Add imports
* Add import
* Add binned stat
* Add fit
* Add more plots
* Add basic env
* Add histogram mode
* Edit expected mass
* Improve expected plots
* Clean up plot
* Improve separation plot
* Update plots
* Edit expected calculation
* Update plotting
* Update plots
* Update plots
* Update plots
* Add conc fraction
* Add halo maker sorting
* Renaming
* Add import
* Add NaN treatment
* add import
* Move cosine smi
* Update plots
* Move similarity
* Fix little bugs
* Shorten documentation
* Update plots
2023-10-17 12:11:15 +01:00
rstiskalek
136c552369
Update dumping
2023-09-05 14:55:31 +01:00
rstiskalek
a63b51bef1
Edit pos
2023-09-05 14:14:28 +01:00
rstiskalek
8f3220bff7
Update dumping
2023-09-05 10:36:15 +01:00
rstiskalek
f135a69b40
Little bug
2023-09-04 14:27:14 +01:00
Richard Stiskalek
9ae93bed14
Add dumping to ASCII of halos ( #89 )
...
* Rename file
* Add argument
* Add ASCII positions paths
* Add halo positions dumping
* Update halo catalogues
2023-09-03 12:11:39 +01:00
Richard Stiskalek
eccd8e3507
Add galaxy sampling ( #88 )
...
* Improve calculations
* Improve flags
* Add smoothed options
* Remove some old comments
* Edit little things
* Save smoothed
* Move files
* Edit imports
* Edit imports
* Renaming imports
* Renaming imports
* Sort imports
* Sort files
* Sorting
* Optionally make copies of the field
* Add quijote backup check
* Add direct field smoothing
* Shorten stupid documentation
* Shorten stupid docs
* Update conversion
* Add particles to ASCII conversion
* Add a short comment
* Add SDSS uncorrected distance
* Adjust comment
* Add FITS index to galaxies
* Remove spare space
* Remove a stupid line
* Remove blank line
* Make space separated
* Add interpolated field path
* Add field sampling
* Sort imports
* Return density in cells
* Clear out observer velocity
* Add 170817 sampling
* Fix normalization
* Update plot
2023-09-01 16:29:50 +01:00
Richard Stiskalek
0af925e26a
Minor code improvements ( #87 )
...
* Shorten docs
* Renames
2023-08-31 09:57:53 +01:00
Richard Stiskalek
ae92fd9b72
Velocity observer ( #86 )
...
* Continue if r200c not defined
* Remove smooth scale
* Remove smooth scale
* Edit Max Matching plot
* Add peculiar velocity
* Add Vobs calculation
* Edit docs
* Add Vobs plot
* Improve plotting
* Edit naming convention
* Make a note
* Add new cat options
* Update density field RSP calculation
* Update field 2 rsp
* Move functions and shorten documentation
* Improve transforms and comments
* Update docs
* Update imports
* Edit calculation
* Add docs
* Remove imports
* Add Quijote flags
* Edit documentation
* Shorten documentation
* Edit func calls
* Shorten
* Docs edits
* Edit docs
* Shorten docs
* Short docs edits
* Simplify docs a little bit
* Save plotting
* Update env
2023-08-30 23:27:20 +01:00
Richard Stiskalek
8e3127f4d9
New plots ( #85 )
...
* Update verbosity messages
* Update verbosity messags
* Update more verbosity flags
* Update the iterator settings
* Add basic plots
* Update verbosity flags
* Update arg parsre
* Update plots
* Remove some older code
* Fix some definitions
* Update plots
* Update plotting
* Update plots
* Add support functions
* Update nb
* Improve plots, move back to scripts
* Update plots
* pep8
* Add max overlap plot
* Add blank line
* Upload changes
* Update changes
* Add weighted stats
* Remove
* Add import
* Add Max's matching
* Edit submission
* Add paths to Max's matching
* Fix matching
* Edit submission
* Edit plot
* Add max overlap separation plot
* Add periodic distance
* Update overlap summaries
* Add nsim0 for Max matvhing
* Add Max's agreement plot
* Add Quijote for Max method
* Update ploitting
* Update name
2023-08-18 19:20:47 +01:00
rstiskalek
36f4177f6b
Add some blank spaces
2023-08-08 11:48:31 +01:00
rstiskalek
7ef36c7ac0
Add mass to name
2023-08-08 11:46:56 +01:00
Richard Stiskalek
c7b600d0ad
Periodic neighbours ( #84 )
...
* Edit the HMF plot
* Add periodic dist 2 points
* Add boxsize to RVSSphere
* Add periodic distance
* Adding periodic distance
* Add imports
* Change arguments
* Update bounds
* Lower min number of particles
* Change kwargs
* Add paths overlap quijote
* Add some comments
2023-08-08 12:19:40 +02:00
Richard Stiskalek
f4a7cb0d16
Remove halo fitting. ( #83 )
...
* Rename file
* Remove old content
* Remove halo fit
* Completely remove fits
* Add utils here
* Account for renaming
2023-08-07 11:33:27 +02:00
rstiskalek
46848a86eb
Edit docs
2023-07-31 20:23:56 +01:00
rstiskalek
f21abc28e7
Add lower and upper limits
2023-07-31 20:13:10 +01:00
Richard Stiskalek
344ff8e091
Fixing overlaps and halo definitions. ( #80 )
...
* Add imports
* Refactor code
* Rename fof velocities
* Clean up and add Quijote
* Edit docstrings
* Update submission script
* Fix bug
* Start loading fitted properties
* Edit docstrings
* Update fitting for new `halo`
* Update CM definition and R200c
* Tune the minimum number of particles
* Enforce crossing threshold & tune hypers
* Fix periodiity when calculating angmom
* Doc strings
* Relax checkip
* Minor edit
* Fix old kwarg bug
* Fix CSiBORG bounds
* Catch warnings!
* Add `mass_kind` and new boundaries
2023-07-31 16:13:21 +02:00
Richard Stiskalek
acb8d9571c
Update units to be consistent. ( #78 )
...
* Fix Quijote units
* Updates to units
* Fix how things are loaded
* Updating definitions & conventions
* Clear up how fiducial observers in quijote work
* Refactorize array manip
* Move function definition
* More code refactoring
* Remove unused argument
* Remove `convert_from_box`
* Make a note
* Converting particle units
* Add notes about units
* Remove box constants
* Add rho_crit0
* Fix spherical overdensity mass units
* Refactor more code
* Edit catalogue kwargs
* Edit the docstring
* Edit bounds
* Add new checks for empty array
* Remove unused import
* Remove old code
* Remove old function
* Update real 2 redshift
* Clear up the RSP conv
* Add comments
* Add some units
2023-07-28 21:07:28 +02:00
Richard Stiskalek
fb4b4edf19
Quijote snapshots support ( #77 )
...
* Renaming
* Edit docs
* Delete old function
* Add a blank space
* Rename particle reader
* Add comments
* Rename
* Rename
* edit get_snapshots
* More renaming
* Remove old correction
* Add import
* Add basics of the Quijote reader
* Add a blank space
* Fix paths
* Rename function
* Fix HID and path
* Add more FoF reading
* Move definition
* Adding arguments
* Renaming
* Add kwargs for backward comp
* FoF Quijote return only hids
* Add sorting of quijote
* Add path to CSiBORG ICs snapshot
* Add support for Quijote
* initmatch paths for quijote
* Add kwargs
* Fix blank lines
* Rename kwarg
* Remove unused import
* Remove hardcoded numbers
* Update for Quijote
* Do not store velocities in QUijote ICs
* Box units mass Quijote
* Fix typo
* Ensure particles are not right at the edge
* Add structfit paths for QUuijote
* Basic CSiBORG units
* Add more quijote halo reading
* Add Quijote fitting
* Docs changes
* Docs changes
2023-07-27 18:41:00 +02:00
Richard Stiskalek
e08c741fc8
Improving halo fits ( #76 )
...
* Add periodic distances
* Little corrections
* Fix little bug
* Modernise the script
* Small updates
* Remove clump
* Add new halo routines
* Fix weights
* Modernise the script
* Add check ups on convergence
* More convergence check ups
* Edit bounds
* Add default argument
* Update fit heuristic and NaNs
* Change maxiter
* Switch NFW minimization to log-sapce
* Remove print statement
* Turn convert_from_box abstract property required for all boxes.
* Move files
* Simplify script
* Improve the argument parser
* Remove optinal argument
* Improve argument parser
* Add a minimum concentration limit
2023-07-25 16:12:58 +02:00
Richard Stiskalek
eb8d070fff
CSiBORG FoF switch ( #75 )
...
* Add moving FoF membership files
* add FoF membership path
* Add notes where its PHEW
* Add FoF catalogue path
* Correct typo
* Add more functionalities
* Make work with halo IDs from FoF
* Edit print statement
* Fix copy bug
* copy
* Add FoF catalogue reading
* Clean up script
* Fix typo
* Little edits
* Fix naming convention
* Rename key
* Remove loading substructure particles
* Rename CSiBORG Cat
* Rename clumps cat
* Rename cat
* Remove misplaced import
* Switch to halos
* rm import
* structfit of only halos
* Add FoF halo reading
* Add a short comment
* Fix __getitem__ to work with int
* Fix problems
* Improve __getitem__
* Add more conversion
* Fix indexing
* Fix __getitem__ assertion
* Fix numbers
* Rename
* Fix verbosity flags
* Add full Quijote HMF option
* Add plot of Quijote only
* Add quijote full paths
* Fix the fit_init script
* Renam arg
* Update .gitignore
* add default argument name
* Change default verbosity flag
* Modernise script structure
* Fix dictionary
* Fix reading to include m200c
* Modernise script
* Add args
2023-07-24 14:10:21 +02:00
Richard Stiskalek
de7def61d5
New RSP density distinction ( #72 )
...
* Edit docs
* Fix kernel units
* Add NGP interpolation
* Add Vobs but not implemented
* Add BORG density option
* Add BORG mcmc path
* Organise imports
* Add new density field distinction
2023-06-26 20:41:07 +01:00
Richard Stiskalek
27e1c181a2
Moving environment to RS ( #70 )
...
* Drag several fields at once
* add env in RSP
* Add docstrings
* Make __main__
* Fix bug
* Fix plotting little bug
2023-06-18 11:42:21 +01:00
Richard Stiskalek
35ccfb5c67
New matches ( #69 )
...
* Remove old file
* Add velocity plotting
* add smooth scale
* Fix bug
* Improve paths
* Edit plotting
* Add smoothed density
* Update boundaries
* Add basics
* Further docs
* Remove blank
* Better catalog broadcasting
* Update high res size
* Update plotting routines
* Update routine
* Update plotting
* Fix field saving name
* Add better colormap for environemnt
2023-06-17 19:52:26 +01:00
Richard Stiskalek
cbfd1cbc99
Fixing tidal field calculation ( #68 )
...
* Add import
* Fix tidal calculation
* Add env to paths
* Improve plot routines
* Add env classification
2023-06-16 18:31:43 +01:00
Richard Stiskalek
ccbbbd24b4
Add better diagnostics & plotting ( #67 )
...
* Add caching functions
* Add limts
* Add new mass runs
* Update .gitignore
* Edit which CDFs are loaded
* Stop saving cross hindxs
* Change dist to half precision
* New nearest path
* Add neighbour counting
* Add neighbour kwargs
* Update work in progress
* Add new counting
* Add how dist is built
* Collect dist to 1 file
* Update reading routine
* Delete Quijote files
* Remove file
* Back to float32
* Fix bugs
* Rename utils
* Remove neighbuor kwargs
* Rename file
* Fix bug
* Rename plt utils
* Change where nghb kwargs from
* line length
* Remove old notebooks
* Move survey
* Add white space
* Update TODO
* Update CDF calculation
* Update temporarily plotting
* Merge branch 'add_diagnostics' of github.com:Richard-Sti/csiborgtools into add_diagnostics
* Start adding documentation to plotting
* Remove comments
* Better code documentation
* Some work on tidal tensor
* Better plotting
* Add comment
* Remove nb
* Remove comment
* Add documentation
* Update plotting
* Update submission
* Update KL vs KS plots
* Update the plotting routine
* Update plotting
* Update plotting routines
2023-06-16 14:33:27 +01:00
Richard Stiskalek
63b6cdbe72
Forward model fields to RSP ( #66 )
...
* Add radial velocity field
* Add overdensity plot
* Flip velocities too
* Add field calculations
* Add RSP mapping
* Add potential in RSP
* Add projected field plotting
2023-06-05 17:24:20 +02:00
Richard Stiskalek
dbf93b9416
Lagrangian patch + HMF calculation ( #65 )
...
* Rename lagpatch
* Fix old bug
* Fix small bug
* Add number of cells calculation
* Fix a small bug
* Rename column
* Move file
* Small changes
* Edit style
* Add plot script
* Add delta2ncells
* Add HMF calculation
* Move definition around
* Add HMF plot
* pep8
* Update HMF plotting routine
* Small edit
2023-06-01 14:45:52 +01:00
Richard Stiskalek
f1dbe6f03f
Add plotting ( #64 )
...
* Add verbosity statements
* More verbosity
* Save masses too
* Add CDF new plot
* Blank line
* Fix RVS sampling bug
* Add R200 conversion
* Simplify plotting routines
* Remove imoprt
2023-05-27 00:08:39 +01:00
Richard Stiskalek
2185846e90
CDF for nearest neighbour ( #63 )
...
* Updat ebounds
* fix mistake
* add plot script
* fix which sims
* Add Poisson
* Just docs
* Hide things to __main__
* Rename paths
* Move old script
* Remove radpos
* Paths renaming
* Paths renaming
* Remove trunk stuff
* Add import
* Add nearest neighbour search
* Add Quijote fiducial indices
* Add final snapshot matching
* Add fiducial observer selection
* add boxsizes
* Add reading functions
* Little stuff
* Bring back the fiducial observer
* Add arguments
* Add quijote paths
* Add notes
* Get this running
* Add yaml
* Remove Poisson stuff
* Get the 2PCF script running
* Add not finished htings
* Remove comment
* Verbosity only on 0th rank!
* Update plotting style
* Add nearest neighbour CDF
* Save radial distance too
* Add centres
* Add basic plotting
2023-05-21 22:46:28 +01:00
Richard Stiskalek
255bec9710
Quijote kNN adding ( #62 )
...
* Fix small bug
* Add fiducial observers
* Rename 1D knn
* Add new bounds system
* rm whitespace
* Add boudns
* Add simname to paths
* Add fiducial obserevrs
* apply bounds only if not none
* Add TODO
* add simnames
* update script
* Fix distance bug
* update yaml
* Update file reading
* Update gitignore
* Add plots
* add check if empty list
* add func to obtaining cross
* Update nb
* Remove blank lines
* update ignroes
* loop over a few ics
* update gitignore
* add comments
2023-05-15 23:30:10 +01:00
Richard Stiskalek
1d847cbd06
Add Quijote ( #61 )
...
* Rename paths object
* Remove redshift calculation
* Explicit keywrod arg
* Rename box units
* Basic renaming
* Little docs
* Rename paths
* add imports
* Sort imports
* Add Quijote cat
* Split boxes
* add Quijote path
* Add origin argument
* Update nbs
2023-05-13 17:37:34 +01:00
Richard Stiskalek
b3fd14b81f
Gravitational potential calculation ( #60 )
...
* Rename file
* add velocity field
* Add velcoity field
* Move smoothening
* Add verbosity flags
* remove blank
* Simplify paths
* Add potential calculation
* Update paths
* Add potential field
* Add potential calculation
* Move away sky matching
* Move interpolation functions
* Update nbs
2023-05-12 13:07:58 +01:00
rstiskalek
26ef1661a4
Add verbosity flag
2023-05-09 17:43:18 +01:00
Richard Stiskalek
b710b8e89c
Add new ICs ( #59 )
...
* edit IC paths
* Remove import
* Edit path
* Change naming
* Add __main__
* Script to match everything
* Edit docs
* Remove test statement
* Move import
* Update nb
2023-05-09 16:18:01 +01:00
rstiskalek
ab8199be2c
Rename folder
2023-05-08 14:24:15 +01:00
Richard Stiskalek
cb5d35b1be
Density field calculation ( #58 )
...
* Add iterative density field generation
* Edit particle halfwidth selection
* Update import
* Remove old file
* Add position wrapping
* Add RSD support
* Add density field calculation
* Edit paths to the density field
* Flip argument order
* Edit documentation
* Edit docs
* update nb
2023-05-08 14:02:20 +01:00
Richard Stiskalek
4265315b47
Revert "Run density field estimator ( #56 )" ( #57 )
...
This reverts commit 98d0578fa7
2023-05-08 13:59:55 +01:00
Richard Stiskalek
98d0578fa7
Run density field estimator ( #56 )
...
* Add iterative density field generation
* Edit particle halfwidth selection
* Update import
* Remove old file
* Add position wrapping
* Add RSD support
* Add density field calculation
* Edit paths to the density field
* Flip argument order
2023-05-08 13:58:12 +01:00
Richard Stiskalek
51b670d30b
Updates to the overlap reader. ( #54 )
...
* Flag for loading clumps_cat
* Optionally load clumps catalogues
* Update single pair reading
* Edit reading many pairs
2023-05-07 16:34:55 +01:00
Richard Stiskalek
56e39a8b1d
Overlapper improvements ( #53 )
...
* Store indices as f32
* Fix init sorting
* Organise imports
* Rename pathing
* Add particle loading
* Improve particle reading
* Add h5py reader
* edit particle path
* Update particles loading
* update particles loading
* Fix particle dumping
* Add init fitting
* Fix bug due to insufficient precision
* Add commnet
* Add comment
* Add clumps catalogue to halo cat
* Add comment
* Make sure PIDS never forced to float32
* fix pid reading
* fix pid reading
* Update matching to work with new arrays
* Stop using cubical sub boxes, turn off nshift if no smoothing
* Improve caching
* Move function definitions
* Simplify calculation
* Add import
* Small updates to the halo
* Simplify calculation
* Simplify looping calculation
* fix tonew
* Add initial data
* Add skip condition
* Add unit conversion
* Add loading background in batches
* Rename mmain index
* Switch overlaps to h5
* Add finite lagpatch check
* fix column name
* Add verbosity flags
* Save halo IDs instead.
* Switch back to npz
* Delte nbs
* Reduce size of the box
* Load correct bckg of halos being matched
* Remove verbosity
* verbosity edits
* Change lower thresholds
2023-05-06 16:52:48 +01:00
Richard Stiskalek
1a9e6177d7
Switch to h5py format ( #52 )
...
* Edit the particle paths
* Remove script
* Add h5py to dumping
* Minor adjustments
* add h5py support
* remove split path
* h5py support
* Type
* Edit initmatch paths
* Shorten func
* dist_centmass to work with 2D arrays
* Forgot to return the centre of mass
* Fixed code
* Fix halo bug
* Start MPI broadcasting
* Mini bug
* Remove commenting
* Remove test statement
* Fix index
* Printing from rank 0 only
* Move where clump index stored
* Add dtype options
* Add dtype options
2023-05-02 13:57:13 +01:00
Richard Stiskalek
553eec8228
Update field calculations ( #51 )
...
* Update density field routines
* Add utils
* Add possibility to return 2d array
* Styling
* Fixed density field
* Fix bugs
* add mpc2box
* Fix evaluating sky
* Fix sky map making
* Rename file
* Add paths if only positions
* Add option to dump particles only
* Add comments
2023-04-29 22:57:05 +01:00
Richard Stiskalek
b5fefe4196
Update init matching procedure ( #50 )
...
* pep8
* fix convention
* Update script
* enforce optimisation boundaries to be finite
* Update TODO
* Remove sky matching
* FIx a small bug
* fix bug
* Remove import
* Add halo fitted quantities
* Update nbs
* update README
* Add load_initial comments
* Rename nbs
* Delete nb
* Update imports
* Rename function
* Update matcher
* Add overlap paths
* Update the matching script
* Update verbosity
* Add verbosity flags
* Simplify make_bckg_delta
* bug fix
* fix bug
* lala
* la
* Add overlap paths
* Update limit
* pep8
* Some pep8 stuff
* pep8
* Minor corrections
* Update paths
* Add number of particles check
* Fix bug
* pep8 comments
2023-04-29 14:38:21 +01:00
Richard Stiskalek
c14be720b5
Particle dumping ( #49 )
...
* Add paths
* Add dumping script
* Remove snap from density fields
* Add progress statement
* Update import ordering
* Update nb
2023-04-28 13:57:23 +01:00
rstiskalek
b0752efdf4
fix indexing bug
2023-04-27 09:21:00 +01:00
Richard Stiskalek
f48eb6dcb0
Dump radial profile information ( #48 )
...
* add radial position path
* pep8
* Add basic fit profile dumping
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* Update TODO
* Fix parts is None bug
* Update nb
2023-04-27 00:18:30 +01:00
rstiskalek
1a115f481d
Rename file
2023-04-21 00:40:11 +01:00
Richard Stiskalek
04119a5314
Update initial matching & overlaps ( #47 )
...
* pep8
* fix convention
* Update script
* enforce optimisation boundaries to be finite
* Update TODO
* Remove sky matching
* FIx a small bug
* fix bug
* Remove import
* Add halo fitted quantities
* Update nbs
* update README
* Add load_initial comments
* Rename nbs
* Delete nb
* Update imports
* Rename function
* Update matcher
* Add overlap paths
* Update the matching script
* Update verbosity
* Add verbosity flags
* Simplify make_bckg_delta
* bug fix
* fix bug
2023-04-21 01:35:06 +02:00
Richard Stiskalek
39b3498621
Correct fitting script for both clumps and halos ( #46 )
...
* Minor typos
* fix minor bugs
* pep8
* formatting
* pep8
* Fix minor bugs
* New path & pep8
* add splitt
* Updates
* Improve calculation within radius
* pep8
* pep8
* get the script working
* Add matter overdensity
* Add m200m to the script
* Fix looping bug
* add parents support
* add import
* Optionally concatenate velocities
* Make optional masking
* Ignore the error message
* Start reading in raw data
* Fix cat reading
* Additional units conversions
* Add clump reading
* Fix indexing
* Remove old comment
* Remove old comment
* set npart to 0 instead of overflow from NaN
* fix docs
* rm boring stuff
* Remove old stuff
* Remove old stuff
* Remove old comment
* Update nb
2023-04-19 16:39:35 +02:00
Richard Stiskalek
fdb0df8d4c
Add mmain and other major updates ( #44 )
...
* Move paths to a separate file
* Add mmain reader
* Add a verbosity flag
* Fix imports
* Fix bug
* Rename files
* Return ultimate parents
* Add script to generate mmain
* Remove mmain path
* edit path
* Add mmain path
* Change function name
* Rename function
* Turn off verbose
* Fix list requirement
* Edit init match paths
* Fix init pathing
* Edit paths docs
* Edit dumpdir name
* Rename path
* Fix split paths
* Remove unused import
* Add comment
* Update readme
* remove read mmain
* Rename haloatalogue
* Fix minor bugs
* Update nbs
* Add create directory option
* Move split jobs
* Move spliot jobs
* Remove splitting
* Add import
* Edit script
* Deeper split folder
* Fix paths bug
* Rename catalogue
* Rename Catalogue
* Add new clumpread
* Edit paths
* add knn paths
* Update commenting
* Update imports
* Add more conversions
* Update temp file
* Add a note
* Add catalogue
* Cooment
* Update TODO
* Update script
* add nb
* Update
* pep8
* edit paths & pep8
* Fix knn auto paths
* add paths docs
* Add auto and cross knn paths
* Add new paths
* Simplify tpcf reading
* pep8 patch
* update readme
* Update progress
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* pep8
* Pep 8 and restructure
* add lambda spin
* add clump and halo
* add checks
* Edit halo profile fit
* Update gitignore
* backup script
2023-04-18 11:02:36 +02:00
Richard Stiskalek
e0d3854277
Add cleaner paths ( #43 )
...
* ignore venv
* ignore pylians if unavaible
* Edit loading paths
* Edit paths
* Stop having default paths
* Add nb
* Add glamdring paths
---------
Co-authored-by: Richard Stiskalek <dc-stis1@login8b.pri.cosma7.alces.network>
Co-authored-by: Richard Stiskalek <dc-stis1@login8a.pri.cosma7.alces.network>
2023-04-14 23:24:52 +01:00
Richard Stiskalek
1344fa40b6
Add 2PCF calculation ( #42 )
...
* Add 2PCF calculation
* Add 2PCF reader
* Add tpcf scripts
* Add random state
* Fix marked typo
* Fix marks and more randoms
* Stop calculating projected
* Add mean 2PCF
* Remove pimax
* Edit submit script
* Update nb
* Add corrfunc check
2023-04-14 23:05:48 +01:00
Richard Stiskalek
5784011de0
kNN-CDF secondary halo bias ( #40 )
...
* Add seperate autoknn script & config file
* edit ics
* Edit submission script
* Add threshold values
* Edit batch sizign
* Remove print
* edit
* Rename files
* Rename
* Update nb
* edit runs
* Edit submit
* Add median threshold
* add new auto reader
* editt submit
* edit submit
* Edit submit
* Add mean prk
* Edit runs
* Remove correlation file
* Move split to clutering
* Add init
* Remove import
* Add the file
* Add correlation reading
* Edit scripts
* Add below and above median permutation for cross
* Update imports
* Move rvs_in_sphere
* Create utils
* Split
* Add import
* Add normalised marks
* Add import
* Edit readme
* Clean up submission file
* Stop tracking submit files
* Update gitignore
* Add poisson field analytical expression
* Add abstract generators
* Add generators
* Pass in the generator
* Add a check for if there are any files
* Start saving average density
* Update nb
* Update readme
* Update units
* Edit jobs
* Update submits
* Update reader
* Add random crossing
* Update crossing script
* Add crossing with random
* Update readme
* Update notebook
2023-04-09 20:57:05 +01:00
Richard Stiskalek
8d34b832af
Improve paths and documentation ( #38 )
...
* pep8
* style issue
* Documentation edits
* Remove old files
* Remove more old files
* Doc & stop setting snap and nsim in paths
* add nsnap, nsim support
* Remove blank space
* docs
* Docs edits
* Reduce redudant code
* docs
* Documentation
* Docs
* Simplify
* add nsnap nsim arguments
* Remove redundant docs
* Fix typos
* Shorten catalogue
* Remove import
* Remove blank line
* Docs only edits
* Rearrange imports
* Remove blank space
* Rearrange imports
* Rearrange imports
* Remove blank space
* Update submission scritp
* Edit docs
* Remove blank line
* new paths
* Update submission scripts
* Update file handling
* Remove blank line
* Move things around
* Ne paths
* Edit submission script
* edit paths
* Fix typo
* Fix bug
* Remove import
* Update nb
2023-04-04 15:22:00 +01:00
Richard Stiskalek
522ee709c9
Joint kNN-CDF calculation ( #36 )
...
* Add joint kNN CDF
* add jointKNN calculation
* change sub script
* Update readme
* update sub
* Small changes
* comments
* update nb
* Update submisison script
2023-04-01 11:16:11 +01:00
rstiskalek
8d933e7ba9
Add batch size to submission
2023-04-01 07:15:44 +01:00
Richard Stiskalek
63ab3548b4
kNN-CDF implementation ( #34 )
...
* Rewrite doc
* add kNN
* edit loading of samples with no init
* Add verbosity flag
* add KNN submission script
* do not make peaked cdf by default
* Add submit script
* stop ignore sh
* Add mass thresholding
* Edit gitignore
* edits
* Space points in logspace
* Calculate for all ICs
* Update TODO
* Add dtype support
* Update readme
* Update nb
2023-03-31 18:13:41 +01:00
rstiskalek
4d7827006a
test
2023-03-27 13:38:01 +01:00
Richard Stiskalek
5dd8c668fa
Gaussian smoothing of density fields ( #33 )
...
* Simplify smoothing support and looping over nonzero
* Simplify comments
* add now()
* add cat length
* add smoothed calculation
* add smoothing
* Add sorting
* Edit what is ignored
* Move notebooks
* Add nonsymmetric smoothed overlap
* Update NB
* Add support for reading in the smoothed overlap
* Switch to the true overlap definition
* Reader of the true overlap
* rem occups
* Import moved to a class
* Move definition
* Edit submission script
* Update to account for the new definition
* backup nb
* Switch back to properly initialising arrays
* Fix addition bug
* Update NB
* Fix little bug
* Update nb
2023-03-27 09:22:03 +01:00
Richard Stiskalek
153f1c0002
Overlap reader thresholds ( #31 )
...
* Improve data
* Add comment
* Update how KNN is called
* Bring back indices
* New function output
* return catx["index"]
* Remove unnecessary arguments
* Remove useless arguments
* Rename output
* thin up catalogues
* Add thresholding
* Update README
2023-03-15 20:55:50 +00:00
Richard Stiskalek
f2cef73977
Final overlap definition update.. ( #29 )
...
* Delete whitespace
* Update overlap definition
* Add documentation
* add overlap args
* Make the background array store only high density region
* Add nsims to args
* Remove conc as done inside the func
* Simplify the overlap calculation
* Rename variable
* Just some docs
* Docs
* Correct overlap definition
* Undo debug comment
* Remove old debug code
* Add prob of no match and exp couunterpart mass
* docs
* Update the overlap definition
2023-03-09 14:55:38 +00:00
Richard Stiskalek
1c859dbdac
Overlap reader ( #28 )
...
* Edit output path
* remove matching of many sims
* Add a class skeleton
* Add a setter for only nsim
* Deletei mport
* Simplify cross and delete combined halo catalogue
* Simplify reading in catalogues
* Add distance calculation
* Add mass ratio
* Clean up the submission script
* Add the summer overlap
* Docs
* Add array copying
* Add import
* Add resampling mean
2023-02-08 12:05:32 +00:00
Richard Stiskalek
8dea3da4de
Speed up overlap ( #27 )
...
* Edit improt
* Simplify patch size calculation
* Add patch size percentiles
* Add various percentiles
* Remove comment
* Update TODO
* Change to 95th percentile
* Add import
* Add KNN properties
* Add new matching initial condition
* Add import
* Remove import
* Add fast neighbours option
* Further edits to fast neighbours
* add imports
* add new overlap calculation and non-zero things
* Remove print
* Clean up code
* Fix small bug
* Remove comment
* Add run single cross match
* change values
* Edit hyperparams
* Add comment
* Add the argument parser
* Add new lagpatch calc
* New lagpatch calc
* Delete old patch definitions
* Make clump dumping once again optional
* Add lagpatch to the catalogue
* Edit print statement
* Fix small bug
* Remove init radius
* Change to lagpatch key
* Fix a small bug
* Fix little bug
2023-02-05 11:46:19 +00:00
Richard Stiskalek
beb811e84c
Switch initial radius definition ( #26 )
...
* Add search for neighbours at z = 0
* Add initial snapshot KNN
* Add initi search either z =0 or z = 70
* Add import
* Add clumps_pos2cell
* Add function argument
* Add import
* Add spherical overlap and speed up make_delta
* Add clump limits calculation
* Sped up make_delta
* Add patch size conversion
* Add patch sizes
* Add patch size calculation
* Force catalogues to be in float32
* Optimised script
* Option to remove points with no overlap
* Add spherical aproximate overlap
* Fix subboxing bug
* Remove print diagnostics
* Add Lagrangian patch size
* Add patch size documentation
* Move when clumpsx converted to int
* Edit docs
* Remove spherical overlap
* New Langrangian patch calculation
2023-01-31 15:09:35 +00:00
rstiskalek
88a109157d
Change file naming convention
2023-01-17 11:46:07 +00:00
Richard Stiskalek
f0720243a2
Updates to the submission script ( #22 )
2023-01-01 21:43:52 +00:00
Richard Stiskalek
2e99b901ac
Environmental properties ( #20 )
...
* rm get_positions
* Add comment
* add halfwidth func
* Update docs
* Add imprt
* Evaluate multiple fields simulatenously
* add halfwidth selection
* Change order of grav field and tensor field
* Add gravitational field norm
* Add eigenvalue calculation
* Sorted eigenvalues
* add init script
* add progress
* Add surveys
* Add more survey flexibility
* Minor changes
* add survey names
* rm name
* Fix list bug
* Fig bugs when running the script
* add phi to dtype
* fix dump bug
* Add comment
* Add smoothing options
* Add further comment
* Update TODO
2022-12-31 17:46:05 +00:00
Richard Stiskalek
3dc559fec1
Overlap calculation ( #19 )
...
* Rm comment
* Add new fixed particle overlap
* Fix overlap calculation
* Update docs
* Add new overlapper support in matcher
* add filter optin
* Add the real space filter.
* Change filtering to real space only
* Update TODO
* add high-resolution switch
* add cross script
* Remove comment
* Add smoothing option
2022-12-27 18:32:12 +01:00
Richard Stiskalek
18f09767f4
Overlap calculation ( #18 )
...
* Move cosine similarity out
* Basic overlap calculation
* add overlap import
* Add clump0 dict path
* Update README
* fix path bug
* Save as dict instead
* Change format to array of arrays
* Update paths
* Take fewer ICs for now
* Change to structured array
* Add overlap calculation
* Start saving IDs
* Add a blank space
* Update TODO
2022-12-19 11:58:22 +01:00
Richard Stiskalek
13a9d11afe
Particle init CM separation ( #17 )
...
* Rename
* rename module
* add new_ic_ids
* Add to_new support
* rm unnecessary comment
* Remove JUlia!
* add new package attempt
* add dump script
* simplify positions getter
* Remove unused code
* change dumpdirs
* update project
* Add sorting
* fix indexing bug
* add comment
* Add particle dumping
* Delete blank line
* Delete Julia things
* Update README
* add working script
* fix assignment bug
* fix bug
* save temp to temp folder
* fix indexing bug & path
* do not always crreate folder
* Rename file
* add initcm reader
* add x0, y0, z0 transforms
* add initcm to halo cat
* docs update
* add initial KNN
* add attempt at init match
* add initial fiducial radius
* rm blank space
* Add further KNN Init pos support
* Add init radius support
* Remove message about the bug
* Get rid of an error msg
* purge mention of knn_init
* Add init CM distance calc
2022-12-16 11:18:16 +01:00
Richard Stiskalek
c3c686fa60
Power spectrum stuff ( #16 )
...
* add autok1d
* update TODO
* add basic cross pk
* del blank line
* fix small bugs
* fix loop bug
* rm nbs
* edit docstring
* Update TODO
* get rid of verbose flag
* take all ICs
* attempt better memory
* simplify ASCII dump
* rm verbose statement to get rid of bugs!
* close fortran files
* add MAS options
* add boxsize setter
* Move comment up
* paths chagne
* Update TODO
* Update TODO
* Forgotten units in Xpk
* Attempt at fixing the distance bug
* Update TODO
* Update TODO
* Remove comment
* add summary reader
* Update gitignore
2022-12-12 14:42:44 +00:00
Richard Stiskalek
1bb4255874
SDSS galaxies and catalogs rework ( #15 )
...
* add SDSS skeleton
* rm nb
* update gitignore
* add docs
* add survey volume
* add masking
* add SDSS mask
* add SDSS comments
* add reference
* rm
* move planck to fits clusters
* Rm comment
* update masking routines
* rm masks
* Update docs
* rm data
* make MCXC fits surv
* change MCXC name
* Move to extrasapce
* Update paths
2022-12-05 11:36:41 +00:00
Richard Stiskalek
91beb4df50
Angular momentum ( #14 )
...
* add ang mom
* Update TODO
* add ascii dump script
* add comment
* Updating TODO
* add angmom
* add garbage collector
2022-11-30 16:12:39 +00:00
rstiskalek
4e87a96b97
fix bug(again)
2022-11-25 12:42:49 +00:00
rstiskalek
a3c7d03ceb
fix bug
2022-11-25 12:28:41 +00:00
rstiskalek
377d6ebb7e
add lambda200c support
2022-11-25 12:24:06 +00:00
Richard Stiskalek
161c27d995
Particle match & file system & phase space ( #11 )
...
* Create file system
* add doc
* add n_sim n_snap directly to paths
* Move things to a single particle reader for consistency
* add docstring
* add srdcir, dumpdir and mmain_path
* make boxunits work with paths
* switch to using paths
* add tempdumpdir
* rm dependence on old functions
* rm comment
* rm unused import
* go back to all imports
* fix import bug
* rm dependence on old functions
* modernize code!
* fix typo
* fix typo
* update fits to new data structureing
* change docs
* add julia repo
* add setup
* add install commands
* ignore install files
* add array flattening
* update dependene
* add positions reader
* update manifest and projects
* add func
* update gitignore
* pos matching progress
* move file
* rm comment
* add velocities getter
* fix bug
* fix name bug
* fix path bug
* fix args func
* add redshift calculation to catalogues
* add shortcut to set n_sim and n_snap
* if cond bug
* add the cosine similarity
* add verbosit iterator
* add docs
* update README
* update README
* update README
2022-11-25 10:44:08 +00:00
Richard Stiskalek
c748c87e45
New filesystem, basic neighbours calculation ( #10 )
...
* simplify Planck catalogue
* add MCXC and base survey
* Add 2MPP classes
* move match to MCXC to Planck
* add halo catalogue
* rm comment
* rm unused imports
* Move conversion to box
* add min mass
* Run on all simulations
* rm old function
* add combined catalogue
* add halo positions
* add knn neighbors
* set to 5 sims for testing
* add docstring
* Switch to neighbours in a catalogue
* rename io to read
* fix indentation
* rename to read
* io -> read
* add import
* add RealisationMatcher
* io -> read
* add docstrings
* add search_sim_indiices
* update todo
* keep make_cat at 10 for now
* update nb
2022-11-22 10:00:01 +00:00
rstiskalek
f5bf04c46e
update nb
2022-11-20 14:11:35 +00:00
rstiskalek
3352b06c1c
add units
2022-11-20 14:11:25 +00:00
rstiskalek
1d9789c08e
add vx
2022-11-20 14:11:19 +00:00
rstiskalek
06d3fede95
add max snapshot
2022-11-14 16:24:53 +00:00
rstiskalek
76349bab3f
add MPI and do for all boxes
2022-11-14 15:52:52 +00:00
Richard Stiskalek
17badf2652
X-ray data ( #9 )
...
* add xray reading
* add import
* add more comments
* add data
* add nb
* add MCXC shortcut
* add match indxs to Planck of MCXC
* add import
* update TODO
* update readme
* add famous clusters
* add 2mpp_groups
* shorten paths
* add 2M++ groups
* add import
* Update TODO
2022-11-10 07:40:44 +00:00
rstiskalek
d69012424e
simplify loading
2022-11-06 20:55:49 +00:00
rstiskalek
9291414572
update nbs
2022-11-06 20:53:58 +00:00
Richard Stiskalek
d32eb5c134
Data loading shortcut ( #7 )
...
* add load_processed
* update TODO
2022-11-06 10:26:24 +00:00
Richard Stiskalek
7f58b1f78c
JAX and fix conc ( #6 )
...
* change to log10 initlogRs
* add uncertainty
* add check if hessian negative
* update TODO
* update TODO
* output the error too
* save e_logRs
* add concentration calculation
* calcul concentration
* missed comma in a list
* Update TODO
* fix bug
* add box units and pretty status
* make uncertainty optional
* add BIC function
* remove BIC again
* add new overdensity calculation
* change defualt values to max dist and mass
* change to r200
* new halo find
* speed up the calculation
* add commented fucn
* update TODO
* add check whether too close to outside boundary
* Update TODO
* extract velocity as well
* calculate m200 and m500
* update nb
* update TODO
2022-11-05 21:17:05 +00:00
Richard Stiskalek
ba98ecc299
Mass enclosed in a sphere calculation ( #5 )
...
* rename nb
* add merge func
* rm bug..
* add static methods
* save rmin, rmax
* add properties for box units
* move radial info to the clump
* add Npart explicit
* move where rmin, rmax obtained
* add box cosmo
* add __getattr__
* add comments in units
* add enclosed spherical mass
* rm unused variables
* add clump mass setter
* add the halo index
* add enclosed overdensity
* add enclosed_overdensity
* simplify loss func
* opt result to nan
* change Rs to log10
* change back to attribs
* add H0 and h props
* add setattrs
* Remove global constants
* move Msuncgs above
* add crit density
* add dots
* add r200c and r500c
* add M200 and M500
* make lowercase
* output r200, r500, m200, m500
* update TODO
* update README
* fit NFW only up to r200
* add r178, m178
* add m178, r178
* update TODO
2022-11-01 10:10:54 +00:00
Richard Stiskalek
8a56c22813
Within halo work and NFW fit ( #4 )
...
* add listing of snapshots
* change distance to comoving
* ignore cp files
* rename nb
* add str to list
* add NFW profile shapes
* add fits imports
* Rename to Nsnap
* in clumps_read only select props
* make clumpid int
* expand doc
* add import
* edit readme
* distribute halos
* add profile & posterior
* add import
* add import
* add documentation
* add rvs and init guess
* update todo
* update nb
* add file
* return end index too
* change clump_ids format to int32
* skeleton of dump particle
* update nb
* add func to drop 0 clump indxs parts
* add import
* add halo dump
* switch to float32
* Update TODO
* update TODO
* add func that loads a split
* add halo object
* Rename to clump
* make post work with a clump
* add optimiser
* add Nsplits
* ignore submission scripts
* ignore .out
* add dumppath
* add job splitting
* add split halos script
* rename file
* renaem files
* rm file
* rename imports
* edit desc
* add pick clump
* add number of particles
* update TODO
* update todo
* add script
* add dumping
* change dumpdir structure
* change dumpdir
* add import
* Remove tqdm
* Increase the number of splits
* rm shuffle option
* Change to remove split
* add emojis
* fix part counts in splits
* change num of splits
* rm with particle cut
* keep splits
* fit only if 10 part and more
* add min distance
* rm warning about not set vels
* update TODO
* calculate rho0 too
* add results collection
* add import
* add func to combine splits
* update TODO
* add extract cols
* update nb
* update TODO
2022-10-30 20:16:56 +00:00
rstiskalek
a47847cb7c
more renaming
2022-10-20 23:37:33 +01:00
Richard Stiskalek
942c36b142
Box units conversion ( #3 )
...
* linting
* fix long line
* rename nb
* rename import
* add group catalog
* move imports out of functions
* add array_to_structured
* add references
* fix subsampling
* fix coord bug
* add 2M++ dists
* save nb
* fix comment
* add snapshot path
* add snapshot path
* add read_info
* Move transforms
* add import radec
* expand docs
* Move flipcols
* update nb
* add flip_cols
* create file
* add blank line
* Move units transfs
* add blank line
* add units import
* rm imports
* add import
* add box_units
* add comments
2022-10-20 23:28:44 +01:00
Richard Stiskalek
53a0629d90
Basic matching ( #2 )
...
* add recarray manipulations
* add cart to radec
* add behav so x can be a list
* add import
* create empty files
* ignore plots file
* add planck data
* add read_mmain file
* add cols_to_structured import
* use cols_to_structured
* add cols_to_structued
* add read_mmain import
* add reading planck
* add mass conversion
* add brute force separation calculation
* update nb
* rename & int dtype
* add func to get csiborg ids
* add list to nd array conversion
* add utils
* rename file
* add 2M++
* add read 2mpp
* add 2mpp shortcut
* add randoms generator
* Change range of RA [0, 360]
* fix ang wrapping
* add code for sphere 2pcf
* rm wrapping
* optionally load only a few borgs
* update nb
2022-10-18 19:41:20 +01:00
rstiskalek
4d24895d2f
update nb
2022-10-11 16:35:21 +01:00
