{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A tutorial to perform a cosmological analysis with BORG"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial is now available in the main ARES-HADES-BORG documentation in [Read the Docs](https://aquila-ares.readthedocs.io/en/latest/user/running.html#running-borg-a-tutorial-to-perform-a-cosmological-analysis)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Downloading and Installing BORG\n",
    "\n",
    "This note provides a step by step instruction for downloading and installing the BORG software package. This step-by-step instruction has been done using a MacBook Air running OS X El Capitan. I encourage readers to modify this description as may be required to install BORG on a different OS. Please indicate all necessary modifications and which OS was used.\n",
    "\n",
    "### Some Prerequisites\n",
    "cmake≥ 3.10\n",
    "automake\n",
    "libtool\n",
    "pkg-config\n",
    "gcc ≥ 7 , or intel compiler (≥ 2018), or Clang (≥ 7)\n",
    "wget (to download dependencies; the flag --use-predownload can be used to bypass this dependency if you already have downloaded the required files in the `downloads` directory)\n",
    "\n",
    "### Clone the repository from BitBucket\n",
    "To clone the ARES repository execute the following git command in a console: \n",
    "```\n",
    "git clone --recursive git@bitbucket.org:bayesian_lss_team/ares.git\n",
    "```\n",
    "\n",
    "After the clone is successful, you shall change directory to `ares`, and execute:\n",
    "```\n",
    "bash get-aquila-modules.sh --clone\n",
    "```\n",
    "\n",
    "Ensure that correct branches are setup for the submodules using:\n",
    "```\n",
    "bash get-aquila-modules.sh --branch-set\n",
    "```\n",
    "\n",
    "If you want to check the status of the currently checked out ARES and its modules, please run:\n",
    "```\n",
    "bash get-aquila-modules.sh --status\n",
    "```\n",
    "You should see the following output:\n",
    "```\n",
    "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n",
    "This script can be run only by Aquila members.\n",
    "if your bitbucket login is not accredited the next operations will fail.\n",
    "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n",
    "Checking GIT status...\n",
    "\n",
    "Root tree\t (branch master) : good. All clear.  \n",
    "Module ares_fg\t(branch master) : good. All clear.  \n",
    "Module borg\t(branch master) : good. All clear.  \n",
    "Module dm_sheet (branch master) : good. All clear.  \n",
    "Module hades\t(branch master) : good. All clear.  \n",
    "Module hmclet\t(branch master) : good. All clear.  \n",
    "Module python\t(branch master) : good. All clear.  \n",
    "```\n",
    "\n",
    "### Building BORG\n",
    "\n",
    "To save time and bandwidth it is advised to pre-download the dependencies that will be used as part of the building procedure. You can do that with\n",
    "```\n",
    "bash build.sh --download-deps\n",
    "```\n",
    "\n",
    "That will download a number of tar.gz which are put in the `downloads/` folder.\n",
    "\n",
    "Then you can configure the build itself:\n",
    "```\n",
    "bash build.sh --cmake CMAKE_BINARY --c-compiler YOUR_PREFERRED_C_COMPILER --cxx-compiler YOUR_PREFERRED_CXX_COMPILER --use-predownload\n",
    "```\n",
    "E.g. (This probably needs to be adjusted for your computer.):\n",
    "```\n",
    "bash build.sh --cmake /usr/local/Cellar/cmake/3.15.5/bin/cmake --c-compiler /usr/local/bin/gcc-9 --cxx-compiler /usr/local/bin/g++-9 --use-predownload\n",
    "```\n",
    "\n",
    "\n",
    "\n",
    "Once the configure is successful you should see a final output similar to this:\n",
    "```\n",
    "------------------------------------------------------------------\n",
    "\n",
    "Configuration done.\n",
    "Move to /Volumes/EXTERN/software/borg_fresh/ares/build and type 'make' now.\n",
    "Please check the configuration of your MPI C compiler. You may need\n",
    "to set an environment variable to use the proper compiler.\n",
    "\n",
    "Some example (for SH/BASH shells):\n",
    "- OpenMPI:\n",
    "    OMPI_CC=/usr/local/bin/gcc-9\n",
    "    OMPI_CXX=/usr/local/bin/g++-9\n",
    "    export OMPI_CC OMPI_CXX\n",
    "\n",
    "------------------------------------------------------------------\n",
    "```\n",
    "\n",
    "It tells you to move to the default build directory using `cd build`, after what you can type `make`. To speed up the compilation you can use more computing power by adding a `-j` option. For example\n",
    "```\n",
    "make -j4\n",
    "```\n",
    "will start 4 compilations at once (thus keep 4 cores busy all the time typically). Note, that the compilation can take some time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running a test example\n",
    "\n",
    "The ARES repository comes with some standard examples for LSS analysis. Here we will use a simple standard unit example for BORG. From your ARES base directory change to the examples folder:\n",
    "```\n",
    "cd examples\n",
    "```\n",
    "To start a BORG run just execute the following code in the console:\n",
    "```\n",
    "../build/src/hades3 INIT borg_unit_example.ini\n",
    "```\n",
    "BORG will now execute a simple MCMC. You can interupt calculation at any time. To resume the run you can just type:\n",
    "```\n",
    "../build/src/hades3 RESUME borg_unit_example.ini\n",
    "```\n",
    "You need at least on the order of 1000 samples to pass the initial warm-up phase of the sampler. As the execution of the code will consume about 2GB of your storage, we suggest to execute BORG in a directory with sufficient free hard disk storage."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analysing results\n",
    "Now we will look at the out puts generated by the BORG run. Note, that you do not have to wait for the run to complete, but you can already investigate intermediate results while BORG still runs. BORG results are stored in two major HDF5 files, the restart and the mcmc files. The restart files contain all information on the state of the Markov Chain required to resume the Markov Chain if it has been interrupted. The restart file also contains static information, that will not change during the run, such as the data, selection functions and masks and other settings. The mcmc files contain the current state of the Markov Chain. They are indexed by the current step in the chain, and contain the current sampled values of density fields, power-spectra, galaxy bias and cosmological parameters etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Opening files\n",
    "\n",
    "## The required python preamble"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Skipping VTK tools\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import h5py as h5\n",
    "import matplotlib.pyplot as plt\n",
    "import ares_tools as at\n",
    "%matplotlib inline\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now please indicate the path where you stored your BORG run:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "fdir='../testbed/'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Investigating the restart file\n",
    "\n",
    "The restart file can be opened by"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "hf=h5.File(fdir+'restart.h5_0')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The content of the file can be investigated by listing the keys of the 'scalar' section"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ARES_version',\n",
       " 'BORG_final_density',\n",
       " 'BORG_version',\n",
       " 'BORG_vobs',\n",
       " 'K_MAX',\n",
       " 'K_MIN',\n",
       " 'L0',\n",
       " 'L1',\n",
       " 'L2',\n",
       " 'MCMC_STEP',\n",
       " 'N0',\n",
       " 'N1',\n",
       " 'N2',\n",
       " 'N2_HC',\n",
       " 'N2real',\n",
       " 'NCAT',\n",
       " 'NFOREGROUNDS',\n",
       " 'NUM_MODES',\n",
       " 'Ndata0',\n",
       " 'Ndata1',\n",
       " 'Ndata2',\n",
       " 'adjust_mode_multiplier',\n",
       " 'ares_heat',\n",
       " 'bias_sampler_blocked',\n",
       " 'borg_a_final',\n",
       " 'borg_a_initial',\n",
       " 'catalog_foreground_coefficient_0',\n",
       " 'catalog_foreground_maps_0',\n",
       " 'corner0',\n",
       " 'corner1',\n",
       " 'corner2',\n",
       " 'cosmology',\n",
       " 'forcesampling',\n",
       " 'fourierLocalSize',\n",
       " 'fourierLocalSize1',\n",
       " 'galaxy_bias_0',\n",
       " 'galaxy_bias_ref_0',\n",
       " 'galaxy_data_0',\n",
       " 'galaxy_nmean_0',\n",
       " 'galaxy_sel_window_0',\n",
       " 'galaxy_selection_info_0',\n",
       " 'galaxy_selection_type_0',\n",
       " 'galaxy_synthetic_sel_window_0',\n",
       " 'gravity.do_rsd',\n",
       " 'growth_factor',\n",
       " 'hades_accept_count',\n",
       " 'hades_attempt_count',\n",
       " 'hades_mass',\n",
       " 'hades_sampler_blocked',\n",
       " 'hmc_Elh',\n",
       " 'hmc_Eprior',\n",
       " 'hmc_bad_sample',\n",
       " 'hmc_force_save_final',\n",
       " 'k_keys',\n",
       " 'k_modes',\n",
       " 'k_nmodes',\n",
       " 'key_counts',\n",
       " 'lightcone',\n",
       " 'localN0',\n",
       " 'localN1',\n",
       " 'localNdata0',\n",
       " 'localNdata1',\n",
       " 'localNdata2',\n",
       " 'localNdata3',\n",
       " 'localNdata4',\n",
       " 'localNdata5',\n",
       " 'momentum_field',\n",
       " 'nmean_sampler_blocked',\n",
       " 'part_factor',\n",
       " 'pm_nsteps',\n",
       " 'pm_start_z',\n",
       " 'powerspectrum',\n",
       " 'projection_model',\n",
       " 's_field',\n",
       " 's_hat_field',\n",
       " 'sigma8_sampler_blocked',\n",
       " 'startN0',\n",
       " 'startN1',\n",
       " 'supersampling',\n",
       " 'tCOLA',\n",
       " 'total_foreground_blocked']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(hf['scalars'].keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example the input galaxy data can be viewed by: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 27376 characters]\n",
      "text/plain": [
       "<Figure size 1152x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "data=np.array(hf['scalars/galaxy_data_0'])\n",
    "\n",
    "#Plot data\n",
    "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(16, 8))\n",
    "ax1.set_title('A Slice through the y-z plane of the data cube')\n",
    "im1=ax1.imshow(data[16,:,:])\n",
    "\n",
    "ax2.set_title('A Slice through the x-z plane of the data cube')\n",
    "im2=ax2.imshow(data[:,16,:])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Investigating MCMC files\n",
    "\n",
    "MCMC files are indexed by the sample number $i_{samp}$. Each file can be opened separately. Suppose we want to open the $10$th mcmc file, then:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "isamp=10 # sample number\n",
    "fname_mcmc=fdir+\"mcmc_\"+str(isamp)+\".h5\"\n",
    "hf=h5.File(fname_mcmc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspect the content of the mcmc files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['BORG_final_density',\n",
       " 'BORG_vobs',\n",
       " 'catalog_foreground_coefficient_0',\n",
       " 'cosmology',\n",
       " 'galaxy_bias_0',\n",
       " 'galaxy_nmean_0',\n",
       " 'hades_accept_count',\n",
       " 'hades_attempt_count',\n",
       " 'hmc_Elh',\n",
       " 'hmc_Eprior',\n",
       " 'powerspectrum',\n",
       " 's_field',\n",
       " 's_hat_field']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(hf['scalars'].keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting density fields\n",
    "\n",
    "We can for instance be interested in plotting inferred initial and final density samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "delta_in=np.array(hf['scalars/s_field'])\n",
    "delta_fi=np.array(hf['scalars/BORG_final_density'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 25568 characters]\n",
      "text/plain": [
       "<Figure size 1152x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(16, 8))\n",
    "ax1.set_title('initial density')\n",
    "im1=ax1.imshow(delta_in[16,:,:])\n",
    "\n",
    "ax2.set_title('final density')\n",
    "im2=ax2.imshow(delta_fi[16,:,:])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting the power-spectrum\n",
    "\n",
    "The ARES repository provides some routines to analyse the BORG runs. A particularly useful routine calculates the posterior power-spectra of inferred initial density fields."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 10352 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ss = at.analysis(fdir)  \n",
    "\n",
    "#Nbin is the number of modes used for the power-spectrum binning\n",
    "opts=dict(Nbins=32,range=(0,ss.kmodes.max()))\n",
    "\n",
    "#You can choose the sample numper\n",
    "isamp=10\n",
    "P=ss.compute_power_shat_spectrum(isamp, **opts)\n",
    "\n",
    "kmode = 0.5*(P[2][1:]+P[2][:-1])\n",
    "P_k   = P[0]\n",
    "\n",
    "plt.loglog(kmode,P_k)\n",
    "plt.xlabel('k [h/Mpc]')\n",
    "plt.xlabel(r'$P(k)$')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Monitoring power-spectrum warm-up phase\n",
    "\n",
    "Rather than looking just at individual posterior sample power-spectra we can follow the evolution of power-spectra across the chain. Suppose you want to monitor the first 100 samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 15212 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "Nsamp=100\n",
    "PPs=[]\n",
    "for isamp in np.arange(Nsamp):\n",
    "    PPs.append(ss.compute_power_shat_spectrum(isamp, **opts))\n",
    "\n",
    "\n",
    "#plot power-spectra    \n",
    "color_idx = np.linspace(0, 1, Nsamp)\n",
    "idx=0\n",
    "for PP in PPs:\n",
    "    plt.loglog(kmode,PP[0],alpha=0.5,color=plt.cm.cool(color_idx[idx]), lw=1)\n",
    "    idx=idx+1\n",
    "plt.xlim([min(kmode),max(kmode)])\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
