Preamble
--------

In everything that follows the paths are relative from
our directory ``//WDSENTINEL/share/CorpCDOs`` on Windows and
``/home/share/CorpCDOs`` on Linux.

Restarting the tasks server
---------------------------

Most of the operation below rely on the task server being
operational. In the unlikely event that it stops, it can be restarted
as follows::

  ssh debian
  sudo /etc/init.d/circusd stop
  sudo /etc/init.d/circusd start

Load data from intex and populate the database
----------------------------------------------

we need to fill up three tables:

* clo_universe
* cusip_universe
* et_collateral

We first create a directory called ``data/Indicative_%Y-%m-%d``.
From the spreadsheet, this is driven by the two shortcuts
``CTRL+Shift+Y`` and ``CTRL+Shift+G``.

* In Intex Options, under Miscellaneous, Change the export settings to
  "Tab delimited"
* First select the cusips of interest, do ``CTRL+Shift+Y`` and paste them
  into a new portfolio tab in intex. On the portfolio tab itself,
  right click, then Export to text, and save into ``data/Indicative_%Y-%m-%d``.
  The intex filename should be called ``Portfolio.txt`` or ``Portfolio(i).txt``
  for some ``i``.
* If the deal is not in our database (i.e. new 2.0 deal),
  "CTRL+Shift+Y" and "CTRL+Shift+G" will not work on the new deals.
  Manually type in the deal name in the above process and manually
  type in the CUSIPs in the process below when you get All Tranches
  info.
* Then in the Collat Detail tab, make sure Asset Detail is
  toggled. "Show Summary Statistics" is toggled OFF. Right click, then
  Export to text, All bonds and extract the resulting zip file into a
  folder named ``data/Collaterals_%Y-%m-%d``
* Last step, select the cusips again, do ``CTRL+SHIFT+G``, paste into a
  new portfolio tab. Click on All Tranches, Right Click, Export to
  text, and save the file into a folder named ``data/Indicative_%Y-%m-%d``.
  The file name should be called ``Portfolio_TrInfo.txt`` or
  ``Portfolio_TrInfo(i).txt`` for some ``i``.
* If "Load Indicative Data" does not work, make sure that CUSIP/ISIN columns are in the 
  "All Tranche" Export Format. Also make sure that Export setting for portfolio tab export is correct.

We're done exporting the data from intex, now we need to load it into
the database. To that end, click on the link ``Load Intex Indicative Data``
in the Excel spreadsheet.

Steps to build the model
------------------------

* Scenarios generation can be triggered by selecting the cusips in the
  excel spreadsheet and using the ``CTRL+Shift+H`` shortcut.
* To disable Reinvestment put "Y" under "DisableReinv" column before
  using ``CTRL+Shift+H``.
* htop to monitor server activities.
* This will trigger three scripts: first it will build the survival
  curves: these are saved into ``Scenarios/Portfolios_%Y_%m_%d``,
  second it will generate the loss and prepay distributions and save
  them into ``Scenarios/Intex curves_%Y-%m%d/csv``, and finally
  generate the sss files that we can load into intex in
  ``Scenarios/Intex curves_%Y-%m%d/sss``.
* You can look at the logs file in ``logs``
  to monitor the process.


Running the scenarios in Intex
------------------------------

* First make sure that Intex look for the sss files in the right
  directory. Go into Tools, Options, Data Access and change it
  appropriately. (e.g. ``Z:\CorpCDOs\Scenarios\Intex
  curves_2013-07-22\sss``)
* In Intex Options, under Miscellaneous, Change the export settings to
  "Tab delimited"
* Select the cusips of interest on the spreadsheet (make sure the
  corresponding sss files exist), do ``CTRL+Shift+I`` and paste them
  into a new portfolio tab in Intex... A new improvement of Intex is
  that the "SSS Forecast" column can not be pasted in directly, so
  paste it into a spreadsheet first and copy/paste column by column.
* Then go onto the Scenarios Tab, and click the cross next to Rate
  Update to make sure it reloads the Interest Rates (otherwise intex
  will run it with libor set at 0... not good)
* Then go onto the Cashflows tab and wait... Make sure format of Intex export
  is in the format required--> ET: CLO_layout. Once it's done, export the cashflows into
  ``Scenarios/Prices_%Y-%m%-%d``.
* Finally, on the server do::

    cd /home/share/CorpCDOs/code/R
    Rscript load_cf.R

  This will load the cashflows and compute the deltas, and save it in
  the database.
* Enjoy your work by typing ``CTRL+T`` on the spreadsheet and see how
  all the columns get filled. Yay!

Calibration
-----------

The model calibration is triggered via a cronjob at 8am. If something
wrong happened or we forgot to save the data, it's possible to
re-trigger it manually as follows::

  cd /home/share/CorpCDOs/code/R
  Rscript calibrate_tranches_MF.R %Y-%m-%d

Bloomberg Mapping
-----------------

If an asset is not priced, but has a valid CUSIP and a price is
observed in bloomberg, you can save the CUSIP to retrieve its price
from bloomberg using ET reloaded. Once the CUSIP is saved, run these
scripts to bring the prices into the database.

In windows - run R script ``\CorpCDOs\code\R\load_bloomberg_data.R``

On server::

  cd /CorpCDOs/code/python/
  python load_bloomberg_data.py

Install lossdistrib
-------------------

If Lossdistrib package is out of date, install in server by::

	cd /home/share/CorpCDOs/code/R
	R CMD INSTALL lossdistrib

Kick off daily tranche runs
---------------------------
::

   cd /home/share/CorpCDOs/code/R
   Rscript calibrate_tranches_BC.R -u -c runs.yml
   Rscript tranches_RV_BC.R -u -c RVruns.yml
   
Start Juypter
-------------
::

	cd /home/share/CorpCDOs/notebooks
	myipython

To install RPostgres
---------------------

To access serenitasdb, I need some R packages... These particular ones are from github ::

	devtools::install_github("rstats-db/DBI")
	devtools::install_github("thrasibule/RPostgres@convert_dates")