Preamble
--------

In everything that follows the paths are relative from
our directory ``//WDSENTINEL/share/CorpCDOs``.

Load data from intex and populate the database
----------------------------------------------

we need to fill up three tables:
- cusip_universe table
- et_collateral
- cusip_universe

This is driven by the ``clo_universe.xslx`` file in the data directory.
It contains a clean deals tab. These deals need to be pasted as a
portfolio in intex. I use a custom layout in the portfolio to export
the column we need.

First thing is to export the data as a text file from the portfolio
tab and save it as ``clo_universe_intex_%Y-%m-%d.txt`` in the data
directory. It can be loaded in the clo_universe table by using the
python script: ``code/python/clo_universe.py``. This script needs to
be edited so that it loads the correct date.

Next, we go the Collat Detail tab in Collateral, make sure Asset detail
is selected, and export all bonds to a text file. It probably needs to
be cut in a few chunks. We extract the zip file generated by intex in
a folder inside ``./`` named ``Collaterals_%Y-%m-%d``. It should be the same
date as for the ``clo_universe_intex_%Y-%m-%d.txt`` file.

Next, we have to load all the cusip information into the
``cusip_universe`` table. To get the list of cusips from the
``clo_universe`` table, we can use the query: ``SELECT UNNEST("Deal
Cusip List") FROM latest_clo_universe;``. The R function
``cusipsfromdealnames`` in ``intex_deal_functions.R`` is a convenient
wrapper around it. The resulting list of cusips needs to be set up as a
new portfolio in intex, and the All Tranches tab exported as text
files. They are saved in the ``Trinfo_%Y-%m-%d`` folder. The script to
load them in the database is called ``cusip_universe.py``.

Steps to build the model
------------------------

- first we need to calibrate the implied factor distribution. This is
  done by using ``calibrate_tranches.R`` inside ``code/R``. The
  spreads data is generated from the ``HY indices.xlsx`` spreadsheet
  in the ``NewSpreadsheet`` directory.
- then for each deal in the universe, we build the portfolio data, and
  scenarios using ``code/R/build_SC.R``