Potentials - November/December 2017 - 12

Two-Regime Plot of S&P 500 Series

SP500 Total Return Index

4,000
3,500
3,000
2,500
2,000
1,500

SP500
Normal
Crash

1,000
500
0

1990

1995

2000

Year

2005

2010

2015

FIG5 Employing trend filtering to discover normal and crash regimes in the S&P 500
Index: 1988-2016 (blue = normal, red = contraction).

Historical Returns for Two Regimes

20
10

(%)

0
-10
-20

U.S. Government
Bond

International
Equity-Emerging

International
Equity-Developed

U.S. Equities

Real Assets

Hedge Fund

Real Estate

Private Equity

-40

Future directions
and challenges

Growth Regime
Contraction Regime

-30

FIG6 Assets category performance is dependent upon the underlying economic regime.
(The graph shows performance under two regimes: normal and crash, 1988-2016).

decisions over a planning horizon
that maximizes an expected discounted objective function. The RL
models fit squarely in the realm of
decision models.
Markov decision models face severe
limitations for financial planning. A
critical barrier is the size of the state
space; the curse of dimensionality
limits real-world applications (Fig. 7).
Approximate dynamic programming
approaches have expanded the range
of solvable problems. A second barrier
entails estimating the environmental
regimes (states) and the transition
matrix across time periods. As discussed previously, we might depict a

12	

■	

N o v e mber /D ecember 2017 	

matrix. Traditionally, analysts employ
econometric and related time series
methods, such as maximum likelihood
and method of moments.
Given the dynamic nature of economic markets over short- to midterm periods, we might be interested
in estimating the Markov transition
matrix by focusing on recent market
performance. Modern reinforcement
learning methods address this issue
through statistical sampling concepts. The parameters of the transition matrix are identified in a -dynamic
(online) fashion as the process evolves
and the algorithm "learns." The sampling procedures require less intervention on the part of the developer.
In theory, by observing the data and
conducting an extensive training exercise, we can estimate the necessary
parameters for solving the discounted
Markov decision model in an online
fashion. An extension of RL employs
deep neural network algorithms to assist with the estimation process. The
combination, called deep RL, has been
employed successfully.

trading market as a set of economic
regimes-normal and crash. Given
these two regimes, we can generate
scenarios over a specified plann-
ing horizon.
The first assumption in RL is the
famous Markov property-path independence: period-t transition will be
dictated solely by the current state at
step/time t and the transition probability matrix. This assumption is identical to independent flips of a fair coin.
Even if the past eight flips have been
heads, there is still a 50% chance of
heads for the next flip of a fair coin.
An important issue involves estimating the parameters in the transition

IEEE POTENTIALS

The global economy is fast shifting to
a data-driven environment with disruptive impacts on numerous industries. Financial services have been
largely spared to date; however, several developments threaten traditional financial firms. The widespread
adoption of automated financial
planning systems, the so-called roboadvisors, will put pressure on human
advisors to lower their fees and provide greater services, even for individuals with modest means. Current
robo-systems are barely noticed in
terms of assets under management,
but there can be exponential growth
of new technologies with scale.
Another potentially disruptive area
involves personal credit and insurance matters through automated
platforms rather than via humans
at traditional banks and insurance
companies. Data-driven firms can
design ML algorithms that, in theory,
are superior to credit evaluation carried out by humans with limited data.
Likewise, there will be improvements



Table of Contents for the Digital Edition of Potentials - November/December 2017

Potentials - November/December 2017 - Cover1
Potentials - November/December 2017 - Cover2
Potentials - November/December 2017 - 1
Potentials - November/December 2017 - 2
Potentials - November/December 2017 - 3
Potentials - November/December 2017 - 4
Potentials - November/December 2017 - 5
Potentials - November/December 2017 - 6
Potentials - November/December 2017 - 7
Potentials - November/December 2017 - 8
Potentials - November/December 2017 - 9
Potentials - November/December 2017 - 10
Potentials - November/December 2017 - 11
Potentials - November/December 2017 - 12
Potentials - November/December 2017 - 13
Potentials - November/December 2017 - 14
Potentials - November/December 2017 - 15
Potentials - November/December 2017 - 16
Potentials - November/December 2017 - 17
Potentials - November/December 2017 - 18
Potentials - November/December 2017 - 19
Potentials - November/December 2017 - 20
Potentials - November/December 2017 - 21
Potentials - November/December 2017 - 22
Potentials - November/December 2017 - 23
Potentials - November/December 2017 - 24
Potentials - November/December 2017 - 25
Potentials - November/December 2017 - 26
Potentials - November/December 2017 - 27
Potentials - November/December 2017 - 28
Potentials - November/December 2017 - 29
Potentials - November/December 2017 - 30
Potentials - November/December 2017 - 31
Potentials - November/December 2017 - 32
Potentials - November/December 2017 - 33
Potentials - November/December 2017 - 34
Potentials - November/December 2017 - 35
Potentials - November/December 2017 - 36
Potentials - November/December 2017 - 37
Potentials - November/December 2017 - 38
Potentials - November/December 2017 - 39
Potentials - November/December 2017 - 40
Potentials - November/December 2017 - 41
Potentials - November/December 2017 - 42
Potentials - November/December 2017 - 43
Potentials - November/December 2017 - 44
Potentials - November/December 2017 - 45
Potentials - November/December 2017 - 46
Potentials - November/December 2017 - 47
Potentials - November/December 2017 - 48
Potentials - November/December 2017 - Cover3
Potentials - November/December 2017 - Cover4
https://www.nxtbook.com/nxtbooks/ieee/potentials_20190102
https://www.nxtbook.com/nxtbooks/ieee/potentials_20181112
https://www.nxtbook.com/nxtbooks/ieee/potentials_20180910
https://www.nxtbook.com/nxtbooks/ieee/potentials_20180708
https://www.nxtbook.com/nxtbooks/ieee/potentials_20180506
https://www.nxtbook.com/nxtbooks/ieee/potentials_20180304
https://www.nxtbook.com/nxtbooks/ieee/potentials_20180102
https://www.nxtbook.com/nxtbooks/ieee/potentials_111217
https://www.nxtbook.com/nxtbooks/ieee/potentials_091017
https://www.nxtbook.com/nxtbooks/ieee/potentials_070817
https://www.nxtbook.com/nxtbooks/ieee/potentials_050617
https://www.nxtbook.com/nxtbooks/ieee/potentials_030417
https://www.nxtbook.com/nxtbooks/ieee/potentials_010217
https://www.nxtbook.com/nxtbooks/ieee/potentials_111216
https://www.nxtbook.com/nxtbooks/ieee/potentials_091016
https://www.nxtbook.com/nxtbooks/ieee/potentials_070816
https://www.nxtbook.com/nxtbooks/ieee/potentials_050616
https://www.nxtbook.com/nxtbooks/ieee/potentials_030416
https://www.nxtbook.com/nxtbooks/ieee/potentials_010216
https://www.nxtbook.com/nxtbooks/ieee/potentials_111215
https://www.nxtbook.com/nxtbooks/ieee/potentials_091015
https://www.nxtbook.com/nxtbooks/ieee/potentials_070815
https://www.nxtbook.com/nxtbooks/ieee/potentials_050615
https://www.nxtbook.com/nxtbooks/ieee/potentials_030415
https://www.nxtbook.com/nxtbooks/ieee/potentials_010215
https://www.nxtbook.com/nxtbooks/ieee/potentials_111214
https://www.nxtbook.com/nxtbooks/ieee/potentials_091014
https://www.nxtbook.com/nxtbooks/ieee/potentials_070814
https://www.nxtbookmedia.com