How Imperfect Knowledge Shapes Financial Markets

Asset markets are indispensable in harnessing society’s diverse views and insights about future business performance. But those views are shaped as much by emotion and crowd mentality as by rational expectations.

Every society must decide how to allocate its scarce savings among the many companies (and households) seeking to finance investment projects. Ideally, the limited savings should be channeled to companies with the highest future returns and lowest risk. Companies with poor prospects should not receive financing at all. The problem, of course, is that there are a great many views about companies’ prospects and no objective way to determine that one view is necessarily better than any other. Financial markets address this problem by placing weight on the views of all its participants in setting prices. Prices fluctuate as news about prospects becomes available. Better prospects lead to price increases and greater access to financial capital. Deteriorating prospects lead to the converse. Once we recognize that no one possesses the truth about the future, we think it is difficult to imagine a better means of allocating society’s savings across companies.

This view of financial markets stands at odds with both the efficient markets hypothesis (EMH) and behavioral finance. EMH rests on the rational expectations hypothesis (REH), which supposes that rational market participants know the true probability distribution governing outcomes and use it to interpret news about companies’ prospects. If true, financial markets would be nearly perfect in setting prices and allocating capital. Behavioral finance also uses REH to characterize rational forecasting. However, behavioral models suppose that some or all market participants lack access to the true probability distribution. In these models, price fluctuations are driven by emotions, crowd psychology, and momentum trading, rather than news about company prospects. According to this view, financial markets often set prices and allocate capital poorly.

The efficient markets view’s embrace of rational expectations is striking. It assumes away the very reason society needs financial markets: to harness the great diversity of views in society about companies’ prospects. Behavioral finance’s reliance on REH is also odd given its emphasis on psychological realism.

Our paper (Cavusoglu, Goldberg, and Stillwagon 2019) on currency returns is part of INET’s imperfect knowledge economics (IKE) research program. This program’s premise is that no one can know the true probability distribution governing future market outcomes. The reason is simple: historical developments that are to some extent novel (e.g., shifts in fiscal and monetary policy, changes in institutions, and technological progress) trigger structural change in the process underpinning market outcomes. The novelty of these developments implies that they are to some extent non-repetitive and that their impact on the data generating process “deal[s] with situations which are far too unique…[to rely solely on] statistical tabulations” (Knight 1921. p. 198). Consequently, no amount of past data and tabulation can deliver the true probability distribution governing future market outcomes. How could one have anticipated, years in advance, the probability and implications of, for example, Brexit, the US government shutdown over a border wall, or Draghi’s “Whatever it takes” speech?

Ignoring the primacy of imperfect knowledge has led to many puzzles in financial markets. A longstanding anomaly is the inability of conventional risk premium models to account for excess returns in asset markets (Fama 2013, and Cochrane 2011). Conventional models rely on expected utility theory. They relate the market’s expected return to individuals’ degree of risk aversion and fundamentals such as asset supplies or consumption growth. The market’s expected return is not observable. Researchers have thus used data on actual, ex post returns. To draw inferences about expected returns, they invoke the rational expectations hypothesis, which implies that expected and ex post returns differ by a white noise – purely random – forecast error. As such, existing empirical studies of risk premium models provide tests of the joint hypothesis of predictions concerning expected excess returns and REH’s prediction of white noise forecast errors. The rejection of this joint hypothesis in study after study has led to the view that returns are not driven by risk or fundamentals.

In our paper on currency returns, we provide evidence that the anomaly stems from a failure of both expected utility theory and REH. Our investigation considers two competing portfolio balance models, one based on Dornbusch’s (1983) international CAPM (ICAPM) and the other developed by Frydman and Goldberg (2007, 2013). The latter model follows Dornbusch (1983) in its basic setup, but replaces expected utility theory (EUT) with Kahneman and Tversky’s (1979) prospect theory (PT). Researchers have found that alternatives to EUT improve the consumption CAPM’s empirical performance. (See Barberis, Huang, and Santos 2001, Backus et al. 2010, and Bansal and Shaliastovich 2013). Such alternatives may also improve the empirical performance of portfolio balance models.

We also sidestep the joint hypothesis problem by using survey data on exchange rate expectations, which provide a proxy for the market’s expected return. These data enable us to test directly the risk premium models’ predictions concerning expected returns, without conflating them with REH.

The Expected Utility approach- and Prospect Theory-based models imply different risk factors. Both models relate the risk premium on foreign currency to the country’s bilateral international debt position (IDP). With EUT, the risk premium also depends on the conditional volatility of returns. But, with prospect theory, the premium depends positively on the gap between the exchange rate and market participants’ assessments of its benchmark value. The “gap effect” is intuitive: the more over- or undervalued a currency becomes, the more risky it is for market participants who speculate on a further over- or undervaluation. The two models’ predictions for sign reversals also differ. Both models relate the risk premium’s sign to the sign of the country’s international debt position. But with the PT model, the sign of the risk premium also depends on the risk assessments of the bulls (who take long positions in foreign exchange) relative to those of the bears (who take short positions). This additional factor gives the model greater potential to explain sign reversals.

A key issue for the empirical analysis is whether excess returns are stationary. Most studies make the assumption that they are. But, there is considerable evidence that excess returns are highly persistent and possibly nonstationary—that is they change over time in unpredictable ways, as opposed to returning to some average value. (See Johansen et al. 2010, Juselius 2014, and Juselius and Assenmacher 2017). Frydman and Goldberg (2007) address this problem with single-equation error correction models. In this paper, we rely on the I(1) cointegrated vector autoregression (CVAR) framework (Johansen 1996, Juselius 2006), which is better suited for handling persistent variables. We find that expected excess returns and other variables in the information set are best characterized as random walks, implying that they are nonstationary.

We also extend Frydman and Goldberg’s (2007) analysis by including measures of exchange rate volatility and bilateral IDP in our information set. This enables us to consider all of the PT model’s predictions and those of the international CAPM. We also consider a hybrid model that includes both the gap and ICAPM risk factors. This is the first paper to compare or combine these two specifications of risk.

To preview our results, we find little support for the ICAPM. By contrast, the PT model’s prediction of a positive gap effect is borne out in all three currency markets examined. In two of these markets, IDP enters the cointegrating relationship as predicted. The PT model also accounts for sign reversals better than the international CAPM. Interestingly, our CVAR results show the strongest evidence for the hybrid model in which the gap, conditional volatility, and IDP drive expected excess returns.

The results suggest that financial markets do a better job in setting prices than implied by behavioral finance. Risk and fundamentals do matter, but to see their importance, we need to move away from EUT and REH.

Behavioral finance has uncovered much evidence that psychological factors such as emotion and confidence underpin market participants’ forecasting. Its reliance on REH, however, leads then to interpret such underpinning as evidence of less than full rationality. After all, if rational participants have access to the true probability distribution, there would be no need to rely on anything else. Doing so would be a sign of irrationality and poorly functioning markets.

IKE models, by contrast, recognize that rational forecasting must rely on psychological factors, as well as fundamentals and calculation (Frydman and Goldberg 2011 and Frydman and Stillwagon 2018). Rational individuals understand that they do not know the truth about future returns and risk. They therefore consider a range of possible scenarios that are underpinned by reason and past data. There is no objective way to determine ex ante which scenario is better than any other. Rational individuals therefore must rely on their emotion, confidence, and intuition in choosing a scenario on which to base their forecasting.

IKE’s recognition that we all must cope with imperfect knowledge leads to an intermediate view of financial markets. They are indispensable in harnessing the diverse views and insights about the future that exist in society. But, because trading is based on imperfect knowledge, markets are not perfect in setting prices. Asset prices undergo persistent swings away from and back toward benchmark values in their normal course of functioning. Short-termism can at times lead to departures that are excessive. It is during these times that Frydman and Goldberg (2011) call for excess dampening macroprudential policies.


Backus, David K., Federico Gavazzoni, Christopher Telmer and Stanley E. Zin (2010) “Monetary Policy and the Uncovered Interest Parity Puzzle.” NBER working paper no. w16218 National Bureau of Economic Research.

Bansal, Ravi and Ivan Shaliastovich (2013)”A Long-run Risks Explanation of Predictability Puzzles in Bond and Currency Markets,” Review of Financial Studies 26(1), 1-33.

Barberis, Nicholas, Ming Huang and Tano Santos (2001) “Prospect Theory and Asset Prices,” Quarterly Journal of Economics, 116(1), 1-53.

Cochrane, John (2011) “AFA Presidential Address: Discount Rates,” The Journal of Finance 66(4), 1047-1108.

Cavusoglu, Nevin, Michael D. Goldberg, and Josh R. Stillwagon (2019) “New Evidence on the Portfolio Balance Approach to Currency Return,” INET Working Paper.

Dornbusch, Rudiger (1983) “Exchange Risk and the Macroeconomics of Exchange Rate Determination,” in The Internationalization of Financial Markets and National Economic Policy, R. Hawkins, R. Levich and C. Wihlborg, Greenwich CT: JAI Press.

Fama, Eugene (2013) “Two Pillars of Finance,” Nobel Prize Lecture, December 8th, 2013.

Frydman, Roman and Michael D. Goldberg (2007) Imperfect Knowledge Economics: Exchange rates and Risk, Princeton, NJ: Princeton University Press.

Frydman, Roman and Michael D. Goldberg (2011) Beyond Mechanical Market: Asset Price Swings, Risk, and the Role of the State, Princeton, NJ: Princeton University Press.

Frydman, Roman and Michael D. Goldberg (2013) “Opening Models of Asset Prices and Risk to Non-routine Change,” in Roman Frydman and Edmund S. Phelps (eds.) Rethinking Expectations: The Way Forward for Macroeconomics, 207-250.

Frydman, Roman and Josh R. Stillwagon (2018) “Fundamental Factors and Extrapolation in Stock-Market Expectations: The Central Role of Structural Change,” Journal of Economic Behavior and Organization 148(12), 189-198.

Johansen, Soren (1995) Likelihood-based Inference in Cointegrated Vector Autoregressive Models, Oxford University Press.

Johansen, Soren, Katarina Juselius, Roman Frydman, and Michael D. Goldberg (2010) “Testing Hypotheses in an I(2) Model with Piecewise Linear Trends. An Analysis of the Persistent Long Swings in the Dmk/$ Rate,” Journal of Econometrics 158(1), 117-129.

Juselius, Katarina (2006) The Cointegrated VAR Model: Methodology and Applications, Oxford University Press.

Juselius, Katarina (2014) “Testing for Near I(2) Trends When the Signal-to-Noise Ratio Is Small,” Economics: The Open-Access, Open-Assessment E-Journal, v. 8.

Juselius, Katarina and Katrin Assenmacher (2017) “Real Exchange Rate Persistence and the Excess Return Puzzle: The Case of Switzerland versus the US,” Journal of Applied Econometrics, September-October, 32(6), 1145-55.

Kahneman, Daniel and Tversky, Amos (1979) “Prospect Theory: An Analysis of Decision under Risk,” Econometrica 47, 263-291.

Knight, F.H. (1921) ” Cost of Production and Price over Long and Short Periods,” Journal of Political Economy, April 1921, 29, 304-35.

Share your perspective