Category: big data

The complexity economics view of substitution

A key problem in economics is how much firms can substitute specific inputs in order to carry out production. This issue has become more relevant now than in the past few decades. First, during the Covid-19 pandemic certain industries completely shut down, depriving some downstream industries of certain critical inputs. Now, a crucial policy question is how much European economies can get away without Russian gas in light of the Ukraine-Russia war. An influential policy report recently provided an answer to this question by using a state-of-the-art general equilibrium model. The report contains an interesting discussion on substitutability. It distinguishes between the “engineering view” of substitutability at the very microeconomic level, according to which lack of inputs that are technologically necessary can completely stop production in specific plants, and the “economic view”, according to which there is more substitutability at the macroeconomic level thanks to the ability of firms to find other suppliers, quickly change their production technologies to rely less on lacking inputs, etc.

In this blog post I argue that these mechanisms of substitution are only implicitly included in general equilibrium models, and that the complexity economics view of substitution provides an alternative that merges the engineering and economic views by explicitly representing the engineering constraints in production and how they can be relaxed. I discuss this at a general level and then give a rudimentary example of what I mean by discussing some work that my coauthors and I did on what we call the “partially binding Leontief” production function.

Everyone agrees that the economy has high capability to adapt. The policy report mentioned above gives some examples. When China implemented an export embargo on rare earths against Japan in 2010, Japanese firms found ways to use less rare earths in production or substitute them altogether. When an oil pipeline reaching Germany was shut down due to oil contamination in 2019, German firms found ways to import oil through other channels. In World War II, because the US were cut out of rubber supply, American firms developed synthetic rubber.

How are these substitution mechanisms captured in state-of-the-art general equilibrium models? The starting point is the nested CES production function. To make stuff (or to provide a service) you need several inputs. First, you need “primary factors”. These are mainly labor, capital (e.g. machines) and land. Second, you need “intermediate goods” which are used up in production. For instance, if you want to produce steel, you use iron and electricity as intermediate goods, but you probably also use restaurant services such as the canteen where employees have lunch. The nested CES production function aggregates intermediate goods into a composite intermediate good, primary factors into a composite primary factor, and then it aggregates the composite intermediate good and the composite primary factor specifying a level of production. These composites are also called “nests”.

CES stands for “Constant Elasticity of Substitution”. A nice property of this production function is that you can specify a given value for the elasticity of substitution, which is the easiness with which firms substitute their inputs. In typical calibrations, such as the one used in the policy report, you specify some values for the elasticity of substitution across intermediates, across primary goods, and between the intermediate composite and the primary composite. This means, among other things, that all intermediate inputs can be substituted equally well. Keeping with the example above, you can substitute iron and electricity with restaurant services. If the elasticity of substitution between intermediates and factors is sufficiently high, you may also easily substitute iron and electricity with labor or land. These elasticities of substitutions are usually calibrated based on a combination of econometric studies and plausibility arguments.

This is what I mean when I say that the standard approach to substitution implicitly incorporates engineering constraints and the way they can be relaxed. By assuming a level of substitutability that is in between zero and infinite, CES production functions implicitly capture the idea that substitution is possible but not so easy. One can play with the elasticity of substitution parameter depending on the question at hand. For instance, because technological change takes time, substitution is easier in the long run than in the short run, so it is reasonable to assume higher elasticities when one is concerned with long run responses of the economy. (It is also reasonable to assume higher elasticities at higher aggregation levels.)

To be fair, CES production functions are the best you can do in a world with limited real-world data and little information about production processes and if you want to work with mathematically elegant models that allow for easily interpretable results and closed-form solutions.

But the stakes are high and policy makers need to be sure about the quantitative reliability of macroeconomic models! Under a 30% reduction in gas imports, according to old style “Leontief” models that do not allow substitution there could be up to a 30% reduction in German GDP. By contrast, the policy report that uses a state-of-the-art general equilibrium model predicts up to a 3% reduction because it assumes that gas can be substituted by other inputs. Whatever result turns out to be true (we may never know if the gas import ban is not enacted), in my view policy decisions should be based on models that incorporate real-world data in a much more granular way, whose predictive performance is tested on past episodes.

For instance, imagine a model of the economy with 629 firms, each representing a 4-digit NACE industry. As an example, consider industry 2420, “Manufacture of tubes, pipes, hollow profiles and related fittings, of steel”. One could consult with engineers working in plants classified in this industry (and in all other 628 industries) and get detailed information about the physical processes that take place, which inputs are absolutely necessary and in which ratios, which alternatives can be considered for which inputs, how long it would take to come up with replacements. This information would be incorporated into a dynamic model in which firms buy inputs, replenish or use up their inventories, produce and sell outputs over time. In this way, users of the model can introduce a shock and explicitly see which input bottlenecks are created in the short run and in the long run, which cascading effects can occur (e.g. some industry stops production, and this leads other industries to stop production as well), and obtain a reliable estimate on the overall economic impact that is explicitly based on the industrial structure of the economy. The empirical performance of this model would be tested against several historical episodes in which some inputs became unavailable.

My coauthors and I built a model like that to assess the economic effects of the Covid-19 pandemic on the UK economy. We conducted a survey of industry analysts to determine which inputs were critical for production in a short time frame. We asked this question for each of 55 2-digit NACE industries, for each of 55 inputs. See the answers in the figure below. A column denotes an industry and the corresponding rows its inputs.  Blue colors indicate critical inputs, red and white non-critical inputs (red is an intermediate case of important inputs). The results shown here indicate that the majority of elements are non-critical inputs (2,338 ratings), whereas only 477 industry-inputs are rated as critical and 365 inputs are rated as important. Electricity and Gas (D35) are rated most frequently as critical inputs in the production of other industries (almost 60% of industries). Also frequently rated as critical are Land Transport (H49) and Telecommunications (J61). At the same time, many manufacturing industries (NACE codes starting with C) stand out as relying on a large number of critical inputs. For example, around 27% of inputs to Manufacture of Coke and Refined Petroleum Products (C19) as well as to Manufacture of Chemicals (C20) are rated as critical.

 

 

 

 

 

 

 

 

 

 

Using these data as a starting point, our model assumes that partial lack of any critical input proportionally stops production because of fixed technological recipes (as in Leontief models), while lack of non-critical inputs does not stop production (which is why we call our production function the partially binding Leontief). Our model is dynamic, i.e. it produces time series of production in all industries that take into account depletion of inventory stocks and cascading effects. In the figure below one can see that when the lockdown starts, production in certain industries decreases immediately, while other industries stop production when they run out of critical inputs, and this makes other industries stop production as well.

 

 

 

 

 

 

We also checked the predictive performance of our model, making an out-of-sample forecast of the reduction in UK GDP that turned out to be more precise than competing estimates.

Our approach should be viewed as a first step in a line of research that tries to explicitly incorporate engineering and technological details to make a realistic macroeconomic model of the production side of the economy. First of all, our survey is still too aggregate, as remaining at the level of 2-digit 55 industries makes it impossible to pin down technological details that differ across firms within any 2-digit industry. Next, our modeling does not consider prices, which were not a major factor during the Covid-19 pandemic but look more important in the current situation. Moreover, we do not allow substitution with imported goods. On a higher level, imprecise assumptions at the micro level may well lead to large errors at the macro level (in machine learning parlance, models that follow our approach may have very little bias but a lot of variance, while general equilibrium models that use uniform elasticities of substitution may have more bias but less variance).

Despite these shortcomings, we consider our approach as an example of the complexity economics view of substitution: we use “a lot” of data [1] to initialize industry-input-level substitutability in a non-equilibrium dynamic model that produces macroeconomic results by explicitly aggregating from technologically micro-founded production units, generating cascading effects and reliable forecasts (at least for the pandemic episode). At this point there is still a lot to do, nonetheless I view this as an exciting area of research that the complexity economics community is already focusing on.

[Thanks to François Lafond and Doyne Farmer for comments.]

[1] Our survey of industry analysts produced 3025 data points, which can be compared to the 4 aggregate elasticities that are qualitatively calibrated from data in the policy report.

Bank of England conference on big data and machine learning

I recently presented our work on big housing data at the Bank of England conference on “Modelling with Big Data and Machine Learning”. This has been a super-interesting conference where I learned a lot. Now that the slides of the workshop have been uploaded online, I thought I would write a blog post to share something of what I learned. I’ll also take this chance to write about how big data are related to this blog and have the potential to influence theoretical economic models.

The first session of the conference was about nowcasting. I particularly liked the talk by Xinyuan Li, a PhD student at London Business School. In her job market paper, she asks if Google information is useful for nowcasting even when other macroeconomic time series are available. Indeed, most papers showing that Google Trends data improve nowcasting accuracy of, say, the unemployment rate, do not check if this improvement still holds once the researcher considers series of payrolls, industrial production, capacity utilization, etc. Li combines macroeconomic and Google Trends time series in a state-of-the-art dynamic factor model and shows that Google Trends add little, if any, nowcasting accuracy. However, if one increases the number of associated Google Trends time series by using Google Correlate, a tool that finds the Google searches most correlated with a given series, nowcasting accuracy improves. So under some conditions Google information is indeed useful.

The first keynote speaker was Domenico Giannone, from the New York FED. The question in his paper is whether predictive models of economic variables should be dense or sparse. In a sparse model only few predictors are important, while in a dense model most predictors matter. To answer this question it is not enough to estimate a LASSO model and count how many coefficients “survive”. Indeed, for LASSO to be well-specified, the correct model must be sparse. The key idea of the paper is to allow for sparsity, without assuming it, and let the data decide. This is done via a “spike and slab” model, that contains two elements: a parameter q that quantifies the probability that a coefficient is positive; and a parameter γ that shrinks the coefficients. The same predictive power can be achieved in principle by only including few coefficients or by keeping all coefficients but shrinking them. In a Bayesian setting, if the posterior distribution is concentrated at high values of q (and so low values of γ) it means that the model should be dense. This is what happens in the figure below, in five out of six datasets in micro, macro and finance. Yellow means high value for the posterior, and only in the case of micro 1 it is high for q ≈ 0. So in most cases a significant fraction of predictors is useful for forecasting, leading to an illusion of sparsity.

The most thought-provoking speech in the panel discussion on “Opportunities and risks using big data and machine learning” was again by Giannone. What he said is best summarized in a paper that everyone interested in time series forecasting with economic big data should read. His main point is that macroeconomists had to deal with “big data” since the birth of national accounting and business cycle measurement. State-of-the-art nowcasting and forecasting techniques that he jointly developed at the New York FED include a multitude of time series at different frequencies, such as the ones shown in the figure below. These series are highly collinear and rise and fall together, as shown in the heat map in the horizontal plane. According to Giannone, apart from a few exceptions, big data coming from the internet have little chance to improve over carefully collected data from established statistical national agencies.

On a different note, in a following Methodology session I found out about a very interesting technique: Shapley regressions. Andreas Joseph from the Bank of England talked about the analogy between Shapley values in game theory and in machine learning. In cooperative game theory Shapley values quantify how much every player contributes to the collective payoff. A recent paper advanced the idea of applying the same formalism to machine learning. Players become predictors and Shapley values quantify the contribution of each predictor. While there exist several ways to quantify the importance of predictors in linear models, Shapley values extend nicely to potentially highly non-linear models. His colleague Marcus Buckmann presented an application to financial crisis forecasting, using data back to 1870 (see figure below). Interestingly, global and domestic credit contribute a lot to forecasting, while current account and broad money are not so important. In general, Shapley regressions might help with the interpretability of machine learning “black boxes”.

The last session I’d like to write about is the one on text analytics. Eleni Kalamara, a PhD student in King’s College, presented her work on “making text count”. The general goal of her project is to see whether text from UK newspapers proxies sentiment and uncertainty and is useful to predict macroeconomic variables. What I found most interesting was the comparison of 13 different dictionaries that turn text into sentiment and uncertainty indicators. Given such a proliferation of metrics, it seems very useful to systematically compare them. Another interesting talk in the same session was given by Paul Soto. In his job market paper “breaking the word bank”, he used Word2Vec to find words related to “uncertainty” in transcripts of banks’ conference calls. Word2Vec is a machine learning algorithm that finds a vector representation for words taking into account both syntactics and semantics. The figure below shows a two-dimensional projection of the vector space; words related to uncertainty are highlighted in yellow to the right. In his paper, Soto shows that banks with higher idiosyncratic uncertainty are less likely to give loans and more likely to increase their liquidity.

There were a lot of other great talks. For example, Thomas Renault from Sorbonne showed how to detect financial market manipulation—in particular, pump and dump schemes—from Twitter. Luca Onorante from the European Central Bank demonstrated how to select the most relevant Google Trends in a context of Bayesian Model Averaging. Emanuele Ciani from the Bank of Italy developed on a method first introduced by Jon Kleinberg to predict the agents that would most benefit from policies, nicely combining ideas from prediction and from causal inference. For the many other interesting talks, please check the program or look at the slides.

So, what do big data have to do with complexity economics? This conference was purely about statistical models. My sense is that economic theorists are not responding to big data as much as empirical economists. True, heterogenous agent models that use micro evidence to discriminate between different macro models that produce the same macro outcomes are increasingly popular, but I don’t think they quite exploit the power of big data. On the other hand, large-scale “microsimulation” Agent-Based Models (ABMs) that are directly feeded with data and solved forward without imposing equilibrium constraints seem more promising to exploit the big data opportunities. A nice example of this is the ongoing work by Sebastian Poledna and coauthors on “Economic forecasting with an agent-based model”, exploiting comprehensive datasets for the Austrian economy. I plan to work on prediction with ABMs too during my postdoc funded by the James S. Mc Donnell Foundation — better out-of-sample forecasting performance would be a compelling motivation for the enhanced realism of ABMs that comes at the cost of other features that are considered important in mainstream theoretical models.