Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Review of Economics

Jahrbuch für Wirtschaftswissenschaften

Editor-in-Chief: Berlemann, Michael

Ed. by Haucap, Justus / Thum, Marcel

3 Issues per year

Online
ISSN
2366-035X
See all formats and pricing
More options …

Beauty and Ugliness of Aggregation over Time: A Survey

Nlandu Mamingi
  • Corresponding author
  • Department of Economics, The University of the West Indies, P.O.Box 64, Cave Hill Campus, Bridgetown BB11000, Barbados
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2017-11-30 | DOI: https://doi.org/10.1515/roe-2017-0027

Abstract

This paper delivers an up-to-date literature review dealing with aggregation over time of economic time series, e.g. the transformation of high-frequency data to low frequency data, with a focus on its benefits (the beauty) and its costs (the ugliness). While there are some benefits associated with aggregating data over time, the negative effects are numerous. Aggregation over time is shown to have implications for inferences, public policy and forecasting.

Keywords: aggregation over time; temporal aggregation; systematic sampling; mixed aggregation

JEL Classification: C10; C22; C32; C43

1 Introduction

Data are at the core of empirical or applied economics and econometrics. Data in this context can be characterized according to their sources (experimental, quasi-experimental or observational), their types (quantitative or qualitative), their configurations (time series, cross-section or panel) or their frequencies (high frequency or low frequency).

This paper delivers an overview of the literature on the effects of aggregation over time, understood as the transformation of high frequency to low frequency data, on a wide range of economic or econometric undertakings. While the topic of temporal aggregation has been around for quite a while, the related full-fledged literature review papers on the topic have been rather scarce 1 and often provide only a partial perspective. The present literature review attempts to fill these gaps. 2 Precisely, the objective of the paper is twofold: (i) to reexamine the beauty (positive effects) of aggregation over time as well as its ugliness (negative effects) and (ii) to point out the findings that need further investigation as well as untreated issues. Methodologically, the paper emphasizes the message rather than the econometric or mathematical derivation of the message, as is appropriate for a survey article.

We show that there are only a few positive effects of aggregation over time, while the negative effects are numerous empirical investigations need to consider, among other factors, the role of the data span in the power and size of some unit root/cointegration tests as well as the impact of structural change on test statistics under different types of aggregation over time. We also show that aggregation-over-time-issues have far reaching effects for inferences, public policy and forecasting. For example, the fact that aggregation over time generally alters causality relations between variables and the exogeneity status of variables might blur policy instruments useful to deal with economic issues such as inflation, budget deficits and output growth dynamics.

The paper proceeds as follows. Section 2 introduces the concept of aggregation over time. Section 3 deals with the beauty of aggregation over time. Section 4 focuses on the literature concerned with the ugliness of aggregation over time. Section 5 essentially deals with issues needing further investigation. Section 6 contains concluding remarks.

2 Aggregation over time

It is often the case that temporally aggregated data are used in public policy evaluations and other empirical studies. This holds true especially for countries with low levels of development. The high costs of collecting frequently and processing (new) data is the major impediment to generating high frequency data. In many situations, researchers and/or policy makers only have aggregated data to work with. “Yet, the agent’s time decision interval and the data sampling interval do not necessarily coincide” (Mamingi, 1992, 95). For example, the agent’s decision interval may be monthly while the sample data interval is quarterly; that is, quarterly observations are used instead of monthly observations. This situation leads to many problematic issues which are due to aggregation over time (Mamingi, 1992, 2006a).

Before discussing the effects of aggregation over time, it is useful to define the concept itself. Aggregation over time can takes two forms. It can be a shift from continuous time to discrete time. The major contributions to this perspective are Phillips (1956), Sims (1971), Bergstrom (1984) and Phillips (1991). Aggregation over time can also be a shift from small discrete time units (high data frequency) to large discrete time units (low data frequency). In this paper, we concentrate on the discrete approach.

Aggregation over time encompasses the following scenarios: temporal aggregation, systematic sampling and mixed aggregation. Temporal aggregation deals with time dimension variables or variables whose values have been either averaged or summed over an interval of time. Examples of these variables, also known as flow variables, include consumption per year, income per year, saving per year, yearly fiscal deficits, investment per year, profits per month and rates of return.

Systematic sampling, “a type of temporal aggregation appropriate for stock variables” (Granger, 1990, 26), deals with variables that are systematically sampled; that is, their values are recorded at particular points in time. Examples of these variables, also called stock variables or variables without time dimension, include money supply, stock of cash, savings, unemployment rate, wealth, labor force, inventory and national debt.

Mixed aggregation arises in the context of relationships between variables. For example, in the framework of a bivariate regression, mixed aggregation arises whenever one variable is temporally aggregated and another one is systematically sampled. Two cases have to be distinguished. In the first case, the explained variable is a flow variable and the explanatory variable a stock variable; here, we refer to this relationship as mixed aggregation of type 1. The second case concerns the stock-flow relationship; here, we refer to the relationship as mixed aggregation of type 2. The stock adjustment model, where inventories and sales are stock and flow variables, respectively, as well as the growth-cum-debt model that treats debt and income as stock and flow variables, respectively, are typical examples of mixed aggregation (Mamingi, 2006a).

One problem in this context is a lack of common terminology for the concepts explained above. Often aggregation over time is referred to as temporal aggregation. This may of course lead to confusion as temporal aggregation in the latter sense encompasses temporal aggregation, systematic sampling and mixed aggregation. Other authors use the terms skip sampling or end-of-period sampling for systematic sampling. A harmonization of the terminology would be helpful. In this paper, aggregation over time encompasses temporal aggregation, systematic sampling and mixed aggregation.

3 The beauty of aggregation over time

There are a few positive effects attributed to aggregation over time, particularly in the context of the mean model.

First, aggregation over time does not affect stationarity or non-stationarity 3 of time series. Thus, a time series which is stationary (non stationary) at the disaggregated level remains so at the aggregated level. This property has been directly or indirectly derived among others by Telser (1967), Amemiya and Wu (1972), Tiao (1972), Brewer (1973), Tiao and Wei (1976), Wei (1981), Harvey (1981), Ahsanullah and Wei (1984), Weiss (1984), Stram and Wei (1986), Christiano and Eichenbaum (1987), Rossana and Seater (1995) and Pierse and Snell (1995). Most of these studies resorted to analytical tools, accompanied by Monte Carlo experiments and empirical evidence to draw and back up their conclusions.

Second, temporally aggregated data are less noisy than their disaggregated counterparts (Friend and Taubman, 1964; Haitovsky, Treyz, and Su, 1974; among others).

Third, aggregation over time does not affect cointegratedness of variables. To recall, cointegration of variables refers to the long-run equilibrium between or among non-stationary variables or even among non-stationary and stationary variables. While Granger and Weiss (1983) conjectured the property (invariance of cointegratedness of variables under aggregation over time), Stock (1987), Phillips (1991) and Mamingi (2006a) provided the formal proof using either a continuous framework or a discrete context. Not only does cointegration survive aggregation over time but also the cointegrating vector remains unchanged under all types of aggregation. In Appendix A we provide the proof for mixed aggregation, a yet unresolved case (see Granger and Weiss, 1983, 264).

Finally, in variance or conditional heteroscedasticity models there is a property which points to the closeness of weak GARCH processes at the univariate as well as multivariate levels. That is, a weak GARCH model at the disaggregate level remains so at the aggregate level (see Drost and Nijman, 1993; Hafner, 2004).

Most of the above properties have been confirmed empirically by quite a number of studies. As an example, Marcellino (1999) used the term structure of interest rates for Canada for which the disaggregated data are monthly observations on the Canadian 10-year government bond yield (RL) and 90-day deposit Rate (RS). He found that each variable was integrated of order one and uncovered one cointegration vector: RSβ1RL+β2with β1=1 and β2>0. He then constructed the quarterly and half-yearly systematic sampling counterparts of the monthly observations and likewise the average counterparts. He uncovered the unit root in each aggregate component and also found that cointegration holds for systematic sampling (quarterly and half-yearly) and temporal aggregation (quarterly and half-yearly).

4 The ugliness of aggregation over time

In comparison to the few positive effects of aggregation over time, there are many negative effects which, however, also depend on the sort of aggregation over time (temporal aggregation, systematic sampling and mixed aggregation). The negative effects, which will be discussed in the following, have been derived analytically and/or by Monte Carlo experiments, corroborated by empirical examples. The negative effects include: lower precision of estimation and prediction (Wei, 1978; Zellner and Montmarquette, 1971), inability to make short-run forecasts (Zellner and Montmarquette, 1971), aggregation bias in distributed lag models (Brewer, 1973; Engle and Liu, 1972; Moriguchi, 1970; Mundlak, 1961; Tiao and Wei, 1976; Wei, 1978), OLS asymptotic biases of estimates of half-lives in purchasing power parity (Chambers, 2005), alterations of structures of time series (Telser, 1967, Amemiya and Wu, 1972; Tiao, 1972, among others), generation of time series correlation under temporal aggregation (Working, 1960), change in seasonal unit roots (Granger and Siklos, 1995), change in measures of persistence of shocks (Rossana and Seater, 1995), lower power of tests (see, for example, Teles and Wei, 2000; Zellner and Montmarquette, 1971), alterations of power of residual based tests for cointegration (Mamingi, 1992, 2005b), distortions of empirical sizes 4 of residual based tests for cointegration 5 (Mamingi, 1992, 1993, 2006b), distortions of causality relationships in multiple time series models (Geweke, 1978; Sims, 1971; Wei, 1982), vector autoregressive models (see, among others, Breintung and Swanson, 2002; Christiano and Eichenbaum, 1987; Marcet, 1987) and error correction models (Gulasekaran and Abeysinghe, 2003; Mamingi, 1992, 1996, 2006a), modification of exogeneity patterns (Campos, Ericsson, and Hendry, 1990; Hendry, 1992; Marcellino, 1999), alterations of impulse response functions (Swanson and Granger, 1997; Marcellino, 1999), change in trend-cycle decomposition (Lippi and Reichlin, 1991; Marcellino, 1999), change in nonlinearity patterns (Granger and Lee, 1999; Teles and Wei, 2000), change in quality of forecasts (Lütkepohl, 1987) and alterations of semi strong and strong GARCH processes (Drost and Nijman, 1993; Hafner, 2004). For issues in aggregated GARCH models, it is worth consulting Silverstrini and Veredas (2008).

While we refrain here from discussing all mentioned issues in length, we at least comment on a few problems of great importance.

4.1 Aggregation over time and time series structure

The general format of a time series is an ARIMA(p,d,q) process where p is the autoregressive order, d represents the order of integration and q stands for the moving average order. Although aggregation over time does not change the status of stationarity/non-stationarity of time series variables, it generally leads to an alteration of time series structures. That is, the structure of a given series can be transformed into another structure with aggregation over time. For example, an autoregressive process of order one, AR(1), in a disaggregated model theoretically changes into an ARMA(1,1) process if temporally aggregated. By the same token, a random walk process generally becomes an integrated moving average process of order one, IMA(1,1), under temporal aggregation but remains to be a random walk process under systematic sampling (Amemiya and Wu, 1972; Telser, 1967; Tiao, 1972, among others). The limiting result of an ARIMA(p,d,q) process and an IMA(d,q) process is an IMA(d,l) process with ld1 under systematic sampling (see Wei, 1978a) and an IMA(d,d) process under temporal aggregation (Stram and Wei, 1986). Rossana and Seater (1995) noted that the latter limiting process can become an IMA(d,d-1) process if the increase of standard error is bigger than the increase of the autocorrelation estimated coefficients. Under systematic sampling, when d=0, the limiting model of a stationary process becomes a white noise. The authors utilized a series of US economic variables obtained from Citibase to show how their structures change using three sets of frequencies: monthly, quarterly and annual. For example, the durable consumption follows an ARI(4,1) process at monthly frequency, a random walk process at quarterly and annual frequencies. Here, an IMA(1,1) process competed with a random walk process and only lost ground on the basis of the Schwarz criterion.

Unemployment follows an ARI(24,1) process at monthly frequency, an ARIMA(4,1,4) process at quarterly frequency and a random walk process at annual frequency. For the latter frequency, an IMA(1,1) process was also acceptable but not preferable. Consumer price index follows an ARI(24,1) process at monthly frequency, an ARIMA(4,1,4) process at quarterly frequency and an IMA(1,1) process at annual frequency. For the latter frequency, a random walk process was acceptable but not preferred.

It is worth noting that while the theoretical results are to a greater extent not disputable, the empirical results are a different story as the Box-Jenkins procedure teaches us. Indeed, it is known, for example, the empirical correlogram does not often correspond exactly to the theoretical correlogram due to a certain number of reasons (common roots, etc.). Thus, caution should be exercised when dealing with the structure of either temporally aggregated or systematically sampled data. In addition, the change of structure of time series might possibly, in quite a number of situations, lead to the change of the power of some tests as well as the distortion of the empirical sizes of some statistical tests.

4.2 Alteration of power of tests

In general, aggregation over time brings about a decrease in the power of tests (see for example, Zellner and Montmarquette, 1971). Note that we care about the power of tests because a test with good power enables us to reject the null hypothesis when it is appropriate. The issue of decreasing power of tests statistics under aggregation over time has been documented in the context of cointegration. In this context, at least four questions can be asked (see Mamingi, 2005b):

  1. Do (residual) tests for cointegration preserve their power ranking under aggregation over time?

  2. How do different (residual) based tests for cointegration compare in terms of their powers across the different types of aggregation over time?

  3. Does the degree of integration affect the power of (residual) based tests for cointegration under aggregation over time?

  4. How does the data span affect the power of (residual) based tests for cointegration?

It should be noticed that the word “residual” was set in parentheses to highlight that the analysis can be generalized by concentrating on the power of tests for cointegration in general. That said, a look at the literature reveals that while the first three questions have been systematically examined, this is not the case for the last question. Following the pioneering work of Shiller and Perron (1985) as well as Perron (1987, 1989)) in unit root context, Hakkio and Rush (1991), Mamingi (1992, 2005b)), Hooker (1993), Lahiri and Mamingi (1995), Pierse and Snell (1995), Otero and Smith (2000) and Haug (2002) examined the impact of the data span on the power of tests for cointegration. However, with the exception of Mamingi (1992, 2005b) no study has examined explicitly the issue in the context of the three scenarios of aggregation over time.

The findings with respect to the four earlier mentioned questions are the following (see especially Mamingi, 2005b):

  1. Tests for cointegration do preserve their power ranking, that is, tests that are more powerful than others in the disaggregated model remain so under the aggregated models.

  2. Under local alternatives, the power of tests for cointegration can vary substantially across types of aggregation.

  3. The power of residual-based tests for cointegration is affected by the degree of cointegration. The higher the degree of cointegration, the higher the power of the test.

  4. The data span does affect the power of test statistics through two channels. First, with the same number of observations, the larger the data span, the higher the power. Second, a large data span with a small sample size yields, in general, higher power than a small data span with a large sample size, at least under local alternatives. Nevertheless, the second channel is less present in mixed aggregation as well as with some forms of the ADF test statistic. The power of the ADF test can substantially increase with a large data span. 6

To illustrate the importance of the data span in the context of residual-based tests for cointegration, Pierse and Snell (1995) use the relationship between real non-durable consumption and real net wealth for UK data for different data spans. The residual-based cointegration tests of interest are: the CRDW (cointegration Durbin Watson), the ADF (augmented Dickey-Fuller) and the Zt (Phillips-Ouliaris) test. Table 1 indicates that while for quarterly data (1966–1981) and annual data (1966–1981) the lack of cointegration is not rejected by the ADF and Zttests, the null hypothesis is convincingly rejected with annual data covering 1957 to 1981. This illustrates that boosting the data span increases the power of tests for cointegration.

Table 1:

Tests for the cointegration of UK non-durable consumption and wealth.

In the context of issue (d), Otero and Smith (2000) studied the effects of increasing the frequency of observations and the data span on the Johansen cointegration tests. They found that the power of the tests depend more on the total sample length than the number of observations. To illustrate this theoretical finding, they examined the relationship between long-term and short-term interest rates for the US. More precisely, they considered monthly values of the 3-month treasury-bill-rate in the secondary market (R3) and long-term US government securities (RL) over the 1959–1998 period. They then derived the quarterly and annual versions of the two interest rates by averaging and skip-sampling observations and tested for unit roots using the ADF and PP tests. The series exhibit a unit root at all frequencies. Table 2 presents the cointegration results using the two versions of the Johansen test (maximum eigen-value LR test and LR trace test). The VAR order is chosen by the Schwarz criterion and the constant term is included in the cointegration vector. Irrespective of the frequency of observations, the presence of one cointegration vector is acknowledged with the two longest sample periods (1959–1998 and 1969–1998). There is no evidence of the presence of cointegration between the two types of interest rates when the two shortest data samples are used (1989–1998 and 1979–1998), regardless of the type of data. Similar results are obtained when using systematically sampled observations, though cointegration only appears with annual frequency. This example illustrates the role of the data span in boosting the power of tests for cointegration.

Table 2:

Cointegration between US short-term and long-term interest rates using the johansen tests.

When examining the factors that affect the accuracy of estimation (and, to a larger extent, the power of tests), we can easily understand why the data span is important. As it is well known (see e. g. Koop, 2000 or Mamingi, 2005a), the accuracy of estimation is affected in the first instance by the sample size, meaning that a regression with less observations is less reliable than a regression with a larger number of data. Similarly, the quality of the information content matters. In general, the data span captures the quality of information content. That is, the larger the data span, the better the quality of the information content in principle. In terms of our topic this means that a large data span helps boost the power of tests.

Summing up, the data span, the degree of cointegration, the sample size and the type of aggregation are important determinants of the power of tests for cointegration.

4.3 Distortion of the empirical size of residual-based tests for cointegration under aggregation over time

Most of the known tests are subject to empirical size distortions under aggregation over time. As far as cointegration tests are concerned, it is known at least for residual-based tests for cointegration that these distortions largely occur under temporal aggregation and mixed aggregation. Systematic sampling and the ADF test in general do not cause size distortions. Table 3 below, which concentrates on residual-based tests for cointegration, illustrates this finding. The results are based on the data generating process presented in Appendix B. As it is shown in the table, the size distortions of the test statistics of interest can reach alarming proportions in the context of temporal aggregation and mixed aggregation, at least within the realm of the data generation process used here. For example, at the 0.05 level of significance, the DF test has an empirical size of 0.006 for a sample size of 50 observations under temporal aggregation and of 0.585 under mixed aggregation. On the contrary, it is 0.052 for systematic sampling and also undistorted for the appropriate ADF test. This peculiar result about systematic sampling is due to the fact in theory, systematic sampling tends to preserve the time series structure more than temporal aggregation or mixed aggregation. Thus, as seen above a random walk process systematically sampled remains a random walk. This means that the empirical size remains unchanged. Also, the ADF test largely confirms its good behavior size wise such as pointed out in the literature.

Table 3:

Empirical sizes for 5 % level residual-based tests for cointegration with 3,000 replications.

Overall, the size distortion of residual-based tests for cointegration largely depends on the type of aggregation over time with systematic sampling behaving well and the type of test used, with the ADF test being the most stable one. Especially the size distortion of the Johansen tests for cointegration turns out to be large.

At least two pathways are available to deal with the described size distortions. First, one might simply use the ADF test which has been proven to be well behaved in terms of size distortion. Second, since the issue of size distortion is due to the use of incorrect critical values, generating the correct critical values for a given test is recommended (see Mamingi, 1992).

4.4 Aggregation over time, error correction models and granger causality distortion

The issue of Granger causality behavior is especially important in the context of public policy because here it is often important to detect the correct causality direction between variables. How aggregation over time affects Granger causality is therefore one of the major concerns of this subsection.

Mamingi (1992, 1996, 2006a) studied the impact of aggregation over time on the form of error correction models as well as the causal relationship between variables. Among others, he attempted to answer the following two key questions:

  1. Does aggregation over time alter the form of error correction models?

  2. Does aggregation over time alter the Granger-causality-relationship between cointegrated variables?

Using Monte Carlo experiments and analytical tools, Mamingi (1992, 1996, 2006a) uncovered the following results: (i) as expected, the form of ECMs is often altered under aggregation over time; (ii) there are in general Granger causality distortions under aggregation over time, which depend on the type of aggregation over time, the data span, the sample size and the degree of cointegration. Particularly, the lower the degree of cointegration, the higher the likelihood of distortion (change in causal relationships or ECM form) as well as the higher the level of distortion in the stock-flow relationship (mixed aggregation of type 2) compared to the case of flow-stock relationship. The latter result needs to be analyzed further since economically, the stock-flow relationship is more pervasive than the flow-stock relationship.

Surprisingly, systematic sampling brings about far less Granger causality distortions than the other types of aggregation over time. Thus, there is more concordance of results of Granger causality for systematic sampling for variables which are stationary (Sims, 1971; Cunningham and Vilasuso, 1997, for example) as well as variables which are non-stationary but cointegrated. Gulasekaran (2004) as well as Abeysiinghe and Gulasekaran (2004) showed that while systematic sampling preserves Granger causality with stationary variables, this is not the case with nonstationary variables for which spurious Granger causality (bi-directional causality instead of unidirectional causality) occurs. This issue needs further investigation. In any case, Gulasekaran and Abeysinghe (2003, 2008) devised a sign rule to remedy the distortion of the sign of the adjustment coefficient of an error correction model. By doing so, they claim to uncover the true causal relationship between cointegrated variables.

4.5 Exogeneity

Exogeneity, which is an important issue in the design of policies, is generally influenced by aggregation over time. This is the case for strict and strong exogeneity. In fact, the Lucas critique may be spuriously validated under aggregation over time. Marcellino (1999) delivers a good example for exogenity alteration and other topics discussed earlier. As mentioned earlier, Marcellino (1999) uses the term structure of interest rates to illustrate some of the disadvantages of temporal aggregation. The disaggregated data are monthly observations on the Canadian 10-year government bond yield (RL) and the 90-day deposit rate (RS). The model is a VAR(g) with the vector containing two variables with g lags determined by a recursive F test of significance. The author then studies the effects of different temporal aggregation schemes on exogeneity, Granger non-causality, the presence of common trends, and common cycles, having approximated the aggregated process by a VAR model. According to the results (see Marcellino (1999), Table 4) at the disaggregate level, there is one cointegration relationship with the following vector RSβ1RL+β2 with β1=1 and β2>0. RL is weakly exogenous for the parameter of the cointegration vector. Since the lack of significance of the lags of ΔRSwith Δ as the first difference operator is rejected in the error correction model for ΔRL, RL is not a strongly exogenous variable. The presence of common cycles among the two interest rates is rejected, as well as that of nonsynchroneous common cycles NSCC. In the next step quarterly aggregated variables are constructed using point-in-time scenario (QP) and average (QA) from the corresponding disaggregated form (monthly). The cointegration scheme is uncovered with the same cointegration vector, weak exogeneity of RL is still valid. However, because the lags of ΔRS in the error correction model for ΔRL are insignificant, RL is not Granger-caused by RS, and RL is known to be strongly exogenous for the long-run parameters. Moreover, some non-synchronous common cycles are detected now. For half yearly data, there is still one cointegration vector. RL is no longer weakly exogenous for the long-run coefficients. As implication, RL is no longer strongly exogenous. The number of cofeature 7 vectors does not decrease.

4.6 Aggregation over time and forecasts

Another issue of interest is whether temporally aggregated data has an impact on forecast accuracy. The answer to the question depends on the scenario to be studied. In a first scenario, under the assumption of availability of only temporally aggregated data, it is well documented that while the use of temporally aggregated data generally yields acceptable long-run forecasts, it is not often the case for short-run forecasts. Zellner and Montmarquette (1971), for example, underline the impossibility of making meaningful short-run forecasts with temporally aggregated data. By smoothing series, temporally aggregated data deliver better long-run forecasts as they concentrate on the long-run trend. The second scenario is characterized by the presence of multiple time series data, temporally aggregated at different levels. Here, combining different forecasts derived from these data yields superior forecasts. The burgeoning literature on multiple aggregation prediction (algorithm), MAPA, and mixed data sampling or MIDAS, is at the forefront of forecasting and modeling multiple time series with different frequencies. Athanasopoulos et al. (2015), Kourentzes, Petropoulos, and Trapero (2014), and Petropoulos and Kourentzes (2014) are respectable representatives of MAPA. Guay and Maurin (2015), Bangwayo-Skeete and Skeete (2015), Ghysels and Miller (2014), Ghysels, Santa-Clara, and Valkanov (2004), Miller (2003) and their precursors Zellner and Montmarquette (1971), Hsiao (1979) and Palm and Nijman (1982) are representatives of MIDAS.

5 Agenda for further research

Among the few issues or findings for which there is no firm consensus among researchers, two are particularly important and require further investigation. First, there is the role of data span and sample size in the power and size of tests for cointegration under aggregation over time, particularly questioned by Giles in his blog (GILES, Blogspot, Monday, May 26, 2014). Based on Pierse and Snell (1995, 336) he argues that asymptotically or even in finite samples, “temporally aggregating or selective sampling has no consequence of size distortion or loss of power for the ADF, Phillips-Perron test, or Hall’s (1994) IV based unit root test”. As seen above, quite a number of authors have a view different from GILES’. Second, the role of structural breaks in the unit root/cointegration setting with data with diverse degrees of aggregation over time needs to be explored. The key question is how the power and the size of tests of unit roots/cointegration under aggregation over time are affected by the presence of structural breaks. Moreover, although the variance model was only a footnote here for reasons of choice and space, it would be interesting to study how EGARCH processes behave under aggregation over time.

6 Concluding remarks

This paper dealt with the advantages and problems surrounding data aggregated over time. There are a number of recommendations that can be made concerning the issues discussed in this survey. Ideally, above all, it is advisable to use the data frequency that corresponds to the agent’s decision interval. Since this solution is not always possible, particularly for many developing countries given the high cost of collecting information, at least three recommendations can be made. First, there is a need to use in some situations rules that may re-establish the true properties of the time series or relationships. Thus, the promising research by Gulasekaran and Abeysinghe (2003, 2008) on designing a rule that can “enable” to uncover the “true” relationship in the lower frequency data is, for example, a way forward to solving Granger causality distortions due to temporally aggregated data. Second, in some situations there is the possibility to temporally disaggregate data following some appropriate scheme (Chow-Lin, Fernandez, Litterman, Denton-Cholette, Denton, Lisman-Sandee, etc., see Sax and Steiner, 2013). However, these methods also have their problems because of the lack of knowledge about the data generating process. Third, under certain circumstances there is the possibility to recur to the innovative path which attempts to exploit appropriate techniques that allow the use of both aggregate and disaggregate data at the same time. The bourgeoning literature on MIDAS (mixed data sampling) can provide some insights on solving data configuration issues, at least in the multivariate context.

Summing up, the overall lesson to be learned directly or indirectly from this paper is that in any empirical econometric undertaking it is imperative to understand the issues surrounding the data in use. Failure to examine properly data issues or properties may lead to wrong inferences or possibly wrong public policy prescriptions.

Acknowledgements

I would like to thank the editor-in-chief of this review, his collaborators and Mahalia Jackman for ably editing the paper. I am also indebted to Stephen Harewood for useful comments. All remaining errors are my own.

References

  • Abeysiinghe, T. and R. Gulasekaran (2004): The Consequences of Systematic Sampling on Granger Causality. Econometric Society 2004 Australasian Meetings 250, Econometric Society. Google Scholar

  • Ahsanullah, M. and W. W. S. Wei (1984): The Effects of Time Aggregation of the AR(1) Process, Computational Statistics Quarterly 1, 343–352. Google Scholar

  • Amemiya, T. and R. Y. Wu (1972): The Effect of Aggregation over Prediction in the Autoregressive Model, Journal of the American Statistical Association 67, 628–632. CrossrefGoogle Scholar

  • Athanasopoulos, G., R. J. Hyndman, N. Kourentzes and F. Petropoulos (2015): Forecasting with Temporal Hierarchies. Working Paper 2015: 3, Lancaster University Management School, Working Paper Series. Google Scholar

  • Bangwayo-Skeete, P. and R. W. Skeete (2015): Can Google Data Improve the Forecasting Performance of Tourist Arrivals? Mixed-Data Sampling Approach, Tourism Management 46, 454–464. CrossrefGoogle Scholar

  • Bergstrom, A. R. (1984): Continuous Time Stochastic Models and Issues of Aggregation over Time, in: Z. Griliches and M. D. Intriligator (eds.) Handbook of Econometrics. North Holland, Amsterdam, Vol. 2 (chap 20). Google Scholar

  • Breintung, J. and N. Swanson (2002): Temporal Aggregation and Spurious Instantaneous Causality in Multiple Time Series Models, Journal of Time Series Analysis 23, 651–665. CrossrefGoogle Scholar

  • Brewer, K. R. W. (1973): Some Consequences of Temporal Aggregation and Systematic Sampling for ARMA and ARMAX Models, Journal of Econometrics 1, 133–154. CrossrefGoogle Scholar

  • Campos, J., N. Ericsson and D. F. Hendry (1990): An Analogue Model of Phase-Averaging Procedures, Journal of Econometrics 43, 275–292. CrossrefGoogle Scholar

  • Chambers, M. J. (2005): The Purchasing Power Parity, Temporal Aggregation and Half-life Estimation, Economics Letters 86, 193–198. CrossrefGoogle Scholar

  • Choi, I. and B. S. Chung (1995): Sampling Frequency and the Power of Tests for Unit Root: A Simulation Study, Economics Letters 49, 131–136. CrossrefGoogle Scholar

  • Christiano, L. J. and M. Eichenbaum (1987): Temporal Aggregation and Structural Inference in Macroeconomics, Carnegie-Rochester Conference on Public Policy 26, 63–130. Google Scholar

  • Cunningham, S. R. and R. J. Vilasuso (1997): Time Aggregation and the Money-Real Output Relationship, Journal of Macroeconomics 19, 675–695. CrossrefGoogle Scholar

  • Drost, F. C. and T. E. Nijman (1993): Temporal Aggregation of GARCH Processes, Econometrica 61, 909–927. CrossrefGoogle Scholar

  • Engle, R. F. and C. W. J. Granger (1987): Cointegration and Error Correction: Representation, Estimation and Testing, Econometrica 55, 251–276. CrossrefGoogle Scholar

  • Engle, R. F. and S. Kozicki (1993): Testing for Common Features, Journal of Business and Economic Statstics 11(4), 369–395. Google Scholar

  • Engle, R. F. and T. C. Liu (1972): Effects of Aggregation over Time on Dynamic Characteristics of an Econometric Model, in: B. G. Hickman (ed.) Cyclical Behaviors. Columbia University Press, New York, 663–667. Google Scholar

  • Friend, I. and P. Taubman (1964): A Short-Run Forecasting Model, Review of Economics and Statistics 46, 229–236. CrossrefGoogle Scholar

  • Geweke, J. (1978): Temporal Aggregation in the Multiple Regression, Econometrica 46, 643–661. CrossrefGoogle Scholar

  • Ghysels, E. and I. J. Miller (2014): Testing for Cointegration with Temporally Aggregated and Mixed-frequency Time series, mimeo. Google Scholar

  • Ghysels, E., P. Santa-Clara and R. Valkanov (2004): The MIDAS Touch: Mixed Data Sampling Regressions Model, UNC and UCLA Discussion Paper. Google Scholar

  • Giles, D. E. (2014): The Econometrics of Temporal Aggregation: 1956 – 2014, The A.W.H. Phillips Memorial Lecture, N.Z. Association of Economists Annual Conference, Auckland, July. Google Scholar

  • Granger, C. W. J. (1980): Aggregation of Time Series Variables: A Survey, in: T. Barker and H. Pesaran (eds.) Disaggregation in Econometric Modelling. Routledge, London, 17–34. Google Scholar

  • Granger, C. W. J. and T. H. Lee (1999): The Effect of Aggregation on Nonlinearity, Econometric Reviews 18(3), 259–269. CrossrefGoogle Scholar

  • Granger, C. W. J. and A. J. Morris (1976): Time Series Modelling and Interpretation, Journal of Royal Statistical Society A(139), 246–257. Google Scholar

  • Granger, C. W. J. and P. L. Siklos (1995): Systematic Sampling, Temporal Aggregation, Seasonal Adjustment and Cointegration: Theory and Evidence, Journal of Econometrics 66, 357–369. CrossrefGoogle Scholar

  • Granger, C. W. J. and A. A. Weiss (1983): Time Series Analysis of Error Correction Models, in: S. Karlin, T. Amemiya and L. A. Goodman (eds.) Studies in Econometrics, Time Series, and Multivariate Analysis. Academic Press, New York, 255–278. Google Scholar

  • Guay, A. and A. Maurin (2015): Disaggregation Methods Based on MIDAS Regression, Economic Modelling 50, 123–129. CrossrefGoogle Scholar

  • Gulasekaran, R. (2004): Impact of Systematic Sampling on Causality in the Presence of Unit Roots, Economics Letters 84, 127–132. CrossrefGoogle Scholar

  • Gulasekaran, R. and T. Abeysinghe (2003): Temporal Aggregation, Causality Distortions and a Sign Rule. Departmental Working Paper WP0406, Department of Economics, National University of Singapore. Google Scholar

  • Gulasekaran, R. and T. Abeysinghe (2008): Temporal Aggregation,Cointegration and Causality Inference, Economics Letters 101, 223–226. CrossrefGoogle Scholar

  • Hafner, C. M. (2004) Temporal Aggregation of Multivariate Processes. Econometric Institute, Report 2004-29, Erasmus University Rotterdam, the Netherlands. Google Scholar

  • Haitovsky, Y., G. Treyz and Y. Su (1974): Forecasts with Quarterly Macroeconomic Models. National Bureau of Economic Research, New York. Google Scholar

  • Hakkio, C. S. and M. Rush (1991): Cointegration: How Short Is the Long-Run, Journal of International Money and Finance 10, 571–581. CrossrefGoogle Scholar

  • Hall, A. (1994): Testing for a Unit Root in Time Series with Pretest Data-based Model selection, Journal of Business and Economic Statistics 12, 461–470. Google Scholar

  • Harvey, A. C. (1981): Time Series Model. John Wiley, New York. Google Scholar

  • Haug, A. (2002): Temporal Aggregation and the Power of Cointegration Tests: A Monte Carlo Study, Oxford Bulletin of Economics and Statistics 64, 389–412. Google Scholar

  • Hendry, D. F. (1992): An Econometric Analysis of TV Advertising Expenditure in the United Kingdom, Journal of Policy Modelling 14, 281–311. CrossrefGoogle Scholar

  • Hooker, A. M. (1993): Testing for Cointegration: Power versus Frequency of Observations, Economic Letters 41, 359–362. CrossrefGoogle Scholar

  • Hsiao, C. (1979): Linear Regression Using Both Temporally Aggregated and Temporally Disaggregated Data, Journal of Econometrics 10, 243–252. CrossrefGoogle Scholar

  • Koop, G. (2000): Analysis of Economic Data. John Wiley & Sons, Chichester. Google Scholar

  • Kourentzes, N., F. Petropoulos and J. R. Trapero (2014): Improving Forecasting by Estimating Time Series Structural Components across Multiple Frequencies, International Journal of Forecasting 30(2), 291–302. CrossrefGoogle Scholar

  • Lahiri, K. and N. Mamingi (1995): Testing for Cointegration: Power versus Frequency of Observation: Another View, Economics Letters 49, 121–124. CrossrefGoogle Scholar

  • Lippi, M. and L. Reichlin (1991): Trend-Cycle Decompositions and Measures of Persistence. Does Time Aggregation Matter?, Economic Journal 101, 314–323. CrossrefGoogle Scholar

  • Lütkepohl, H. (1987): Forecasting Aggregate Vector ARMA Processes. Springer-Verlag, New York. Google Scholar

  • Mamingi, N. (1992): Essays on the Effects of Misspecified Dynamics and Temporal Aggregation on Cointegrating Relationships. unpublished Ph.D. thesis, State University of New York, Albany. Google Scholar

  • Mamingi, N. (1993): Residual Based Tests for Cointegration: Their Actual Size under Aggregation over Time. Albany Discussion Papers 93-09, Department of Economics, State University of New York, Albany. Google Scholar

  • Mamingi, N. (1996): Aggregation over Time, Error Correction Models and Granger Causality: A Monte Carlo Investigation, Economics Letters 52, 7–14. CrossrefGoogle Scholar

  • Mamingi, N. (2005a): Theoretical and Empirical Exercises in Econometrics. UWI Press, Kingston. Google Scholar

  • Mamingi, N. (2005b): Power of Tests for Cointegration under Aggregation over Time (Temporal Aggregation, Systematic Sampling and Mixed Aggregation): A Monte Carlo Investigation, Asian-African Journal of Economics and Econometrics 4, 99–115. Google Scholar

  • Mamingi, N. (2006a): Aggregation over Time, Cointegration, Error Correction Models and Granger Causality: An Extension, Asian-African Journal of Economics and Econometrics 6, 171–183. Google Scholar

  • Mamingi, N. (2006b): Empirical Size Distortions of Residual Based Tests for Cointegration under Aggregation over Time: A Monte Carlo Investigation, Asian-African Journal of Economics and Econometrics 6, 13–26. Google Scholar

  • Marcellino, M. (1999): Some Consequences of Temporal Aggregation in Empirical Analysis, Journal of Business and Economic Statistics 17(1), 129–136. Google Scholar

  • Marcet, A. (1987) Temporal Aggregation and Economic Time Series. Unpublished Ph. D. Thesis, University of Minnesota. Google Scholar

  • Miller, J. I. (2003): Mixed-Frequency Cointegrating Regressions with Parsimonious Distributed Lag Structures, Journal of Financial Econometrics 12, 684–615. Google Scholar

  • Moriguchi, C. (1970): Aggregation over Time in Macroeconomic Relationships, International Economic Review 11, 427–440. CrossrefGoogle Scholar

  • Mundlak, Y. (1961): Aggregation over Time in Distributed Lag Models, International Economic Review 2, 154–163. CrossrefGoogle Scholar

  • Otero, J. and J. Smith (2000): Testing for Cointegration: Power versus Frequency of Observation – Further Monte Carlo Results, Economics Letters 67, 5–9. CrossrefGoogle Scholar

  • Palm, F. C. and T. E. Nijman (1982): Linear Regression Using Both Temporally Aggregated and Temporally Disaggregated Data, Journal of Econometrics 19, 333–343. CrossrefGoogle Scholar

  • Perron, P. (1987): Test Consistency with Varying Sampling Frequency. Cahier de Recherche, 4187, Université de Montréal. Google Scholar

  • Perron, P. (1989): Testing for A Random Walk: A Simulation Experiment of Power When the Sampling Interval Is Varied, in: B. Raj (ed.) Advances in Econometrics and Modelling. Kluwer Academic Publisher, Dordretcht. Google Scholar

  • Petropoulos, F. and N. Kourentzes (2014): Improving Forecasting via Multiple Temporal Aggregation, Foresight: the International Journal of Applied Forecasting 34, 12–17. Google Scholar

  • Phillips, A. W. H. (1956): Some Notes on the Estimation of Time-Forms of Reactions in Interdependent Dynamic Systems, Economica 23, 99–113. CrossrefGoogle Scholar

  • Phillips, P. C. B. (1991): Error Correction and Long Run Equilibrium in Continuous Time, Econometrica 59, 967–980. CrossrefGoogle Scholar

  • Pierse, R. G. and A. J. Snell (1995): Temporal Aggregation and the Power of Tests for A Unit Root, Journal of Econometrics 65, 333–345. CrossrefGoogle Scholar

  • Rossana, R. J. and J. J. Seater (1995): Temporal Aggregation and Economic Time Series, Journal of Business and Economic Statistics 13, 441–451. Google Scholar

  • Sax, C. and P. Steiner (2013): Temporal Disaggregation of Time Series, The R Journal 5, 80–87. Google Scholar

  • Shiller, R. J. and P. Perron (1985): Testing the Random Walk Hypothesis: Power versus Frequency of Observations, Economics Letters 18, 381–386. CrossrefGoogle Scholar

  • Silverstrini, A. and D. Veredas (2008): Temporal Aggregation of Univariate and Multivariate Time Series Models: A Survey, Journal of Economic Surveys 22, 458–495. CrossrefGoogle Scholar

  • Sims, C. A. (1971): Discrete Approximation to Continuous Time Distributed Lag in Econometrics, Econometrica 39, 545–563. CrossrefGoogle Scholar

  • Stock, J. H. (1987): Temporal Aggregation and Structural Inference in Macroeconomics: A Comment, In Carnegie Rochester Series on Public Policy 26, 131–140. CrossrefGoogle Scholar

  • Stram, D. O. and W. W. S. Wei (1986): Temporal Aggregation in the ARIMA Process, Journal of Time Series 7, 279–292. CrossrefGoogle Scholar

  • Swanson, N. R. and C. W. J. Granger (1997): Impulse Response Function Based on A Causal Approach to Residual Orthogonalization in Vector Autoregressions, Journal of the American Statistical Association 92, 357–367. CrossrefGoogle Scholar

  • Teles, P. and W. W. S. Wei (2000): The Effects of Temporal Aggregation on Tests of Linearity of a Time Series, Computational Statistics & Data Analysis 34, 91–103. CrossrefGoogle Scholar

  • Telser, L. G. (1967): Discrete Sample and Moving Sums in Stationary Stochastic Processes, Journal of the American Statistical Association 62(318), 484–499. CrossrefGoogle Scholar

  • Theil, H. (1954): Linear Aggregation of Economic Relationship. North-Holland Publishing Company, Amsterdam. Google Scholar

  • Tiao, G. C. (1972): Asymptotic Behavior of Temporal Aggregates of Time Series, Biometrika 59, 525–531. CrossrefGoogle Scholar

  • Tiao, G. C. and W. W. S. Wei (1976): Effect of Temporal Aggregation on the Dynamic Relationship between Two Time Series Variables, Biometrika 63, 513–523. CrossrefGoogle Scholar

  • Wei, W. W. S. (1978): The Effect of Temporal Aggregation on Parameter Estimation in Distributed Lag Model, Journal of Econometrics 8, 237–246. CrossrefGoogle Scholar

  • Wei, W. W. S. (1981): Effect of Systematic Sampling on ARIMA Models, Communication in Statistics, Theory, Math A10(23), 2389–2398. Google Scholar

  • Wei, W. W. S. (1982): The Effect of Systematic Sampling and Temporal Aggregation on Causality: A Cautionary Note, Journal of the American Statistical Association 378, 316–319. Google Scholar

  • Weiss, A. A. (1984): Systematic Sampling and Temporal Aggregation in Time Series Models, Journal of Econometrics 26, 271–287. CrossrefGoogle Scholar

  • Working, H. (1960): Note on the Correlation of First Difference of Averages in a Random Chain, Econometrica 28, 335–342. Google Scholar

  • Zellner, A. and C. Montmarquette (1971): A Study of Some Aspects of Temporal Aggregation Problems in Econometric Analyses, Review of Economics and Statistics 5(3), 335–342. Google Scholar

Appendix

A Proof of cointegration invariance under mixed aggregation

Following Mamingi (2006a), consider the following relationship: yt=βxt+utt=1,2,3,...,N(1)

where yt~I(1)andxt~I(1) and utI(0).whereytI(1),xtI(1)andutI(0). This means that ytandxt are cointegrated with (1β) as the cointegration vector. In addition, without loss of generality, assume that utAR(1)process: (1ρL)ut=et(2)

where 0ρ<1, L is the backward shift operator and et is a white noise series.

Proposition 1. Given the above conditions, the following is true:

  1. the mixed aggregation counterpart of Equation (1) remains cointegrated;

  2. the cointegrating vector,(1β), remains invariant under mixed aggregation.

The proof is adapted from Stram and Wei (1986), Wei (1981), Weiss (1984) and particularly Mamingi (2006a).

Define the following filter P(L)=(M(L)S(L))(3)

where S(L)=(1ρkLk)(1ρL)(4)

M(L)=S(L)S(L)(5)

k is the sampling interval or order of aggregation over time and L is defined as above.

Using Equation (3) in Equation (1) with systematically sampled variables yields P(L)ykTβxkT=P(L)ukTukT(6)

where T is the time index of aggregated variables. Equation (6) can be rewritten as follows: (M(L)S(L))ykTβxkT=(M(L)S(L))ukTukT(7)

Expanding Equation (7) yields: M(L)ykTβS(L)xkT=M(L)ukT+S(L)ukT(8)

Multiplying Equation (8) by (1ρL) gives rise to (1ρL)M(L)ykTβ(1ρL)S(L)xkT=(1ρL)M(L)ukT+(1ρL)S(L)ukT(9)

Part (a)

At the outset, we develop the right-hand side of Equation (9) to prove Part (a) of the proposition. Rewriting the right-hand side of Equation (9) and using Equations (4) and (5) yield (1ρL)M(L)ukT+(1ρL)S(L)ukT=(1ρkLk)S(L)ukT+(1ρkLk)ukT(10)

Inserting Equation (2) into Equation (10) yields (1ρL)M(L)ukT+(1ρL)S(L)ukT=M(L)ekT+S(L)ekT(11)

Combining Equations (10) and (11) gives rise to (1ρkLk)S(L)ukT+(1ρkLk)ukT=M(L)ekT+S(L)ekT(12)

or

(1ρkLk)(S(L)ukT+ukT)=M(L)ekT+S(L)ekT(13)

The left-hand side of Equation (13) is simply UTρkUT1, where UT consists of temporally aggregated and systematically sampled parts, S(L)ukT and ukT, respectively. Mamingi (2005b) has shown that M(L)ekT is an MA(1) process and S(L)ekT is a white noise series. It is known that the sum of an MA(1) process and a white noise process is an MA(1) process (see, for example, Granger and Morris, 1976). Hence, the mixed aggregated counterpart of ut, that is, UT, follows an ARMA(1,1) process. It means that the error remains stationary (I(0)). Thus, cointegration continues to hold with this type of aggregation over time. Q.E.D.

Part (b)

Using the left-hand side of Equation (9) as well as Equations (4) and (5) yields (1ρkLk)S(L)ykTβ(1ρkLk)xkT=(1ρkLk)S(L)ukT+(1ρkLk)ukT(14)

where S(L)ykT=YT is the temporally aggregated counterpart of yt and xkT=XT is the systematically sampled counterpart of xt.

Dividing Equation (14) by (1ρkLk) yields S(L)ykT=βxkT+UT(15)

or YT=βXT+UT(16)

where UT=S(L)ukT+ukT is an ARMA(1,1) process as shown in Part (a). Cointegration is thus preserved with the same cointegrating vector (1β). QED

The proof also holds if the roles of the variables are interchanged. An alternative proof can be found in Stock (1987).

B DGP for Table 3 (see Mamingi, 2006b)

The data generation process (DGP) due to Engle and Granger (1987) is of interest mainly for reasons of comparability with numerous studies that used it. It is defined as follows: yt+axt=w1t,w1t=w1t1+ε1tyt+βxt=w2t,w2t=ρw2t1+e2t(17)

where ytand xt are the variables of interest and the w’s are the error terms, the εs are iid(0,1) and ρ<1.

The reduced form of system (17) shows that yt and xt are individually integrated of order one: xt=1/αβw2tw1tyt=1/αβαw1tβw2t(18)

As in Engle and Granger (1987), α=1and β=2. Under the alternative hypothesis of cointegration, the coefficient of autocorrelation ρ in Equation (17) is ρ<1.

The disaggregated model is of the following type: yt=c+axt+utt=1,2,,N(19)

where the two variables of interest are those from Equation (18), and t=1,2,3, …,N. Note that while under the null hypothesis of no cointegration utw1t, under the alternative hypothesis of cointegration utw2t. Thus, under H0,utfollows a random walk process and under H1, it follows an AR(1) process

The aggregated model, analogous to Equation (19), is: YT=c+aXT+UTT=1,2,,M(20)

where T=k t is the time index in aggregated models, k is the sampling interval or order of aggregation and capital letters stand for aggregated variables (see the text for details of types of aggregation over time). For implementation, see Mamingi (2006b).

Footnotes

  • 1

    Theil (1954), Granger (1990) and Silverstrini and Veredas (2008) are among the few exceptions. 

  • 2

    I have just, while browsing the internet, come across a conference paper by Giles (2014) entitled “The Econometrics of Temporal Aggregation.” I acknowledge it. 

  • 3

    A time series is stationary in the second order sense if its mean does not depend on time, its variance is finite and its covariance depends only on the distance between variables. 

  • 4

    Empirical size distortion means that the actual level of significance is different from the nominal level of significance. 

  • 5

    The Dickey-Fuller test (the Engle-Granger test), the Augmented Dickey-Fuller test (the Augmented Engle-Granger test) and the Phillips-Ouliaris test. 

  • 6

    In the context of unit root, Choi and Chung (1995) emphasise sample size rather than data span in boosting the power of the ADF test. This calls for further investigation. 

  • 7

    Cofeature means common features or the simultaneous presence of common trend and common cycles. In fact, in many instances, it simply means common cycles (see Engle and Kozicki, 1993). 

About the article

Published Online: 2017-11-30

Published in Print: 2017-11-27


Citation Information: Review of Economics, Volume 68, Issue 3, Pages 205–227, ISSN (Online) 2366-035X, ISSN (Print) 0948-5139, DOI: https://doi.org/10.1515/roe-2017-0027.

Export Citation

© 2017 Oldenbourg Wissenschaftsverlag GmbH, Published by De Gruyter Oldenbourg, Berlin/Boston. Copyright Clearance Center

Comments (0)

Please log in or register to comment.
Log in