What is Gained from Past Learning

1 Introduction

Assume that we have learned a certain relation R in environment that is governed by a probability function P. Now the environment changes and P turns into P∗. We would still like to estimate R, but in the new environment, P∗. The basis of much works on “Transfer Learning,” “Robust Learning,” “Domain Adaptation,” and “Life Long Learning” (L2L) hinges on the intuition that it would be a great waste to start learning R(P∗) from scratch, instead of amortizing what we learned in P.

This intuition assumes, of course, that the two environments share some features in common, and that the shared features are significant in determining R. Surely, if the two environments are totally different, then we might as well start learning things from scratch – there is simply no other choice. Similarly, if the target relation R is defined exclusively on the novelty part of P∗, no advantage would be realized by transferring what was learned in P.

To anchor this intuition in a formal setting^[1] let us assume that the target relation R can be decomposed into a set S of sub-relations, and that these sub-relations fall into two categories:

SA

– sub-relations on which P and P∗ agree, and

SD

– sub-relations on which P and P∗ disagree.

One obvious saving that can be realized from knowing SA is in learning time. If we have trained the learner on 100 cases from each distribution, we can estimate SA using all 200 samples, and SD using the 100 samples of P∗. The net result being that some portions of R receive extra samples, which render them more precise, thus making the estimate of R more precise (i. e., less susceptible to sampling bias). Conversely, if we aspire to achieve a given precision in R, less samples, or shorter learning time, would be realized overall.

A simple example can illustrate this logic.

This simple example raises a fundamental question: Is it always beneficial to decompose a relation into components, estimate each component individually, some with improved precision, then recombine the results?

A competing intuition might claim that the exercise of decomposing, estimating, and combining introduces new sources of noise, compared to, say, estimating the relation in one shot.

The question is further complicated by the fact that decompositions are not unique. Eq. (1), for example can also be written as:

This calls for refraining from learning P∗(y) directly in the new environment, but estimating P∗(y|x) and P∗(y|x′) for all x′ at the new environment, then averaging the results to get a composite estimate of P∗(y) as shown in the denominator.

It is not at all clear that the refinement offered by the denominator of (2) would improve precision over the estimator defined in (1). Assume for example that Y is a single binary variable, whereas X is a vector of continuous variables. Decomposing the P(y) as in the denominator of Eq. 1 would entail estimating all factors P(y|x′) and averaging the estimates. Estimating P(y) from scratch, in contrast, may offer definite advantages, despite the fact that we have not borrowed any information from P.

We thus ask the following questions:

1. Given a relation R, which of its decompositions gains by borrowing and which does not?

2. Which relations R have a beneficial decomposition and which do not?

3. Given that borrowing is beneficial, can we quantify the benefit?

2 The Transfer Benefit Ratio (TBR)

To get a theoretical handle on the problem, let us take the simple problem of estimating the regression coefficient τ of Y on X in the chain model of Fig. 1.

Figure 1

A chain model where b changes and a remains the same.

Here b is the regression coefficient of Y on Z,

and, based on the chain structure:

Let us assume that we estimate a and b using Ordinary Least Square (OLS) on a large number (N1) of cases from P. Now b changes to b∗, while a remains the same. How are we to estimate τ in P∗, if we can draw only a small number of samples (N2) in the new environment?

We have two options:

1. We ignore the estimates obtained in the training environment and estimate τ from scratch, obtaining

   τˆ=∑i=1nxiyi∑i=1nxi2.

2. We estimate a and b separately, and multiply their estimates, with a receiving samples from both environments and b from P∗ only.

The ratio of the asymptotic variances of these two estimators will measure the merit of transferring knowledge from one environment to another, and will be called here the Transfer Benefit Ratio (TBR).

This measure translates directly to improvement in the learning speed. When TBR is high, a small number of cases (N2) in the novel environment would be sufficient to achieve a given precision, whereas a low TBR would require a high number of cases to achieve such precision.

Intuitively, the benefit of transfer would be more pronounced when the part shared by the two environments is noisy and the novel part is noiseless. Under such conditions, assessments of the target quantity τ are highly vulnerable to inaccuracies in estimating the relation between X and Z, and it is here that the training conducted in P can be most beneficial.

Exact analysis (see Appendix I) reveals that, for N2≪N1, the TBR is given by the following formula

where ρa2 and ρb2 are the squared correlation coefficients

Equation (3) quantifies the intuition that transfer learning is more beneficial when the novelty between the two environments is almost deterministic (ρb approaches 1) so that the few observations conducted in the new environment would suffice to complete the adaptation.

Appendix I generalizes this result to any N1/N2 ratio and presents 3-dimensional charts of how the TBR varies with both the N1/N2 ratio and the statistics of X,Y, and Z. Remarkably, it shows that TBR is greater than unity even for N1=N2. This means that there is benefit to the two-step estimation of τ (using the product aˆbˆ over the single step estimator τˆ, even when the environment does not change and we are faced with the problem of estimating τ given the chain model of Fig. 1. This phenomenon reflects a more general pattern in estimation: proper utilization of modeling assumptions can improve estimation efficiency, provided those assumptions are valid [3], [4].

Clearly, this exercise is oversimplified in that it assumes just two linear relationships X→Z and Z→Y one invariant and one novel. Yet, such rudimentary analysis must be conducted to understand the speed-up provided by prior learning, the factors that determine this speed-up, and how to optimize those factors.

In more realistic situations, it is not at all clear that a speed-up would be achieved regardless of the problem structure. In our example, we capitalized on the chain structure, which rendered X and Y conditional independent given Z. Under such conditions, the product estimator is superior to the one-shot estimator even when no environmental change takes place (i. e., N1=N2). On the other hand, when N1≫N2, the benefit of transfer learning is realized even in the absence of independence constraints.

We have so far not considered the possibility of minimizing the number of variables needed to be measured in the new environment. Cases exist where, despite differences between P and P∗, R can be estimated entirely in the source environment, without taking any measurements in P∗. In other cases, some measurements in the new environments are needed, but the number of variables involved can be minimized by proper design [1].

3 Conclusions

We have demonstrated by simple examples that it is possible to quantify the benefit of borrowing information from previous learning, and that this benefit depends on the structure of the data generating model. This leaves open the general question of deciding, for any given relation R, how can it best benefit from previous learning, and how robust can it be to changes in the target environment? We conjecture that the understanding of such theoretical questions is necessary for designing algorithms that take maximum advantage of previous learning and spend minimum resources on re-learning that which could be borrowed.