To ensure that the calculations for each iteration are valid, it must be confirmed that ${\mathcal{D}}_{t}({v}_{1},{v}_{2})={\rho}_{t}^{a,b}({v}_{1}){\rho}_{t}^{a,b}({v}_{2})$is true at each step (here *v*_{1} *∈ S*^{a} and *v*_{2} *∈ S*^{b}). To show this, we use mathematical induction, assume that this is true for round *t*, and when it comes to round *t* + 1, according to the produces, we have

$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& {\mathcal{D}}_{t}({v}_{1},{v}_{2})\\ & =& \frac{{\rho}_{t}^{a,b}(v){e}^{-{\alpha}_{t}^{a,b}{f}_{t}(v)}}{\sum _{{v}^{\prime}\in {S}^{a}}{\rho}_{t}^{a,b}({v}^{\prime}){e}^{-{\alpha}_{t}^{a,b}{f}_{t}({v}^{\prime})}}\\ & \phantom{\rule{1em}{0ex}}& \times \frac{{\rho}_{t}^{a,b}(v){e}^{{\alpha}_{t}^{a,b}{f}_{t}(v)}}{\sum _{{v}^{\prime}\in {S}^{b}}{\rho}_{t}^{a,b}({v}^{\prime}){e}^{-{\alpha}_{t}^{a,b}{f}_{t}({v}^{\prime})}}\\ & =& {\rho}_{t}^{a,b}({v}_{1}){\rho}_{t}^{a,b}({v}_{2}).\end{array}$$Note that our final ontology function has the form $f(v)=\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}(v),$and we can set *Θ* : *V* × *V → {*−1, 0, 1} as

$$\mathrm{\Theta}({v}_{1},{v}_{2})=\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}(\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{1})-\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{2})).$$That is to say, if $\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{1})-\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{2})>0$then *Θ*(*v*_{1}, *v*_{2}) = 1; if $\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{1})=\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{2}),$then *Θ*(*v*_{1}, *v*_{2}) = 0; and if$\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{1})-\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}_{2})<0$then *Θ*(*v*_{1}, *v*_{2}) = −1. For each pair of (*a*, *b*) with 1 ≤ *a* < *b* ≤ *k*, if function *Θ*(*v*^{a}, *v*^{b}) ≠1 where *v*^{a} ∈ S^{a} and *v*^{b} ∈ S^{b}, then it implies the error by the multi-dividing rule. Thus, the generalization error (expect risk) of *Θ* in multi-dividing ontology setting is denoted as

$$\begin{array}{rcl}\mathrm{\Delta}(\mathrm{\Theta})& =& \sum _{a=1}^{k-1}\sum _{b=a+1}^{k}{\mathbb{P}}_{v\sim {\mathcal{D}}_{a},{v}^{\prime}\sim {\mathcal{D}}_{b}}\{\mathrm{\Theta}(v,{v}^{\prime})\ne 1\}\\ & =& \sum _{a=1}^{k-1}\sum _{b=a+1}^{k}{\mathbb{E}}_{{\mathcal{D}}_{a},{\mathcal{D}}_{b}}\mathrm{I}(\mathrm{\Theta}(v,{v}^{\prime})\ne 1),\end{array}$$where I is the truth function, i.e., I(*x*) = 1 if *x* is true, otherwise I(*x*) = 0. Given the ontology training set *S* = (*S*^{1}, *S*^{2}, · · · , *S*^{k}) *∈ V*^{n1} × *V*^{n2} × ··· × *V*^{nk} which consists of a sequence of ontology training samples ${S}_{a}=({v}_{1}^{a},\cdots ,{v}_{{n}_{a}}^{a})\in {V}^{{n}_{a}}\phantom{\rule{thinmathspace}{0ex}}(1\le a\le k),$the expected empirical error of *Θ* can be denoted as

$$\hat{\mathrm{\Delta}}(\mathrm{\Theta})=\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\frac{1}{{n}_{a}{n}_{b}}\sum _{i=1}^{{n}_{a}}\sum _{j=1}^{{n}_{b}}\mathrm{I}(\mathrm{\Theta}({v}_{i}^{a},{v}_{j}^{b})\ne 1).$$The results presented in our paper aim to show that the difference between$\hat{\mathrm{\Delta}}(\mathrm{\Theta})\phantom{\rule{thinmathspace}{0ex}}and\phantom{\rule{thinmathspace}{0ex}}\mathrm{\Delta}(\mathrm{\Theta})$is small with large possibility. Setting *Γ* as the function space for functions *Θ* we have the following theorem.

**Theorem 1** Suppose all the weak ontology functions belong to function space F^{′} with a finite VC dimension *K*, the ontology functions *f* (as the weighted combinations of the weak ontology functions) belong to function space F. Let *S* = (*S*^{1}, *S*^{2}, · · · , *S*^{k}) *∈ V*^{n1} × *V*^{n2} × ·· ·× *V*^{nk} be ontology training set which consists of a sequence of ontology training samples ${S}^{a}=({v}_{1}^{a},\cdots ,{v}_{{n}_{a}}^{a})\in {V}^{{n}_{a}}$and *S*^{a} ∼ D_{a} (1 ≤ *a* ≤ *k*). We have with probability at least 1 − *δ* (0 < *δ* < 1), the following inequality holds for any *f ∈* F:

$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& |er(f)-\hat{er}(f)|\\ & \le & 2\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\{\sqrt{\frac{{K}^{\prime}(\mathrm{log}\frac{2{n}_{a}}{{K}^{\prime}}+1)+\mathrm{log}\frac{18}{\delta}}{{n}_{a}}}\\ & \phantom{\rule{1em}{0ex}}& +\sqrt{\frac{{K}^{\prime}(\mathrm{log}\frac{2{n}_{b}}{{K}^{\prime}}+1)+\mathrm{log}\frac{18}{\delta}}{{n}_{b}}}\},\end{array}$$where *K*^{′} = 2(*K* + 1)(*T* + 1) log_{2}(*e*(*T* + 1)), *T* is the number of weak ontology functions in ontology algorithm and *e* is the base of the natural logarithm.

**Proof of Theorem 1** First, we show that for each pair of (*a*, *b*) with 1 ≤ *a* < *b* ≤ *k*, and each *δ* > 0, there is a small number *ε* satisfying

$$\mathbb{P}\{\mathrm{\exists}\mathrm{\Theta}\in \mathrm{\Gamma}:|\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\frac{1}{{n}_{a}{n}_{b}}\sum _{i=1}^{{n}_{a}}\sum _{j=1}^{{n}_{b}}\mathrm{I}(\mathrm{\Theta}({v}_{i}^{a}-{v}_{j}^{b})\ne 1)$$$$-\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}{\mathbb{E}}_{{v}^{a},{v}^{b}}\mathrm{I}(\mathrm{\Theta}({v}^{a},{v}^{b})\ne 1)|>\epsilon \}\le \delta ,$$where the value of *ε* will be determined later.

Define $\mathrm{\Xi}:V\times V\to \{0,1\}\phantom{\rule{thinmathspace}{0ex}}as\phantom{\rule{thinmathspace}{0ex}}\mathrm{\Xi}({v}^{a},{v}^{b})=\mathrm{I}(\mathrm{\Theta}({v}^{a},{v}^{b})\ne 1)$. Clearly, $\mathrm{\Xi}$indicates whether *Θ* makes mistake or not for the ontology vertices pair (*v*^{a}, *v*^{b}) for *v*^{a} ∈ S^{a} and *v*^{b} ∈ S^{b} according to the multi-dividing rule. We infer

$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& \sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\frac{1}{{n}_{a}{n}_{b}}\sum _{i=1}^{{n}_{a}}\sum _{j=1}^{{n}_{b}}\mathrm{I}(\mathrm{\Theta}({v}_{i}^{a}-{v}_{j}^{b})\ne 1)\end{array}$$$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& -\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}{\mathbb{E}}_{{v}^{a},{v}^{b}}\mathrm{I}(\mathrm{\Theta}({v}^{a},{v}^{b})\ne 1)\\ & =& \sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\{\frac{1}{{n}_{a}{n}_{b}}\sum _{i=1}^{{n}_{a}}\sum _{j=1}^{{n}_{b}}\mathrm{\Xi}({v}_{i}^{a},{v}_{j}^{b})\\ & \phantom{\rule{1em}{0ex}}& -\frac{1}{{n}_{a}}\sum _{i=1}^{{n}_{a}}{\mathbb{E}}_{{v}^{b}}\{\mathrm{\Xi}({v}_{i}^{a},{v}^{b})\}+\frac{1}{{n}_{a}}\sum _{i=1}^{{n}_{a}}{\mathbb{E}}_{{v}^{b}}\{\mathrm{\Xi}({v}_{i}^{a},{v}^{b})\}\\ & \phantom{\rule{1em}{0ex}}& -{\mathbb{E}}_{{v}^{a},{v}^{b}}\{\mathrm{\Xi}({v}_{i},{v}_{j})\}\}\\ & =& \sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\{\frac{1}{{n}_{a}}\sum _{i=1}^{{n}_{a}}(\frac{1}{{n}_{b}}\sum _{j=1}^{{n}_{b}}\mathrm{\Xi}({v}_{i}^{a},{v}_{j}^{b})\\ & \phantom{\rule{1em}{0ex}}& -{\mathbb{E}}_{{v}^{b}}\mathrm{\Xi}({v}_{i}^{a},{v}^{b}))+{\mathbb{E}}_{{v}_{b}}\{\frac{1}{{n}_{a}}\sum _{i=1}^{{n}_{a}}\mathrm{\Xi}({v}_{i}^{a},{v}^{b})\\ & \phantom{\rule{1em}{0ex}}& -{\mathbb{E}}_{{v}^{a}}\mathrm{\Xi}({v}^{a},{v}^{b})\}\}.\end{array}$$Obviously, it is enough to show that there exist *ε*_{1} and *ε*_{2} with *ε*_{1} + *ε*_{2} = *ε* such that (*∃v*^{a} ∈ V ^{a} in each pair of (*a*, *b*) for (1) and *∃v*^{b} ∈ V^{b} in each pair of (*a*, *b*) for (2))

$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& \mathbb{P}\{\mathrm{\exists}\mathrm{\Xi}\in \mathrm{{\rm Y}}:|\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\frac{1}{{n}_{b}}\sum _{j=1}^{{n}_{b}}\mathrm{\Xi}({v}^{a},{v}_{j}^{b})\\ & \phantom{\rule{1em}{0ex}}& -\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}{\mathbb{E}}_{{v}^{b}}\mathrm{\Xi}({v}^{a},{v}^{b})|\ge {\epsilon}_{1}\}\le \frac{\delta}{2},\end{array}$$(1)and

$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& \mathbb{P}\{\mathrm{\exists}\mathrm{\Xi}\in \mathrm{{\rm Y}}:|\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\frac{1}{{n}_{a}}\sum _{i=1}^{{n}_{a}}\mathrm{\Xi}({v}_{i}^{a},{v}^{b})\\ & \phantom{\rule{1em}{0ex}}& -\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}{\mathbb{E}}_{{v}^{a}}\mathrm{\Xi}({v}^{a},{v}^{b})|\ge {\epsilon}_{2}\}\le \frac{\delta}{2}\end{array}$$(2)respectively, where *Υ* is the function space of *Ξ*.

Now, we only prove (2) in light of standard results, and (1) can be yielded in the same fashion. Let *Υ*_{vb} be the set of all such functions *Ξ* for a given *v*^{b}, then the selection of *Ξ* in (2) is from function space *∪*_{vb}Υ_{vb} . In view of theorem of Vapnik [32] which provides a selection of *ε*_{2} in (2) relying on the size *n*_{a} of *S*^{a} for each pair of (*a*, *b*), complexity *K*^{′} of *∪*_{vb}Υ_{vb} (considered as VC Dimension), and the possibility *δ*. Specifically, for any *δ* > 0, set

$${\epsilon}_{3}=2\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\sqrt{\frac{{K}^{\prime}(\mathrm{log}\frac{2{n}_{a}}{{K}^{\prime}}+1)+\mathrm{log}\frac{18}{\delta}}{{n}_{a}}},$$we have

$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& \mathbb{P}\{\mathrm{\exists}\mathrm{\Xi}\in {\cup}_{{v}^{b}}{\mathrm{{\rm Y}}}_{{v}^{b}}:|\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\frac{1}{{n}_{a}}\sum _{i=1}^{{n}_{a}}\mathrm{\Xi}({v}_{i}^{a},{v}^{b})\\ & \phantom{\rule{1em}{0ex}}& -\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}{\mathbb{E}}_{{v}^{a}}\mathrm{\Xi}({v}^{a},{v}^{b})|\ge {\epsilon}_{3}\}\le \delta .\end{array}$$Next, we need to determine the VC Dimension of *∪*_{vb}Υ_{vb} : *K*^{′}. For a given *v*^{b} ∈ V^{b}, we obtain

$$\begin{array}{rcl}& \phantom{\rule{1em}{0ex}}& \mathrm{\Xi}({v}^{a},{v}^{b})=\mathrm{I}(\mathrm{\Theta}({v}^{a},{v}^{b})\ne 1)\\ & =& \mathrm{I}(\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}(\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}^{a})-\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}^{b}))\ne 1)\\ & =& \mathrm{I}(\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}^{a})-\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}^{b})\ge 0)\\ & =& \mathrm{I}(\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}^{a})-c)\ge 0)\end{array}$$where $c=\sum _{t=1}^{T}{\alpha}_{t}{f}_{t}({v}^{b})$is a constant since *v*^{b} is given. It reveals that the functions in the space *∪*_{vb}Υ_{vb} are the subset of all potential thresholds of all the linear combination of *T* ontologyweak functions. Using the standard result on VC Dimension of weak functions, we yield that if the ontology weak function space has VC Dimension *K* bigger than two, then *K*^{′} can’t exceed to 2(*K* + 1)(*T* + 1) log_{2}(*e*(*T* + 1)).

Therefore, we get the desired conclusion.

According to Theorem 1 above, the generalization bound converges to zero at a rate of $O(\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}max\{\sqrt{\frac{\mathrm{log}({n}_{a})}{{n}_{a}}},\sqrt{\frac{\mathrm{log}({n}_{b})}{{n}_{b}}}\}).$

For each pair of (*a*, *b*) with 1 ≤ *a* < *b* ≤ *k*, the shatter coefficient is denoted as *r*^{a}^{,b}(F, *n*_{a}, *n*_{b}) (see Gao and Wang [33] for more details). Then we deduce the following result. **Theorem 2** Let F be the real valued ontology function space on *V*, then with probability at least 1 − *δ* (0 < *δ* < 1), for any *f ∈* F, we have

$$\begin{array}{c}|er(f)-\hat{er}(f)|\le \\ \sum _{a=1}^{k-1}\sum _{b=a+1}^{k}\sqrt{\frac{8({n}_{a}+{n}_{b})(\mathrm{log}\frac{4}{\delta}+\mathrm{log}{r}^{a,b}(\mathcal{F},2{n}_{a},2{n}_{b}))}{{n}_{a}{n}_{b}}}.\end{array}$$Theorem 2 implies that if the ontology function is a linear function in the one-dimensional function space, then for each pair of (*a*, *b*) with 1 ≤ *a* < *b* ≤ *k*, *r*^{a}^{,b}(F, *n*_{a}, *n*_{b}) are constants, regardless of the values of *n*_{a} and *n*_{b}, and thus the bound converges to zero at a rate of $O(\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}max\{\frac{1}{\sqrt{{n}_{a}}},\frac{1}{\sqrt{{n}_{b}}}\}).$It further reveals that the bound yield in Theorem 2 is sharper than bound obtained in Theorem 1. However, if the ontology function is a linear function in the *d*-dimensional function space (where *d* ≥ 2), then *r*^{a}^{,b}(F, *n*_{a}, *n*_{b}) with order *O*((*n*_{a}n_{b})^{d}), and in this case the bound in Theorem 2 has convergence rate relying on VC dimension, i.e., still $O(\sum _{a=1}^{k-1}\sum _{b=a+1}^{k}max\{\sqrt{\frac{\mathrm{log}({n}_{a})}{{n}_{a}}},\sqrt{\frac{\mathrm{log}({n}_{b})}{{n}_{b}}}\}).$

## Comments (0)

General note:By using the comment function on degruyter.com you agree to our Privacy Statement. A respectful treatment of one another is important to us. Therefore we would like to draw your attention to our House Rules.