Investigation of space-continuous deformation from point clouds of structured surfaces

: One approach to estimate space-continuous deformation from point clouds is the parameter-based epochal comparison of approximating surfaces. This procedure allows a statistical assessment of the estimated deformations. Typically, holistic geometric models approximate the scanned surfaces. Regarding this, the question arises on how discontinuities of the object’s surface resulting from e.g. single bricks or concrete blocks, inﬂuence the parameters of the approximating continuous surfaces and in further consequence the derived deformation. This issue is tackled in the following paper. B-spline surfaces are used to approximate the scanned point clouds. The approximation implies solving a Gauss–Markov-Model, thus allowing accounting for the measurements’ stochastic properties as well as propagating them on the surfaces’ control points. A parametric comparison of two B-spline surfaces can be made on the basis of these estimated control points. This approach is advantageous with regard to the transition of the space-continuous deformation analysis to a point-based task, thus ensuring the applicability of the well-established congruency model. The inﬂuence of the structure’s geometry on the surfaces’ control points is investigated using terrestrial laser scans of a clinker facade. Points measured in the joints are eliminated using an own developed segmentation approach. A comparison of the results obtained from segmented as well as from unsegmented laser scans for the B-spline approximation and the subsequent deformation analysis provides


Introduction
Monitoring artificial objects like dams or bridges is a crucial safety task.The increasing use of continuously improving laser scanners demands ongoing development and investigation of methods to detect space-continuous deformations.This paper shall contribute to those investigations and introduces an approach belonging to the category of parameter-based point cloud comparisons.It comprises the approximation of the scanned point clouds with B-spline surfaces adopting in each epoch the same number of control points and knot vectors.With this condition, the B-spline approximation processes of both epochs can be expressed with a linear Gauss-Markov-Model (GMM), wherein solely the control points of the B-spline surfaces are unknowns.Herein a stochastic model based on error propagation of the variances within the measuring configuration and the instrument's precisions is incorporated.The final evaluation of deformation consists in the geometrical comparison of the estimated control points of the approximating B-spline surfaces in the classical congruency model [1].An identified change of a control point indicates a local space-continuous deformation within the B-spline surface.A validation will take place with reference to deformation identified by classic terrestrial measurements including stable-and object-points.The major subject tackled within the newly investigated approach is the influence of the scanned object's structure on the deformation results.Therefore, a segmentation approach is applied to the point clouds that separates regular patterns using an extended region growing approach.In previous related work, including [2][3][4][5] or [6], space-continuous deformation of different masonry structures are investigated by applying different approaches of deformation analysis: Paffenholz et al. [4] applied a geometry-based point cloud comparison incorporated in the M3C2-algorithm to derive deformations of a bridge under load.In Paffenholz and Wujanz [5] this approach is compared to other commercial point cloud comparison methods.The same specimen was used by [3] applying a parameter-based deformation analysis.Here the scanned point clouds are approximated by B-spline surfaces.The derived deformation is based on the calculated Hausdorff-Distance between points of the approximating surfaces.Alba et al. [2] investigated the deformation of a large dam applying point cloud-based comparison methods like mesh-to-mesh or polynomial surface approximations to resampled point clouds.Kalenjuk et al. [6] conducted a parameter-based deformation monitoring by comparing the orientation of normal vectors of fitted planes in scanned concrete panels of retaining walls.The method introduced in this paper is applied and evaluated on terrestrial laser scan data of an aqueduct arc of the Viennese "Hochquellwasserleitung" (see Figure 1).It shows the structure of a brick wall enhancing geometrical differences in the bricks and the joints.The aqueduct contains a gravity channel dimensioned by <1.5 m width and <2 m height.
The volume of floating water comprises about 1.9 m 3 of water per meter, which corresponds to a uniform load of 1.9 t/m.Every three months, the pipes are cleaned within a procedure called "Abkehr".At this process the drinkable water is impounded at a reservoir and the channel is released from the water weight.In this context, the question arises whether deformation resulting from the absence of mechanical loads occurs and whether it can be detected in a statistically sound way.A validation of the space-continuous deformation is performed using a point-based deformation analysis.Here targets installed on top of the aqueduct arc are monitored in the same temporal intervals as the point clouds are scanned.
The paper is structured as follows: In chapter 2, the methodological principles utilized within this approach are introduced, followed by the description of the conducted measurement campaigns in chapter 3.In chapter 4 the results are presented.In chapter 5 the results are interpreted and discussed.A summary and outlook on further issues that shall be investigated beyond the contents of this paper is given in chapter 6.

Segmentation: extended region growing approach
B-spline approximations incorporating knot vectors whose internal knots are of multiplicity one, demand point clouds with fairly regular point distribution in order to circumvent local singularities [7].Within the point clouds of the investigated measuring object the geometrical differences between the bricks and the joints cause discontinuities that might influence the quality of the approximating B-spline surfaces and further the derived deformations.Therefore, points within joints are discharged within the segmentation process, which leads to only small, negligible gaps between the bricks.The used approach for the segmentation is based on an initial region growing (RG) [8] solution incorporating two conditions (see Equation ( 1)): on the one hand the difference between intensity values of neighboring points and on the other hand the scalar product between normal vectors of neighboring points.Red-Green-Blue (RGB) values are not incorporated because they were not available within the scanning procedures.where This initial result is generated using Opals [9] and ideally ascribes points of one brick to one segment.Insufficiencies occur on the one hand as points within a segment are excluded due to intensity fluctuation or on the other hand as points lying in joints are mistakenly included in segments and furthermore, several bricks are incorporated into one segment.To eliminate these insufficiencies, an improving approach is applied on the initial RG-segmentation solution with the inclusion on information about the dimensions and the orientation of the immured bricks.Here the latter cases are remedied by excluding segments with significantly higher number of points and a large geometrical extent.
The former cases are tackled by fitting bounding boxes of known extent and orientation into the segments.Therefore, the Eigenvalues and Eigenvectors of a segment (resulting from RG) reduced by its centroid are calculated using the Principal Component Analysis (PCA) [10].The direction of the eigenvector corresponding to the smallest eigenvalue points into the direction orthogonal to the scanned bricks.The eigenvector connected to the largest Eigenvalue points into the direction of the longest side of a brick.The cross-product of these two Eigenvectors completes a threedimensional coordinate system.A transformation of this coordinate system into a local up-right coordinate system with its center in the centroid of a segment states the next step of the approach.In a following step, a bounding box can be placed within the centroid of the segment incorporating the initially-known dimensions of the brick.The assumption that the centroid of one segment equals the centroid of one brick is stated as facilitation.When placing the bounding box, all inlying points are ascribed to the same segment.After processing all segments accordingly, the segments get aligned in each row, respectively.This is done by calculating the mean alignment of the segments for each row (mean eigenvector corresponding to the largest eigenvalue).The related edges of the bounding boxes are rotationally corrected by their offset to the calculated mean value.Final step of the segmentation is the gap filling.Here unsegmented areas are considered.Segments are inserted in correspondence to the surrounding segments' position and orientation.

B-spline-approximation with inclusion of the variance covariance matrix through error propagation
The basis of the developed deformation analysis lies on the space-continuous approximation of the scanned point clouds using B-spline surfaces.A point on the surface Ŝ can be described in dependence of the surface parameters u and , equated with the weighted sum of locally relevant so-called control points P i, j .

Ŝ(u, 𝑣)
The net of control points can be regarded as a scaffold of the B-spline surface.A local deformation in the scaffold realized through a change in the location of one or more control points results in a local deformation of the surface and vice versa.This characteristic allows the transition of an initial space-continuous deformation analysis approach to the well-known and wide established analysis of congruent points.The mathematical model of the approximation of a point cloud using a B-spline surface can be described as linear -assuming prerequisite introduction of the surface parameters u and , the number of control points n + 1 and m + 1 in the direction u and , the knot vector and the functions' degrees p and q [7].
The linear model corresponds to a GMM, where the known observations S(u, ) are the coordinates of the points p k of the point cloud and the unknowns are the coordinates of the control points P i, j .Integrating a suitable stochastic model in the GMM is crucial.Within this approach the Variance-Covariance-Matrix (VCM) is built through error propagation of known uncertainty sources of the observations.

Insertion of the control points in the classical congruency model
The control points estimated separately for the two analysed epochs are introduced in the classical congruency model ( [1,11]).As in the point-based deformation analysis, a prerequisite condition for this step is a common definition of the geodetic datum.This is tackled within the present spacecontinuous approach by retaining the same parametrization for both B-spline approximations of epoch 1 and epoch 2. The following introduction of control points in the congruency model transforms the space-continuous deformation analysis into an equivalent point-based problem.This can be done after proofing that for all B-spline approximation models the resulting a posteriori variance factors of unit weight are realisations of the same population.If this condition is statistically proven, the congruency test can be applied, holding the following null hypothesis: The hypothesis states that the expectancy value of the difference d between congruent control points is zero.This results in the following test decision: where If the null hypothesis of the congruency test cannot be rejected at a predefined significance level , one follows that no deformation occurred between the sets of control points and in further consequence the B-spline surfaces are identical.Otherwise, if the null hypothesis is rejected at a confidence level 1 − , deformation occurred within the set of control points and has to be located.For the current study the localisation through decomposition of disclosures according to Gauss is applied [11].

Measurement campaign
The measurement campaign includes two one-day field trips acquiring data for epoch 1 (EP1) and epoch 2 (EP2) respectively.EP1 took place before the "Abkehr" i.e. when the aqueduct arc was under usual load of water weight.EP2 took place during the "Abkehr", i.e. in absence of mechanical loads due to water.As measuring instrument the Leica MS60 was used for the net measurement as well as for the scanning of the specimen.Each epoch includes a scan of the aqueduct arc accompanied by a terrestrial geodetic net measurement.The latter is used, amongst others, for the registration of the scanned point clouds.The measurement net consists of ten net points, among whom eight points were delimited with survey bolts and two points were delimited with marking pipes with head plates.In Figure 2 the measured net is visualized.
The scanning was conducted from the net point NP02.For an increase in the stationing accuracy the station height was measured with the integrated laser plumb of the instrument.Within each net measurement two object points (SP03 and SP04) were measured as well.Those are equipped with mini prisms installed on the top edge of the aqueduct arc.This allows an initial information whether deformation can be detected within the point-based comparison and further states a validation approach of the results obtained with the space-continuous deformation method introduced in the next chapter.The raw point clouds were centred in the instruments' centroid and registered by applying scan position coordinates as translation and the station orientation values as rotation around the z-axis, both resulting from the net adjustment.

Net adjustment
The software JAG3D [12] was employed for the adjustment of the terrestrial measurements as well as for the subsequent point-based deformation analysis.Firstly, separated net adjustments of the two measured epochs EP1 and EP2 were conducted.Here outliers were detected individually and the a-priori measurement uncertainties were adjusted using variance component estimation.When the individual statistical test of the net adjustments passed respectively, a joint net adjustment with the uncertainty components of the single net adjustments was conducted.Within this joint net adjustment, the coordinates of the net points were determined solely once.In Table 1 the resulting parameters of the single net adjustment and the joint net adjustment are shown.
In accordance with the joint net adjustment, the measured object points were introduced into a point-based deformation analysis.The results of the deformation analysis states that the object points did not move (hypothesis stated in Equation (3) could not be rejected at the confidence level of 95%).The difference between the coordinate elements of the object points measured in the two epochs are visualized together with their standard deviation in Figure 3.

Registration of scans
As mentioned in chapter 3, the point clouds obtained in the two epochs were registered using the adjusted coordinates of the scanning position (NP02) obtained from the joint net adjustments.The orientation parameter o was applied as rotation parameter along the z-axis.Corresponding values of the two station orientations result from the joint net adjustments.The registration parameters and their uncertainties are noted in Table 2.

Segmentation
The segmentation process described in chapter 2 was conducted using the following parameters: Within the initial RG segmentation in Opals the used thresholds of acceptance (see Equation ( 1)) were chosen as t 2 = 0.995 and t 1 = 25.
In the next step of the segmentation process the bricks' dimensions were incorporated -the immured bricks are of dimension 21.5 × 13.5 × 6.5 cm.In Figure 4 the results of the segmentation process are shown.

B-spline approximation
One initial task of the B-spline approximation of the point clouds is setting the optimal number of control points.For this, the Bayes' Information Criterion (BIC) [13] was used [14].Within the range of 4-20 control points in both, u and v-direction, 13 × 5 control points resulted as optimal choice, with a degree of 3 in both directions.The functional model consists therefore of a (3k × 3 • 65) design matrix A, where k is the number of points within a point cloud.The initial Bspline parameter values introduced in A are determined in a preceding step by projection of the points onto an initial surface, i.e.Coons Patch [15].The initial stochastic model results by error propagation of the measurements' uncertainties and the uncertainties of the scanning position as well as its orientation.The station coordinates' uncertainties are estimated within the net adjustment to be in the range of 0.2 to   The setup covariance matrix is introduced in the GMM to calculate the coordinate components and the uncertainties of the control points.If the null hypothesis of the global test of the GMM is rejected, the stochastic model is iteratively adapted.Observations, i.e. coordinate components of the single points of the point cloud, with a significant value of normalized residuals [11] are downweighted.However, these observations are not directly downweighted, but the uncertainties of the corresponding polar measurement components, i.e. the standard deviations of the horizontal directions and zenith angles as well as slope distances.Those are scaled by a factor of 1.05 in each iteration step.This is based on the finding that local fluctuations in the measuring directions mainly cause the outliers -not the stations' uncertainty values.In the following Table 4 the a-posteriori variance factor of the accepted null hypothesis and the maximum values of adapted standard deviations of horizontal directions, zenith angles and slope distances are shown for the point clouds of both epochs EP1 and EP2, in segmented and unsegmented form, respectively.The maximum values occur for the same points having significant normalized residuals in several iteration steps and thus being downweighted more times.The locations of the points of the point cloud, whose variances were scaled up during the iteration process are in every case distributed all over the arc, commonly rather on the edge regions of the scans.The parameter values in Table 4 show that within the approximation of the unsegmented point clouds the maximum values of downweighted observation are higher than when using segmented point clouds.

Deformation analysis
The deformation analysis is performed by inserting the control points estimated in the two epochs, in the congruency model.The model is set up for both cases of segmented and unsegmented data, in order to investigate the influence of the object's structure on the outcome of the analysis.In a prerequisite step the a-posteriori variance factors are tested if they are realisations of the same population.In both cases the null hypothesis stating that they belong to the same population was accepted, such that the a-posteriori variance factors were merged.Table 5 shows the resulting test values calculated for the global congruency test and the corresponding quantile for both cases of segmented and unsegmented data.
In both cases the global congruency test indicates a significant test value, meaning that deformation occurs in the collectivity of the control points.In the further localization process, which was implemented as localization by means of decomposition of disclosures according to Gauss [11], the deformed control points are identified.In case of unsegmented point clouds four control points are identified as deformed.Using the segmented point clouds, the number of deformed control points was three.In both cases the deformed control points are not centered in a specific area of the arc.In Figure 5 the position of the deformed control points is marked red for both cases.The white points are stable.The surface underneath the control points is the approximated B-spline surface of EP1, respectively.The coordinate difference of the deformed control points are visualized in Figure 6, together with their corresponding standard deviation.It can be noticed that for three cases (20, 50, 51) the coordinate differences lie below their corresponding standard deviation.The components of all difference vectors lie below their doubled standard deviation (2s).
As a consequence of these results and in view of the results obtained in the point-based case (see Section 4.1) the assumption that there might be an unrecognized rigid

Epoch comparison T F(𝜶 = 5%)
Unsegmented PC 1.48 1.17 Segmented PC 1.39 body movement left leading to falsely identified unstable points occurred.Therefore, the Iterative Closest Point (ICP) Algorithm [16] was applied on the already registered point clouds as an additional pre-processing step.The resulting translation and rotation parameters are given in Table 6.
The quantity criteria of the B-spline approximation remains unaffected by the application of the ICP-Algorithm.Therefore, the values are the same as listed in Table 4.The test values of the congruency tests changed noticeably.They indicate that no deformation occurred between the two epochs, for both cases of unsegmented and segmented point clouds.The test values and quantiles are stated in Table 7.

Analysis of the results
Several factors are crucial in the purposed approach.First factor refers to the influence of the object's structure on the B-spline approximation and in further consequence on the outcome of the deformation analysis.Considering the parameters in Table 4, an influence of the object's structure on the epoch-wise B-spline approximation is noticeable as the maximum values of downweighted observations are significantly smaller using segmented point clouds.This consequence is logic because local discrepancies of the point cloud to the approximated surface in the joints are eliminated.Considering the derived test values T in the global  congruency tests in Tables 5 and 7, both results show a smaller test value T using the segmented point clouds.In this context it is relevant to state that the imported point clouds were subsampled to a point density of 5 cm before the B-spline approximation, due to computational limits.Nonetheless, an increase of the test value in the congruency test, using unsegmented point clouds, does occur.Still, the resulting statements of the two global congruency tests are in both cases of unsegmented and segmented point clouds the same.In the case of the first global congruency test (Table 5), the null hypothesis is rejected either way.After the additional alignment of the point clouds using the ICP algorithm, the null hypothesis cannot be rejected.The results of the latter test comply with those obtained from the point-based deformation analysis.This result underlines the

Epoch Comparison T F(𝜶 = 5%)
Unsegmented PC 0.95 1.17 Segmented PC 0.94 importance of a very precise registration of the compared point clouds.In the present case, the registration information extracted from the net adjustment is not sufficient, as it neglects a small but crucial bias.Further causes for the detected inconsistencies can be simplifications of the processing procedure like the coordinate-wise approximation of the B-spline surfaces or the neglection of interepochal correlations.Their impact will be investigated in further studies.This causes the detection of non-existent deformations.Some arguments for an erroneously identification of the moved points in Figure 5 are: the identified points are distributed over the approximated B-spline surface.However, confirming the definition of B-splines a local deformation might presumably affect several neighbouring control points.Furthermore, coordinate differences of the unstable points are mostly below their corresponding standard deviation.When neglecting covariances, these differences are insignificant.Finally, the simplification of the applied stochastic model can be another cause for erroneous detection.Kermarrec et al. [3] already investigated the importance of the propagation of the correlations between observations and its effects on the test results of the global congruency test.Herein, the stochastic model results from error propagation and is therefore fully populated for each scanned point.However, at the level of the polar measurements, these were regarded as uncorrelated.Additional influences as correlating factors in the stochastic model of the point clouds as introduced in [17] or [18] are neglected here.

Summary and outlook
In this paper a space-continuous deformation analysis approach based on B-spline approximations was established and tested on point clouds of a regularly structured aqueduct arc.The deformation model can be categorized as parameter-based point cloud comparison as the control points of two registered B-spline surfaces are compared by insertion in the congruency model.Within the course of the evaluation an influence of the scanned surface structure on the B-spline approximation and further on the deformation results was investigated by introduction of unsegmented and segmented point clouds.The latter data set showed no structure as points within joints were eliminated.The investigated model of deformation analysis holds a lot of promise, as deformation within the set of control points could be rejected in a statistically sound way.This result was obtained in conformity with a pointwise object point comparison.An influence of the surface's structure on the B-spline approximations could be registered by comparison of the magnitude of downweighted observations using segmented and unsegmented point clouds.The test values within the congruency test were smaller using segmented point clouds as well.However, the test decision was equivalent using either unsegmented or segmented data.The importance of a highly precise point cloud registration appeared to be crucial within the evaluation.Likewise the incorporation of a sufficiently accurate stochastic model proved to be of utter importance.These aspects will be investigated more deeply in future research.

Figure 2 :
Figure 2: Measurement net with the net points NP01-NP10 and the object points SP03 and SP04.The observations are illustrated as lines between net points.

Figure 3 :
Figure 3: Difference between the object points from the compared epochs with corresponding standard deviation centered on the horizontal axis.

Figure 5 :
Figure 5: Visualization of the stable and deformed control points evaluated using (left) unsegmented point clouds and (right) segmented point clouds.

Figure 6 :
Figure 6: Difference between the deformed control points from the compared epochs with their derived standard deviation centered on the horizontal axis for (l) unsegmented point clouds and (r) segmented point clouds.

Table 1 :
Parameters of measurement adjustments.

Table 2 :
Point cloud registration parameters.

Table 3 :
Uncertainty values used in the error propagation.

Table 5 :
Test values of the congruency tests.

Table 7 :
Test values of the congruency tests.