Accessible Requires Authentication Published by De Gruyter December 16, 2020

Temporal calibration and synchronization of robotic total stations for kinematic multi-sensor-systems

Tomas Thalmann and Hans Neuner

Abstract

Despite the increasing interest in kinematic data acquisition, Robotic Total Stations (RTSs) are still relatively seldom used. No matter if Mobile Mapping Systems or Control & Guidance, GNSS is mostly used as position sensor, which limits the application to outdoor areas. For indoor applications, a combination of relative sensors is usually employed. One reason why RTSs are not used is the challenging time referencing and synchronization. In this work we analyze the challenges of a synchronized kinematic application of RTSs and present solutions.

Our approach is based on a wireless network synchronization to establish a precise temporal reference frame. The achievable synchronization quality is thoroughly examined. In addition we develop a kinematic model of spherical measurements, that incorporates timing related parameters. To estimate these parameters we propose a temporal calibration utilizing an industrial robot. Both parts of our approach are evaluated using a test setup of two total stations, proofing an overall synchronization accuracy of 0.2 ms. An overall horizontal kinematic point accuracy of 2.3 mm reveals the potential of sufficiently synchronized RTSs.

Appendix A Network Time Protocol

The Network Time Protocol (NTP) is used to distribute a reference time system in a computer network. The Protocol’s algorithms can be divided into three steps:

  1. 1.

    Definition of the reference servers.

  2. 2.

    Communication with the reference servers takes place on a regular basis (the polling interval) via UDP packets (User Datagram Protocol) on port 123. The timestamps in these packets are then used to estimate the difference (Offset and Drift, see (3)) to the reference time system.

  3. 3.

    The local clock is corrected in real-time based on the result of the previous step.

The NTP communication is visualized in Figure 16. From the four timestamps (T1 to T4), computer A can calculate the offset θb to reference clock B according to (16). This process is performed twice at each observation time (timestamps T5 to T8).

(16)θb=12T2T1+T3T4δb=T4T1T3T2

This approach assumes that the transmission duration of the packet is direction-independent. In addition, the round trip delay δb is used to calculate a measure for the (doubled) transmission time. The greater δb, the higher the probability that the transmission time will differ on the outward and return paths. Based on the synchronization distance (=δb2) and the transmission jitter (variability of the synchronization distance) the state of the controller clock xtc (offset dtc and drift fc, see Section 2) is estimated from the offset observations θb to n reference servers using robust evaluation methods:

(17)xtc=dtcfc=A(θ0,θ1,,θn,δ0,δ1,,δn,PNTP),

where θi and δi are the observations to the ith reference server, given by (16). A represents the core NTP algorithm and PNTP is a set of internal parameters.

Figure 16 NTP on-wire communication including timestamps (after [31]).

Figure 16

NTP on-wire communication including timestamps (after [31]).

To summarize, the achievable quality of synchronization using NTP depends on a variety of factors. 1) The reference servers and how precise they are in timekeeping. 2) Quality of the Ethernet transmission. This has mainly to do with the direction-independence assumption (symmetry) and is affected by Ethernet or Wifi, network topology and network traffic. 3) Local influences on the client. These result primarily from the Variable Frequency Oscillator (VFO) serving as the time-source in most computer-based systems. The frequency of the generated sinusoidal waveform depends on voltage, temperature and aging of the VFO. It follows, that CPU load also affects the synchronization result too.

Appendix B Taylor series expansion

The derivation of the first-order Taylor series is shown exemplarily for the x-coordinate (eq. (8), leaving out x0). Formulating x(tj) as a function of R and D using the following replacements

(18)D(td+δtd)=D,R(ta+δtd+δta)=R,

gives

x(D,R)=DcosR.

The first-order Taylor series expansion about D0, R0 (with RΔ in units of radian) is:

x(D0+DΔ,R0+RΔ)=x(D0,R0)+xD|D0,R0DΔ+xR|D0,R0RΔ.

Computing the derivatives and rearranging gives:

x(D0+DΔ,R0+RΔ)=D0cosR0+DΔcosR0D0RΔsinR0=D0+DΔcosR0D0RΔsinR0.

Resubstituting

(19)D0=D(td),DΔ=vd(td)δtd,R0=R(ta),RΔ=ω(ta)δtd+δta,

gives

(20)x(tj)=D(td)+vd(td)δtdcosR(ta)D(td)ω(ta)δtd+δtasinR(ta).

Appendix C Model equality

Starting from (10) (again leaving out x0):

(21)x(tj)=D(td)+vd(td)δtdcosR(ta)+ω(ta)δtd+δta.

Using the cosine expansion and small angle approximation for ω(ta)δt for the cosine term in (10) gives:

(22)cosR(ta)+ω(ta)δtd+δta==cosR(ta)cosω(ta)δtd+δta1AAAsinR(ta)sinω(ta)δtd+δtaω(ta)δtd+δtacosR(ta)sinR(ta)ω(ta)δtd+δta

Thus, (10) gives (neglecting second order terms):

(23)x(ts)D(td)+vd(td)δtdcosR(ta)D(td)+vd(td)δtdsinR(ta)ω(ta)δtd+δta==D(td)+vd(td)δtdcosR(ta)D(td)sinR(ta)ω(ta)δtd+δtavd(td)δtdsinR(ta)ω(ta)δtd+δta0D(td)+vd(td)δtdcosR(ta)D(td)ω(ta)δtd+δtasinR(ta),

which corresponds to (20).

Appendix D Common spatial reference frame

For the temporal calibration developed in Section 5 one necessary prerequisite is a common spatial reference frame shared between the reference IR and the RTS under test. This is established by estimating the transformation parameters from the IR coordinate frame u to the RTS coordinate frame r. In the same step the mounting vector tpe describing the prism center w. r. t. the robots end-effector frame e is determined. The transformation model is given in (24), adopted from [17]. The involved parameters x and observations l are described in Table 4.

(24)tpr=Rur(teu+Reutpe)+tur

Table 4

Parameters and Observations of the Transformation Model (24).

SymbolTypeDescription
tprlPrism position measured by the RTS, tpr=epnphpT
teulPosition of the robot end-effector w.r.t to the robot base frame, teu=xeyezeT
ReulMeasured orientation of the robot end-effector frame w.r.t to the robot base frame. Reu=R(ϕ,θ,ψ)
RurxRotation angles α, β, γ forming the rotation matrix Rur from robot base frame to RTS-frame.
tpexPosition of the prism w. r. t. the robot end-effector frame, tpe=mxmymzT
turxOrigin of the robot base frame w. r. t. the RTS-frame, tur=eunuhuT

The corresponding points for a fully automated process must be chosen in a way such that the prism orientation remains constant. Because this leads to correlations between mounting vector tpe and translations tur a semi-automated process was designed, to reduce such correlations. It consists n=52 static poses and requires human interaction at two points in time.

Figure 17 Correlations of estimated transformation parameters.

Figure 17

Correlations of estimated transformation parameters.

Table 5

Results of parameter estimation for a real data set.

Parameterxˆσˆ
α0.4129 gon4.9 mgon
β0.1420 gon5.1 mgon
γ−1.5843 gon4.5 mgon
eu16.15227 m0.09 mm
nu−0.55588 m0.06 mm
hu−0.59648 m0.07 mm
mx0.43 mm0.06 mm
my−65.36 mm0.07 mm
mz137.40 mm0.06 mm

For each pose i=1n it gives 9 observations li=xeyezeϕθψepnphpT with the corresponding variance-covariance-matrix Σlli:

(25)Σlli=σru2I3×3σϕu2I3×3Σir3×3.

With σϕu=1mgon for the orientation accuracy of the robot end-effector, σru=0.18mm for the robot position accuracy, and Σir for the propagated point accuracy for the measured RTS positions this gives the model for a Least Squares Adjustment with condition equations (Gauss-Helmert-Model) to estimate 9 parameters.

The results are given in Table 5 and the corresponding correlation matrix is shown in Figure 17. With this results a sufficient transformation accuracy has been achieved for the subsequent calibration.

References

[1] Alexander Beetz, Ein modulares Simulationskonzept zur Evaluierung von Positionssensoren sowie Filter- und Regelalgorithmen am Beispiel des automatisierten Straßenbaus, Dissertation, DGK series C, Universität Stuttgart, 2012. Search in Google Scholar

[2] Rolf Dach, Simon Lutz, Peter Walser and Pierre Fridez, Bernese GNSS Software version 5.2. User manual, Astronomical Institute, Bern: University of Bern, Bern Open Publishing. https://doi.org/10.7892/boris.72297 (2015). Search in Google Scholar

[3] Claudia Depenthal, A time-referenced 4D calibration system for kinematic optical measuring systems, in: Proc. of 1st International Conference on Machine Control and Guidance, 2008. Search in Google Scholar

[4] Claudia Depenthal, Entwicklung eines zeitreferenzierten 4-D-Kalibrier- und Prüfsytems für kinematische optische Messsysteme, Dissertation, DGK series C, Universität Karlsruhe, 2009. Search in Google Scholar

[5] Alexander Dorndorf, Jens Hartmann, Jens-Andre Paffenholz, Ingo Neumann and Christian Hesse, Validierung und Kalibrierung eines TLS-basierten Multi-Sensor-Systems, in: Photogrammetrie Laserscanning Optische 3D-Messtechnik–Beiträge der Oldenburger 3D-Tage 2015 (Hrsg. Luhmann/Müller), 2015, pp. 85–97. Search in Google Scholar

[6] Matthias Ehrhart and Werner Lienhart, Object tracking with robotic total stations: Current technologies and improvements based on image data, Journal of Applied Geodesy 11 (2017), 131–142. Search in Google Scholar

[7] Henri Eisenbeiss, Werner Stempfhuber and Michael Kolb, Genauigkeitsanalyse der 3D-Trajektorie von Mini-UAVs, DGPF Tagungsband 18 (2009), 407–417. Search in Google Scholar

[8] Christian Eling, Markus Wieland, Christophe Hess, Lasse Klingbeil and Heiner Kuhlmann, Development and evaluation of a uav based mapping system for remote sensing and surveying applications, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives 40 (2015), 233–239. Search in Google Scholar

[9] Jeremy Elson and Kay Römer, Wireless sensor networks, ACM SIGCOMM Computer Communication Review 33 (2003), 149–154. Search in Google Scholar

[10] Karl Foppe, Volker Schwieger and Rudolf Staiger, Grundlagen kinematischer Mess- und Auswertetechniken, Schriftenreihe des DVW 45 (2004). Search in Google Scholar

[11] Erwin A. Frei, Harald Vennegeerts and Daniel Omelanowsky, Kinematische Scanlösungen von p3d systems, Schriftenreihe des DVW 72 (2013), 107–124. Search in Google Scholar

[12] Zan Gojcic, Slaven Kalenjuk and Werner Lienhart, A routine for time-synchronization of robotic total stations, avn - Allgemeine Vermessungsnachrichten 10 (2018), 299–307. Search in Google Scholar

[13] Ask Bjorn Hansen, NTP Servers in Europe, 2019. Search in Google Scholar

[14] Erik Heinz, Christian Eling, Markus Wieland, Lasse Klingbeil and Heiner Kuhlmann, Development, calibration and evaluation of a portable and direct georeferenced laser scanning system for kinematic 3d mapping, Journal of Applied Geodesy 9 (2015), 227–243. Search in Google Scholar

[15] Maria Hennes, Steffen Urban and Sven Wursthorn, Zur Synchronisation von Multi-Sensor-Systemen - Grundlagen und Realisierungen, Schriftenreihe des DVW 75 (2014), 25–37. Search in Google Scholar

[16] Christian Hesse and Harald Vennegeerts, Referenzierung kinematischer Laserscansysteme mit IMU und Tachymeter, Schriftenreihe des DVW 75 (2014), 159–173. Search in Google Scholar

[17] Sabine Horvath and Hans Neuner, System identification of a robot arm with extended Kalman filter and artificial neural networks, Journal of Applied Geodesy 13 (2019), 135–150. Search in Google Scholar

[18] Friedrich Keller and Harald Sternberg, Multi-sensor platform for indoor mobile mapping: System calibration and using a total station for indoor applications, Remote Sensing 5 (2013), 5805–5824. Search in Google Scholar

[19] Gabriel Kerekes and Volker Schwieger, Kinematic positioning in a real time robotic total station network system, in: Proceedings of 6th International Conference on Machine Control & Guidance, 2018, pp. 35–43. Search in Google Scholar

[20] Gabriel Kerekes and Volker Schwieger, Position determination of a moving reflector in real time by robotic total station angle measurements, in: Proceedings of Geoprevi, 2018. Search in Google Scholar

[21] Gerhard Kleemaier, Multisensorsystem Totalstation, Schriftenreihe des DVW 92 (2018). Search in Google Scholar

[22] Lasse Klingbeil and Heiner Kuhlmann, Sensoren für kinematische Anwendungen, avn - Allgemeine Vermessungsnachrichten 121 (2014), 186–191. Search in Google Scholar

[23] Charles Knapp and Glifford Carter, The generalized correlation method for estimation of time delay, IEEE Transactions on Acoustics, Speech, and Signal Processing 24 (1976), 320–327. Search in Google Scholar

[24] Leo Köpf, Totalstationen und die 4. Dimension, in: Internationaler Ingenieurvermessungskurs, April, Graz, 2017. Search in Google Scholar

[25] Stefan Lackner and Werner Lienhart, Impact of prism type and prism orientation on the accuracy of automated total station measurements, in: Proc. Joint International Symposium on Deformation Monitoring (JISDM), p. 8p, 2016. Search in Google Scholar

[26] Otto Lerke and Volker Schwieger, Adaptive control for guidance of tracked vehicles, in: Proceedings of 6th International Conference on Machine Control & Guidance, 2018, pp. 83–94. Search in Google Scholar

[27] Miklos Maroti, Branislav Kusy, Gyula Simon and Akos Ledeczi, The flooding time synchronization protocol, in: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems - SenSys ’04, 2004, p. 39. Search in Google Scholar

[28] Andrew Miles and Carl Fenger, GNSS timing and the rise of small cells, u-blox, Report no. December, 2013. Search in Google Scholar

[29] David L Mills, Measured performance of the Network Time Protocol in the Internet system, 1989, p. 18. Search in Google Scholar

[30] David L Mills, Internet time synchronization: the network time protocol, IEEE Transactions on Communications 39 (1991), 1482–1493. Search in Google Scholar

[31] David L Mills, Computer Network Time Synchronization: the Network Time Protocol on Earth and in Space, 2nd edition ed, CRC Press, 2011. Search in Google Scholar

[32] Frank Neitzel, Sven Weisbrich, Wolfgang Niemeier and Martin Lehmann, GBSAR, TLS und Beschleunigungsmessungen für Schwingungsuntersuchungen, in: Interdisziplinäre Messaufgaben im Bauwesen, Weimar, 2012, pp. 121–148. Search in Google Scholar

[33] Dimitris S. Paraforos, Marcus Reutemann, Galibjon Sharipov, Roland Werner and Hans W. Griepentrog, Total station data assessment using an industrial robotic arm for dynamic 3D in-field positioning with sub-centimetre accuracy, Computers and Electronics in Agriculture 136 (2017), 166–175. Search in Google Scholar

[34] Allen J. Pope, Some pitfalls to be avoided in the iterative adjustment of nonlinear problems, in: Proceedings of the 38th Annual Meeting American Society of Photogrammetry, American Society of Photogrammetry, Washington, 1972, pp. 449–477. Search in Google Scholar

[35] William J. Riley, Handbook of Frequency Stability Analysis, 2008. Search in Google Scholar

[36] Craig Roberts and Peter Boorer, Kinematic positioning using a robotic total station as applied to small-scale UAVs, Journal of Spatial Science 61 (2016), 29–45. Search in Google Scholar

[37] Kay Römer, Philipp Blum and Lennart Meier, Time synchronization and calibration in wireless sensor networks, Handbook of Sensor Networks: Algorithms and architectures 49 (2005), 199–237. Search in Google Scholar

[38] Ruxandra Lupas Scheiterer, Chongning Na, Dragan Obradovic and Günter Steindl, Synchronization performance of the precision time protocol in industrial automation networks, IEEE Transactions on Instrumentation and Measurement 58 (2009), 1849–1857. Search in Google Scholar

[39] Volker Schwieger and Harald Sternberg, Multi-Sensor-Systeme in der Ingenieurgeodäsie - Grundlagen und Überblick, Schriftenreihe des DVW 75 (2014), 3–24. Search in Google Scholar

[40] Jonathan Shannon, Padraig O’Flaithearta, Hugh Melvin and Yusuf Cinar, Enhancing multimedia QoE via more effective time synchronisation over 802.11 networks, in: Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016, 2016, pp. 261–269. Search in Google Scholar

[41] Werner Stempfhuber, Ein integritätswahrendes Messsystem für kinematische Anwendungen, Dissertation, DGK series C, Technische Universität München, 2004, p. 131. Search in Google Scholar

[42] Werner Stempfhuber, Synchronisation von zielverfolgenden Servotachymetern für kinematische Anwendungen, Schriftenreihe des DVW 45 (2004). Search in Google Scholar

[43] Werner Stempfhuber, Verification of the Trimble universal total station (UTS) performance for kinematic applications, Optical 3-D measurement techniques: applications in GIS, mobile mapping, manufacturing, quality control, robotics, navigation, cultural heritage, natural and manmade hazards monitoring, medical imaging, VR generation and animation; papers pr 2009, pp. I–211. Search in Google Scholar

[44] Werner Stempfhuber and Thomas Wunderlich, Leica System 1200: Auf dem Weg zur Sensorsynchronisation von GPS und TPS für kinematische Messaufgaben, avn - Allgemeine Vermessungsnachrichten 111 (2004), 175–184. Search in Google Scholar

[45] Harald Sternberg, Friedrich Keller and Thomas Willemsen, Precise indoor mapping as a basis for coarse indoor navigation, Journal of Applied Geodesy 7 (2013), 231–246. Search in Google Scholar

[46] Tomas Thalmann and Hans Neuner, Evaluation of the network time protocol for synchronization of multi-sensor-systems, avn - Allgemeine Vermessungsnachrichten 125 (2018), 163–174. Search in Google Scholar

[47] U-blox, GPS-based timing: considerations with u-blox 6 GPS receivers, 2011, p. 14. Search in Google Scholar

[48] Harald Vennegeerts, Jens Martin, Matthias Becker and Hansjörg Kutterer, Validation of a kinematic laserscanning system, Journal of Applied Geodesy 2 (2008), 79–84. Search in Google Scholar

Received: 2019-11-18
Accepted: 2020-11-18
Published Online: 2020-12-16
Published in Print: 2021-01-27

© 2021 Walter de Gruyter GmbH, Berlin/Boston