Skip to content
Licensed Unlicensed Requires Authentication Published by Oldenbourg Wissenschaftsverlag October 17, 2018

Applicability of generalized test tables: a case study using the manufacturing system demonstrator xPPU

Anwendbarkeit von Generalized Test Tables: Eine Fallstudie anhand des xPPU-Demonstrators
Suhyun Cha, Alexander Weigl, Mattias Ulbrich, Bernhard Beckert and Birgit Vogel-Heuser


With recent trends in manufacturing automation, control software in automated production systems becomes more complex and has more variability to keep pace with customer and market requirements. Quality assurance also becomes more and more important to ensure that the systems live up to expectations. However, correctness of automation software is rarely verified using formal techniques in spite of their high coverage. One of the main reasons is the lack of specification languages suitable for this application area that are both comprehensible and sufficiently expressive. Generalized test tables (GTTs), which are a specification language for reactive systems, were presented recently as an accessible representation for application engineers. This formalism achieves both the comprehensibility of concrete test tables and the coverage of formal methods. In our approach, the specification provided by GTTs is used for formal verification, especially model checking. In this paper, we present four new features for GTTs: the progression flag, strong repetition, row grouping, and specification on internal variables. We demonstrate the applicability and evaluate the comprehensibility of GTT-based specification and verification using a range of diverse scenarios from the community demonstrator, the extended Pick & Place Unit.


Steigende Kunden- und Marktanfordungen in der Fertigungsautomatisierung erfordern komplexere Steuerungssoftware und kürzere Entwicklungszyklen. Um zukünftig Korrektheit und Zuverlässigkeit sicherstellen zu können, ist eine Anpassung an der Qualitätssicherung erforderlich. Formale Methoden können hierfür nachprüfbare Garantien bieten, aber obwohl Automatisierungstechnik in unternehmenskritischen Bereichen eingesetzt wird, werden formale Methoden dort selten verwendet. Einer der Gründe ist der Mangel an geeigneten Spezifikationssprachen für die Automatisierungsdomäne, die sowohl nachvollziehbar als auch ausreichend aussagekräftig sind. Generalized Test Tables (GTTs) sind eine formale tabellen-basierte Spezifikationssprache für reaktive Systeme für den Entwicklungsingenieur. GTTs erhöhen die Aussagemächtigkeit und Testabdeckung konkreter Testtabellen unter Beibehaltung ihrer Verständlichkeit. In diesem Beitrag analysieren wir die Anwendbarkeit und Verständlichkeit von GTTs. Dazu spezifizieren wir Teile des Anlagenverhaltens diverser Szenarien aus dem Community Demonstrator Pick&Place-Unit (PPU).

Funding source: Deutsche Forschungsgemeinschaft

Award Identifier / Grant number: VO 937/28-2

Award Identifier / Grant number: BE 2334/7-2

Award Identifier / Grant number: UL 433/1-2

Funding statement: Research supported by the DFG (German Research Foundation) in Priority Programme SPP1593: Design for Future – Managed Software Evolution (VO 937/28-2, BE 2334/7-2, and UL 433/1-2).

Appendix A Definition for the new features

In this section we present the updated definitions for the structure of GTT ([3, Def. 3]), and the definition of unrolled instances (D1(T) cf. [3, Def. 3]).

In earlier version the duration interval is an interval [m,n], where mN and nN{}.

In the previous work, the structure of a GTT is defined as a sequence of triples (input, output and duration constraint). Strong repetition and progress flag requires extension on the domain of the duration constraints τ:

Definition 1 (Duration interval τ).

A duration interval is either ω (strong repetition), an (top-open) interval[m,n]with or without the progress flag:

We define a function val(τ) that evaluates a duration constraint into a set of possible repetitions.

Definition 2 (val(τ)).

With the introduction of nested blocks, this sequential structure becomes obsolete. The new mathematical structure is defined inductively to capture the recursive nature of GTTs.

Definition 3 (GTTs as a Tree of Constraints).

LetInVarbe the set of input variables,OutVarbe the set of output variables, andGVarbe the set of global variables; then the set of all GTTsTΣover signatureΣ=InVarOutVarGVar. is defined as


The empty table ϵ and a single row(ϕ,ψ,τ)are GTTs (ϵ,(ϕ,ψ,τ)T, ϕ is the conjunction of all input column constraints of a row, resp. ψ for the output columns and τ defines the duration interval).


LetTTa GTT, then


the sequential compositionT,TTis also a GTT, whereT,TT.


the repetition of a GTT T is also a GTT:[T,τ]Twith an arbitrary duration interval τ andTT.

The set InVar and OutVar also includes the categorized state variables. The constraints formulas ϕ and ψ are Boolean formulas over the signature Σ. The sequential composition with the empty table cover the GTTs as defined in [3]. We use angle brackets to denote the sequence, and parentheses to denote a row. ϵ denotes the empty table. The nest rule creates the nested row groups, denoted by brackets.

Example 1.

The GTT in Fig.4is represented by the following structure for a cycle of 100 ms.

The game [3, Fig. 4] maintains the set S=D1(T), which represents the possible remaining test table. D1(T) defines a infinite language over the alphabet Σ=FmlΣ×FmlΣ, where FmlΣ is the set of all Boolean formulas over signature Σ. We define D1(T)—a version for GTTs after Def. 3. D1(T) transform the recursive data structure into an (infinite) set of (in)finite words.


Note, ϵ denotes the empty word, and Lτ=kval(τ)Lk. With D1(T) we are back into the framework given in [3] with one exception: D1(T) could contain infinite words. The game remains applicable, but an infinite word in D1(T) prevents the win of the system by reaching the table end.

Appendix B Semantic impact: No strict conformance

The following proposition rephrase the termination of the game:

Proposition 1.

Termination of the game [3, Fig. 4] The game terminates in roundk>0

  1. 1.

    ifS=after the turn of the challenger (System wins, Line 13)

  2. 2.

    ifS=after the turn of the system (Challenger wins, Line 17)

  3. 3.

    if the end of the table is reachedDS.|D|=k(System wins, Line 21).

S denotes the set of the currently remaining valid unwound GTTs.

The next proposition is generalization of the statement, that a strong repetition prevents strict conformance.

Proposition 2.

LetSkbe set of remaining test tables in round k. If all remaining test tables are infinitewSk.|w|=ωthen the system can not win by Line 21 (end of table).


The first implication is a direct consequence that k is always finite: kN and nN.n<ω.  □

Moreover for the prevention, we need to require the existence of a challenger that can survive infinitely long. Therefore all input constraints ϕ, especially these the strong repeated rows, need always be satisfiable.

Proposition 3.

If there exist infinite valid challenger stimuliin, i. e.[k]ϕ[k]
then there does not exists a strict conform system.ϕ[k]is the input constraint givenw[k].


Let there be a strict conform system. Also, the system wins against every challenger. The win by end-of-table is disabled and there exists a challenger which has a never-ending valid play. The system can not win. Contradiction!  □

Appendix C Proof: Strong repetition does not extend the expressiveness

After defining the structure in Def. 3. We can define the transformation of T to T formally.

Definition 4 (Construction ofT).

Tfor a given GTTTis constructed by the following recursive functiontrans(T):


Φ(T)is defined by the disjunction(ϕ,ψ,τ)succ(0)ϕof all immediately reachable rowssucc(0)in T from the start (cf. functionsuccin [3]).

We introduce I and O as finite sets that denote the value domain of the input and output variables of the reactive system, determined by the system’s interface (variable type).

Our goal is to prove the equal weak and strict conformance.

Proposition 4 (Equal Conformance).

A reactive system is weak conform toTiff it is conform toT, analog for strict conform.

The conformance of a test table is based on the outcome of all possible plays, i. e. a system is strict conform iff it is a winning strategy and weak conform iff its strategy never loss.

The next lemma reduces the equal conformance on equal game outcome on all plays.

Lemma 1.

Two GTTT,Thave equal conformance if the game outcome for all possible plays is equal.

In rest of the proof for Prop. 4, we need to show that for an arbitrary play, without assumption on the table or the system, the outcome (winner and loser) is equal on T and T. The outcome is determined by the algorithm [3, Fig. 4]. We show that by defining a coupling between both run of the algorithm for T and T. More formally, the algorithm based on a set S (cf. Lemma 1), that holds the remaining possible unwound GTT. S is determined by D1(T) in the beginning. The outcome is decided within one round of the game, especially if S becomes empty. We established a coupling relation between the set S from the run over T and T: Sk and Sk for round 0<k. The coupling relation established a bisimulation.

To define the coupling relation, we need following property on the structure of Sk and Sk.

Lemma 2 (Structure ofSk).

In every roundk0,SkandSkcan be separated:

for a languageαlandβl.

Lfin contains the finite words defined of a test tables, e. g., if a strong repetition is avoidable. It is also possible to select a separation, s. t. αl is the path into an strong repetition, and βl is the strong repetition. There are only a finite amount of strong repetition, the set union is finite. Moreover in the separation Sk, β represents the block (or row), that is strong repeated, which is implicitly described in Lemma 3. This connects the βl with the construction T, especially Φl.

Further, we define the languages VC,VSSk, that represent the words in Sk that are violated by turns of the challenger or the system.

Definition 5.

Let S be language of(ϕ,ψ),p(I,O)a play, andk>0:


(Proof by induction over k).

For k=0, the separation follows immediately from the Definition of D1, S0 is actually regular.

For k>0, in every round Sk is filtered by the current play p with |p|=k, formally Sk=(LfinSk1)VS(S,p,k). From induction hypotheses, we know that Sk1 can be separated and by definition of the game the words in Sk1 have at least the length k, otherwise the game has terminated. Assume w. l. g. that every αl in the separation of Sk1 contains only words with length k, otherwise we would unwind β as many times as needed.


We established a relation between both sets, which follows from the Def. 4.

Lemma 3 (Coupling ofSk and Sk).

There exists a separation ofSkandSk, s. t. bothLfinare equal and for


We show the coupling between both sets are maintain after each, additionally prove a little bit more: The coupling is maintained after the choice of the challenger and the system.

(Proof by induction over k).

The lemma is immediately valid for k=0.

Let Sk1 and Sk1 are be coupled, we need to show that the coupling ensured after the round k for Sk and Sk. We also assume an arbitrary play p(I,O) s. t. p=p·(a,b)|p|=k and there is no winner of the play in all rounds k<|p|, otherwise the game would have terminated (both sets are empty and the coupling relation would immediately hold).

The coupling is stable under set difference: Stable means, that the a coupled αlβlω and αlβl(Φl,false) will stay coupled in all rounds.

In detail: By induction hypotheses the languages of the finite words in D1(T) are coupled, hence Lfin,k1=Lfin,k1. Applying the set difference keeps equivalence:


The same holds for any coupled αlβlω and αlβ(Φl,false) of the separation. W. l. g. we can pump up αl with βl s. t. all words w(αl·βl) are at least greater or equals to k.

Leaving one open case, αl(Φ,false) (we choose ϵβ). Obviously, this word is removed by subtracting with VS(S,p,k) (ψ in position k is false), but this is not valid VC(S,p,k). Here it is possible, that αlβlωVC(Sk1,p,k), but αl(Φl,false)Vc(Sk1,p,k). Note, βl represents the corresponding block for which Φl is built for. By definition, Φl is the disjunction of all ϕ of the first symbols in βl. Therefore, the play p violates Φl iff it violates all first ϕ in β.  □

Prop.4. From coupling in Lemma 3 it follows that Sk=Sk=. (Proof in contraposition)  □

Example 2.

For a better auditability, we show the equal conformance on the special case with


The row(ϕ,false,1)prevents the strict conformance of the system—the system cannot fulfillfalse, the same holds for the strong repetition inT. Therefore, we only need to consider weak conformance. Letp(I,O)be an arbitrary play. The system wins inTifpϕ. At the same time it would win inT, as p is not allowed in both rows. Otherwise, the system loses inT(pϕandpψ) iff it also violatesTas none of both rows is adhered (pψpfalse).


1. Roy Armoni et al. “The ForSpec Temporal Logic: A New Temporal Property-Specification Language.” In: Tools and Algorithms for the Construction and Analysis of Systems. Ed. by Joost-Pieter Katoen and Perdita Stevens. Berlin, Heidelberg: Springer Berlin Heidelberg, 2002, pp. 296–311.Search in Google Scholar

2. Nanette Bauer et al. “Verification of PLC Programs Given as Sequential Function Charts.” In: Integration of Software Specification Techniques for Applications in Engineering: Priority Program SoftSpez of the German Research Foundation (DFG), Final Report. Ed. by Hartmut Ehrig et al. Berlin, Heidelberg: Springer Berlin Heidelberg, 2004, pp. 517–540. DOI: 10.1007/978-3-540-27863-4_28.Search in Google Scholar

3. Bernhard Beckert et al. “Generalised Test Tables: A Practical Specification Language for Reactive Systems.” In: Integrated Formal Methods. Ed. by Nadia Polikarpova and Steve Schneider. Cham: Springer International Publishing, 2017, pp. 129–144. DOI: 10.1007/978-3-319-66845-1_9.Search in Google Scholar

4. Bernhard Beckert et al. “Regression verification for programmable logic controller software.” In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). LNCS 9407. November (2015), pp. 234–251. ISSN: 16113349. DOI: 10.1007/978-3-319-25423-4_15.Search in Google Scholar

5. Sebastian Biallas, Jörg Brauer and Stefan Kowalewski. “Arcade.PLC: A Verification Platform for Programmable Logic Controllers.” In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering. ASE 2012. Essen, Germany: ACM, 2012, pp. 338–341. DOI: 10.1145/2351676.2351741.Search in Google Scholar

6. Roberto Cavada et al. “The nuXmv Symbolic Model Checker.” In: CAV. Ed. by Armin Biere and Roderick Bloem. Vol. 8559. Lecture Notes in Computer Science. Springer, 2014, pp. 334–342. ISBN: 978-3-319-08866-2.Search in Google Scholar

7. Adrien Champion et al. “CoCoSpec: A Mode-Aware Contract Language for Reactive Systems.” In: Software Engineering and Formal Methods. Ed. by Rocco De Nicola and Eva Kühn. Cham: Springer International Publishing, 2016, pp. 347–366, ISBN: 978-3-319-41591-8.Search in Google Scholar

8. G. Frey and L. Litz. “Formal methods in PLC programming.” In: Systems, Man, and Cybernetics, 2000 IEEE International Conference on. Vol. 4. 2000, pp. 2431–2436. DOI: 10.1109/ICSMC.2000.884356.Search in Google Scholar

9. L. Heitmeyer and R. D. Jeffords. “Applying a Formal Requirements Method to Three NASA Systems: Lessons Learned.” In: 2007 IEEE Aerospace Conference. 2007, pp. 1–10. DOI: 10.1109/AERO.2007.352764.Search in Google Scholar

10. Gerard J. Holzmann. “The Logic of Bugs.” In: Proceedings of the 10th ACM SIGSOFT Symposium on Foundations of Software Engineering. SIGSOFT ’02/FSE-10. Charleston, South Carolina, USA: ACM, 2002, pp. 81–87. ISBN: 1-58113-514-9. DOI: 10.1145/587051.587064. URL: in Google Scholar

11. International Electrotechnical Commission, IEC 61131: Programmable controllers – Part 3: Programming languages. Tech. rep. International Electrotechnical Commission, Feb. 2002.Search in Google Scholar

12. S. Kowalewski et al. “Verification of logic controllers for continuous plants using timed condition/event-system models.” In: Automatica 35.3 (1999), pp. 505–518. ISSN: 0005-1098. DOI: 10.1016/S0005-1098(98)00179-4.Search in Google Scholar

13. Antti Pakonen et al. “User-friendly formal specification languages – conclusions drawn from industrial experience on model checking.” In: IEEE International Conference on Emerging Technologies and Factory Automation (ETFA 2016). Vol. 2016-Novem. Berlin, Germany, 2016. ISBN: 9781509013142. DOI: 10.1109/ETFA.2016.7733717.Search in Google Scholar

14. Susanne Rösch and Birgit Vogel-Heuser. “A Light-Weight Fault Injection Approach to Test Automated Production System PLC Software in Industrial Practice.” English. In: Control Engineering Practice 58.Complete (2017), pp. 12–23. DOI: 10.1016/j.conengprac.2016.09.012.Search in Google Scholar

15. Doaa Soliman and Georg Frey. “Verification and validation of safety applications based on PLCopen safety function blocks.” In: Control Engineering Practice 19.9 (2011). Special Section: DCDS’09 – The 2nd IFAC Workshop on Dependable Control of Discrete Systems. pp. 929–946, ISSN: 0967-0661. DOI: 10.1016/j.conengprac.2011.01.001.Search in Google Scholar

16. Markus Spindler et al. “Erstellung von Steuerungssoftware für automatisierte Materialflusssysteme per Drag & Drop.” In: Logistics Journal: Proceedings 2017.10 (2017). DOI: 10.2195/lj_Proc_spindler_de_201710_01.Search in Google Scholar

17. Ofer Strichman. “Regression Verification: Proving the Equivalence of Similar Programs.” In: Computer Aided Verification Ed. by Ahmed Bouajjani and Oded Maler. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, p. 63. DOI: 10.1007/978-3-642-02658-4_8.Search in Google Scholar

18. Kleanthis Thramboulidis. “The 3+1 SysML View-Model in Model Integrated Mechatronics.” In: Journal of Software Engineering and Applications 03.02 (2010), pp. 109–118. ISSN: 1945-3116. DOI: 10.4236/jsea.2010.32014.Search in Google Scholar

19. Sebastian Ulewicz and Birgit Vogel-Heuser. “Automatisiertes Testen von Sondermaschinen - von der Modulbibliothek bis zur Anlage.” In: Tagungsband Automation Symposium. 2015, pp. 53–65.Search in Google Scholar

20. Sebastian Ulewicz et al. “Proving equivalence between control software variants for Programmable Logic Controllers: Using Regression Verification to Reduce Unneeded Variant Diversity.” In: IEEE International Conference on Emerging Technologies and Factory Automation, ETFA. Vol. 2015-October. 2015, pp. 1–5. DOI: 10.1109/ETFA.2015.7301603.Search in Google Scholar

21. Birgit Vogel-Heuser et al. “Evolution of software in automated production systems: Challenges and research directions.” In: Journal of Systems and Software 110 (2015), pp. 54–84. DOI: 10.1016/j.jss.2015.08.026.Search in Google Scholar

22. Birgit Vogel-Heuser et al. “Fault Handling in PLC-Based Industry 4.0 Automated Production Systems as a Basis for Restart and Self-Configuration and Its Evaluation.” In: Journal of Software Engineering and Applications 9.1 (2016), pp. 1–43. DOI: 10.4236/jsea.2016.91001.Search in Google Scholar

23. A. N. I. Wardana, J. Folmer and B. Vogel-Heuser. “Automatic program verification of continuous function chart based on model checking.” In: 2009 35th Annual Conference of IEEE Industrial Electronics, 2009, pp. 2422–2427. DOI: 10.1109/IECON.2009.5415231.Search in Google Scholar

24. A. Weigl et al. “Generalized test tables: A powerful and intuitive specification language for reactive systems.” In: 2017 IEEE 15th International Conference on Industrial Informatics (INDIN), 2017, pp. 875–882. DOI: 10.1109/INDIN.2017.8104887.Search in Google Scholar

Received: 2018-03-05
Accepted: 2018-07-30
Published Online: 2018-10-17
Published in Print: 2018-10-25

© 2018 Walter de Gruyter GmbH, Berlin/Boston