Evaluating the use of a tactile sensor for measuring carton compliance

Abstract This work reports the evaluation of a tactile sensor for the potential of using it to measure the compliance of folding cartons. A tactile sensor would make it possible to measure the mechanical behavior locally around the contact point, in contrast to existing methods that measure the global mechanical behavior of the carton. Research on the haptic sense has shown that the local mechanical behavior is more important than the global behavior when humans assess compliance of objects. It is shown that the response of the tactile sensor correlates strongly with the bending stiffness of the board, but also with geometric features. A method for reducing the 22-dimensional output of the sensor to single meaningful feature using linear discriminant analysis is proposed and tested. The results show that the sensor is a good candidate for a method that incorporates both cutaneous and kinaesthetic information in the measure of carton compliance.


Introduction
The moment of truth for a package is when a potential customer gets hold of it in a retail store (Löfgren 2005). It is now that the decision is made -should I purchase this product or not? Previous research has shown that the mere act of touching a product influences the judgement of it Childers 2006, 2003). For a brand owner, harnessing the power in this experience could be an important marketing tool.
A range of studies have shown examples of this effect in practice. The same water may taste better when served in a glass rather than when served in a plastic cup (Tu et al. 2015) or it may taste better when served in a firm cup than when served in a flimsy cup (Krishna and Morrin 2008). Food may be perceived as more crunchy when its package is given a rougher surface (Piqueras-Fiszman and Spence 2012). Whether a soup is perceived as natural or not may in part depend on the package in which it comes (Labbe et al. 2013).
All these results show that tactile properties can play an important role in the decision to purchase a product. One property that is likely to be important is the stiffness of the package. While there is little data in the open literature, there is evidence of industry interest. There are two patents that mention "grip stiffness" (Ljungstroem andStacy-Ryan 2000, Murata 2013). Furthermore, there is a case study on whisky cartons where grip stiffness came up as a crucial property (Rundh 2013).
The tactile sense is different from many of the other senses in that the sensations require active exploration. The actions taken in order to perceive a tactile property are called an exploratory procedure (EP). Examples of EPs could be enclosing or stroking an object.
When judging compliance, humans can use both kinaesthetic and cutaneous cues. The kinaesthetic cues are basically how the fingers move when a certain force is applied. The cutaneous cues are instead related to what happens around the fingertip and are picked up by receptors in the skin. Srinivasan and LaMotte (1995) showed that cutaneous cues are sufficient for discrimination of compliance, whereas kinaesthetic cues are only required if the object is inhomogeneous. Bergmann Tiest and Kappers (2008) built upon this work and produced stimuli with a compliant surface and a rigid core. When subjects were asked to compare the two types of stimuli, it was found that cutaneous information contributed 75 % of the judgment with the remainder from kinaesthetic information.
Testing packaging in a manner similar to how humans gather kinaesthetic information can be achieved by com-pression between rigid objects (Eriksson and Korin 2017). As discussed above, this information is not what is mainly used when humans judge compliance. In order to provide test results more like human perception, it would be useful to employ test instrumentation that includes a tactile sensor. Ideally such a sensor should mimic both the relevant sensory modalities and the contact mechanics between the human finger and the package. Artificial fingertips have been used to test packaging materials (Shao et al. 2010(Shao et al. , 2009), but these artificial fingertips did not include the sensory capabilities necessary for compliance measurements.
In this work we evaluate a type of tactile sensor that attempts to mimic both the contact mechanics of the human finger as well as its sensory capabilities. The aim is to improve the possibilities to measure important parameters that more fully capture the perception of humans. We implement an EP and test the ability to distinguish packages based on the sensor readings. We focus only on cutaneous cues, since they are the most important for judging compliance.

Materials and methods
To inform the design of the EP, the first step of the study consisted of asking consumers to answer a set of questions about packaging and observing how they explored the packages. The EPs that consumers used were recorded and informed the design of the idealized EP used in the trials with a tactile sensor in phase two.
Based on general mechanical principles, we put forward a set of hypotheses for what values from the tactile sensor correlate with the package mechanical properties. These hypotheses were tested using ordinary least squares regression. To further validate that these are the most important readings from the tactile sensor, we employ linear discriminant analysis to find the readings with the highest explanatory power.

Paperboard packages
The packages used throughout this study were manufactured on a flatbed cutter (Esko-Graphics, Gent, Belgium). The packages were all made from clay-coated virgin paperboard, in a range of grammages from 239-486 g/m 2 . The material was manufactured on an industrial scale paperboard machine. All in all, 8 different materials and 3 different package designs were tested. The packages labeled I-III are summarized in Table 1. Material data for the ma-

Exploratory procedure
The design of the EP for this experiment was based on a pre-study. Subjects were presented with a set of packages of the types I and II and were asked to fill out a simple questionnaire about the package. The packages used were plain white, and subjects were given no other information about the product or the market segment targeted. Package I was filled with two layers of chocolate (weight 530 g) of the type Aladdin (Mondelēz Sverige AB, Upplands Väsby, Sweden). Package II was filled with a bottle of Blossa Glögg (750 ml, 1260 g) (Altia Sweden, Kungsängen, Sweden). Subjects (n = 44) were recruited at a supermarket in Sweden. They were first asked to state their gender, age, and dominant hand. There were 28 female and 16 male participants with median age of 59. Three had a dominant left hand and the remaining 41 had a dominant right hand. They were then requested to pick one out of four words describing each package. The words were stable, unstable, durable, and weak. These words were chosen so that the subjects would employ relevant EPs. Furthermore, the subjects could give a free-text answer.
By studying how the subjects explored the packages during this exercise, it was possible to identify loading directions that are important for the consumers' perception of the package. From the identified load cases, we defined a simplified EP. We chose to simplify the EP by excluding tangential forces. This is justifiable because a paperboard package is a shell structure, and hence can cope well with membrane stresses. The implemented EP is designed to mimic the compression of a package with one finger while the package is resting on a flat rigid surface, see Figure 1.  239  264  292  351  305  352  403  486  BR MD  ISO 2493  mN  199  288  419  689  489  841  1207  1775  BR CD  ISO 2493  mN  110  152  222  397  271  352  573  868  BR  mN  148  209  305  523  364  544  832  1241  SCT MD ISO 9895 kN/m 5.8 6.9 6.7 8.4 7.7 8.4 10.2 9.9 SCT CD ISO 9895 kN/m 4.4 5.2 5.0 6.5 5.9 6.6 7.8 8.0 SCT GM kN/m 5.1 6.0 5.8 7.4 6.7 7.4 8.9 8.9 Figure 1: The modeled exploratory procedure.

BioTac sensor
The tactile sensor used in phase two was a BioTac 2P (Syn-Touch Inc., Los Angeles, CA, USA) (Wettels et al. 2008). The BioTac is modeled to mimic the sensory capabilities of the human finger. The BioTac consists of a rigid epoxy core, surrounded by an elastomer skin, with a conducting liquid filling the void. A total of 19 Electrodes embedded in the core measure the changes in impedance caused by displacement of the liquid. The signals from the electrodes are digitized at 100 Hz. Figure 2 shows the distribution and numbering of the electrodes on the BioTac. Additionally, the BioTac has a pressure sensor and a thermistor. The pressure sensor measures the pressure inside the liquid. This signal is filtered through a low pass filter and digitized at 100 Hz on the P DC channel. The pressure sensor is also high pass filtered, amplified and digitized at 2200 Hz on the P AC channel. The thermistor is also digitized and provided in a low pass filtered version and a derivative version.
The BioTac has been used successfully for force and compliance determination in the previous experiments

Experimental setup
The setup (Figure 3) consisted of the BioTac with data acquisition hardware and software, an AMTI six-axis force plate with DAQ (NI6218) and data acquisition software (NI LabView). The BioTac was mounted on an aluminum bar with screws that replaced the ones that normally hold its fingernail. The angle of the aluminum bar to the vertical line was 79°. This angle caused the flat part of the core of the BioTac (see further discussion below) to be parallel to the package before deformation. The aluminium bar was attached to a manipulator that allowed vertical movement. A thick plywood sheet was fixed to the force plate using double sided tape. This extended the area that packages were placed on in order to make sure that the entire package was supported.
The BioTac was used with the standard sampling sequence. The order is low pass filtered pressure (P DC ), temperature derivative (T AC ), temperature (T DC ), and electrode impedances (E 1 − E 19 ). Between each sample from these channels, the high pass filtered pressure (P AC ) is sampled. The total sampling frequency was 4400 data points per second. The BioTac was slowly pushed against the top surface of the package until a static vertical force of 4 N was recorded. The distance from the edge of the package a was varied between two values (18 and 22 mm) reflecting a likely range of variability in human exploration. BioTac sensory data at 4 N force were generally overlapping for these two distances, as illustrated in Results.
The data acquisition was conducted on two different computers. One acquired the signal from the BioTac and one acquired the signal from the force plate. These signals were synchronized using a custom programmed Bio-Tac board that sent out a 100 Hz clock signal that was used to trigger the acquisition on the NI6218 board.
In some experiments, the trigger function failed for unknown reasons, which lead to the force plate data lagging behind the BioTac data. The external clock still managed to maintain equal spacing of samples. In these cases, the data was realigned by visual inspection of the graphs of the vertical force and BioTac pressure signal. The accuracy was estimated to be 20 ms (two samples), which is slightly worse than the automatic synchronization which should be accurate to at least ±5 ms (half a sample).

BioTac measurements
Only the electrodes and the P DC signal from the BioTac were analyzed. P AC was not studied because it was com-pletely dominated by variations in the compression rate. The thermistor signal was not of interest for this work. We paid particular attention to electrodes 7-10, which are located on a flat region of the rigid core that mimics the apical tuft of the human distal phalanx. The voltages measured by these electrodes are inversely related to the impedance of the conductive liquid that is squeezed between the electrodes and the overlying skin. These voltages have been shown to be strongly influenced by small tilts of flat surfaces against which the finger is pressed, similar to the presumed sensory function of the apical tuft during precision grip in humans (Su et al. 2016). This is particularly true for small tilts around parallel alignment between the flat on the BioTac core and the surface against which it is pressed. Su et al. (2012) showed that P DC is dependent on the compliance of the surface. In the current case, we must consider the structural stiffness of the paperboard. This depends on the bending resistance of the material as well as the geometry of the package. Because both grammage and SCT are strongly correlated with bending stiffness of the material, we did not expect to distinguish among the three in a regression model. Because compliance has been shown to correlate with P DC , it makes sense to include the bending resistance and not SCT, grammage, or another correlated quantity. Our first attempt was the ordinary least squares model where the hat indicates an estimated value, w, d and a are defined in Figure 3 and BR is the geometric mean bending resistance. Inspecting the data of Su et al. (2012) more closely, it looks like the pressure has a linear relationship with durometer A. For a homogenous material, durometer A has an approximate linear relation with the logarithm of Young's modulus (Qi et al. 2003). In the case of thin paperboard packages, Young's modulus is not relevant, but we can use the structural stiffness instead. If we consider a simplified model of the panel as a simply supported beam of length L and bending stiffness EI, the compliance, i. e. the inverse of stiffness, at a point a distance a from one of the supports is In the current case, the inset a is small compared to L so a purely multiplicative model containing the factors w, h, a, and BR would probably be a good fit for estimating the compliance. Therefore, we take the logarithm of all the explanatory variables, but not of the predicted variable. The final model becomeŝ For the sake of completeness, both models were considered in the results. The same model structures were also used for the sum of electrodes 7-10.

Linear discriminant analysis (LDA)
Linear discriminant analysis (LDA) is a supervised dimensionality reduction method that seeks to find the linear subspace of the original feature space that maximizes the difference between classes. In contrast to principal component analysis (PCA), a related unsupervised method for dimensionality reduction, LDA will only retain variance that is meaningful, i. e. has predictive power (Hastie et al. 2001).
In this case, we used LDA to determine which electrodes are informative in distinguishing between packages made from board of varying bending stiffness. While we have already provided hypotheses for this and shown the correlation with bending stiffness, we wanted to verify that these are in fact the electrodes with the most explanatory power. We first applied a scaling transformation to the dataset, bringing the mean to zero and all variances to 1. We then applied LDA, defining each combination of package and grammage as one class, ignoring differences in inset a. This will make sure that the extracted combined features are robust to slight differences in a, which will be useful since the alignment of this dimension is difficult to get accurate.
LDA makes the important assumption that the covariance matrix is identical for all classes. Considering that we have relatively few samples of each class, we do not have enough data to be able to reliable estimate the differences between classes so this assumption should be sensible.

Results and discussion
The values from the BioTac were evaluated at a snapshot when the global force reached 4 N, before the appearance of any visible damage on the packages. The results for P DC and the sum of electrodes 7 to 10 (which would be most sensitive to tilt of the package surface away from the supported edge of the package) are shown in Figure 4 and Figure 5, respectively. Both these variables show similar trends. The level differences between the different data series are explained by the geometric terms in the regressions. The lower values for P DC when packages are stiffer Figure 4: DC pressure at 4 N versus the logarithm of the bending resistance of the board. P DC at a given force is essentially a measure of contact area. Increasing bending stiffness decreases contact are and thus gives lower P DC . Figure 5: Sum of the electrodes E 7 -E 10 at 4 N versus the logarithm of the bending resistance of the board. Increasing bending stiffness leads to smaller contact area and hence more liquid displaced around the contact. Displacement of liquid translates into higher impedance, which is digitized by the BioTac as a lower binary value.
can be explained by the more localized displacement of the skin of the BioTac. If the paperboard bends more easily it can envelop the BioTac and the force will be transferred through the fluid inside the BioTac, raising the pressure. On a more rigid surface the artificial fingertip deforms, and the outer skin makes direct contact with the core. Force can then be transmitted directly without raising the pressure in the fluid. This can also be seen in the increased impedance measured by the tip electrodes. These results match with what has been reported earlier by Su et al.  (2012), who measured the response of the BioTac when contacting rubbers of various compliance and found a similar relationship between compliance and P DC . In general, the three packages were easily distinguished from each other by both the P DC and electrode sum measurements, see Figure 4 and 5 where blue, orange and green marks form separated clusters. The effects of distance of the contact point from the edge of the package a was relatively small. Ordinary least squares showed that the coefficients for all the tested predictors were significant to the p < 0.001 level. The fitted coefficients with confidence intervals are summarized in Table 3. Finally, Figure 6 shows a comparison between the measured numbers and the fitted P DC . The figure shows that most variance has been explained, except at the lowest bending resistance, corresponding to the highest P DC . This shows that there is a potential lower bound to the sensitivity of this device, corresponding to full enveloping of the sensor, similar to what a fully flexible membrane would produce.
The evaluation force of 4 N was chosen to be close to the force at the thumb when lifting Package I, the lightest package in phase one, weighing approximately 400 g. While the other two packages require a greater force to lift, we chose to use the same force so that the values can easily be compared. Humans tend to apply minimal estimated grip force initially and then adjust force later as needed to maintain stable grip during lifting as sensory information related to weight becomes available (Johansson and Flanagan 2008). Figure 7 shows a visualization of coefficients of the first discriminant function, which explains 85 % of the variance. LDA confirms that the most important sensors are the pressure and the tip electrodes 7-10. However, the tip electrodes should have alternating signs, effectively computing a second spatial derivative in the distal direction. This analysis gives a rigorous motivation for the sensors we choose to study in detail.

Linear discriminant analysis
This fits well with the idea that similar to the presumed sensory function of the apical tuft during precision grip in humans, these electrodes have a strong sensitivity to the angle that they are making to the surface (Su et al. 2016).
The difference in magnitude of the coefficients for electrodes 8 and 9 is likely due to asymmetries in the structure of the packages tested. In general, there is no reason to believe that there would be a preferred direction of asymmetry, so it could be argued that when designing a general method one should constrain the coefficient for these electrodes to be equal. Figure 8 shows the samples projected to the space spanned by the first two discriminant functions. The LDA algorithm has ordered the samples by board stiffness. Comparing the bottle packages, the large bottle package comes out as less stiff than the small bottle package with the same material. For example, the smaller package (Package II) made from Material F is ranked similar in the first discriminant to the larger package (Package III) made from Material G, despite Material F having only 65 % of the bending resistance of Material G. Note that the labeling of the training data does not contain an explicit ordering of the classes by stiffness or other measure. Thus, the fact that the algorithm has ordered the classes according to stiffness suggests that this information has been inferred from the tactile sensor data.

Further work
Further work is needed in order to make the method more reliable and to correlate its results with human perception and preferences. An automated method to press the finger at a known, constant velocity or gradually increasing force would enable interpretation of the P AC signal that might reveal sudden events such as crumpling within the paperboard. It should also make the other signals more reliable because paperboard is a rate-dependent material rather than purely elastic.
If the method described here were combined with a measurement of the overall displacement and force of the finger, it might be possible to combine cutaneous and kinaesthetic information in a manner more similar to human perception (Bergmann Tiest and Kappers 2009).
It would also be interesting to investigate the data over time as the force ramps up to maintain stable grip of heavier packages. This analysis may present more information than earlier methods have given. For example, it could be possible to track the influence of factors such as crease quality, as well as transient events such as localized yielding.

Conclusions
We have demonstrated that the BioTac sensor can be used to rank the compliance of paperboard packages. Both the low-pass filtered pressure and the electrode impedances correlate with predictors of the compliance locally around the point of contact. Its elastomeric skin inflated by a displaceable liquid pulp presumably provide a more biomimetic interaction with test objects than a rigid probe.
We have seen that it is possible to construct a feature from the BioTac that can be used to rank the compliance of packages using the four electrodes at the tip of the BioTac and the low pass filtered pressure signal.
By aid of only the electrode and pressure result at a single point in the force ramp-up, it is possible to discriminate the different packages and the different materials. This is promising for further analysis of the signals from individual electrodes