Reproducibility of C. I. P.-compatible dynamic pressure measurements

To ensure the safety of users like hunters and sports shooters, the dynamic pressure inside an ammunition cartridge must not exceed a maximum value. We have investigated the reproducibility of the dynamic measurement of the gas pressure inside civilian ammunition cartridges during firing, when following the rules formulated by the Permanent International Commission for the Proof of Small Arms (C. I. P.).We find an in-house spread of 0.8% between maximum and minimum pressure for runs with the same barrel and of 1.8% among a set of three barrels. This sets a baseline for the expected agreement in measurement comparisonsbetweendifferent laboratories. Furthermore, a difference of more than 3% is found in a preliminary study of the influence of ammunition storage


Introduction
Hunting and target shooting are both a popular pastime and an economic factor. To ensure that these activities do not pose any undue risks to the user of the gun, the gas pressure that develops when the ammunition cartridge is fired must not exceed a safe value. The same holds for technical processes driven by propellants, for instance powder-actuated tools for fastening nails.
Therefore, in many countries there exists an infrastructure that makes sure that each new lot of ammunition is tested for compliance with the maximum pressure allowed for a particular caliber. In 1914 an international treaty established the Commission internationale permanente pour l'épreuve des armes à feu portatives (Permanent International Commission for the Proof of Small Arms, C. I. P.), with members spanning the globe from Chile to Russia. One of the tasks of the C. I. P. is the regulation of methods for safety testing of firearms, including that of ammunition for powder-actuated tools and for civilian guns. Core of the international treaty is that each member accepts the weapons and ammunitions, without further testing, that have been tested according to C. I. P. standards by a proof house of another member state. Therefore, the C. I. P. is one building block of a world without technical barriers to trade, preceding the Mutual Recognition Arrangement (CIPM-MRA) of the International Meter Convention by almost a century [1].
Acceptance of each other's measurements and tests relies on the mutual trust in the measurement capabilities of each proof house. One mechanism to build this trust is the conduction of measurement comparisons. Each participant receives ammunition from the same manufacturing lot and measures the maximum pressure (and the bullet velocity for ammunition with bullets) with his local measurement setup. This allows the degree of compatibility between proof houses to be established.
In the past years C. I. P. has organized a number of international comparisons for different calibers. However, within such a comparison the results of the different proof houses can span a range on the order of ten percent. Therefore, it cannot be excluded that in one country a lot of ammunition might be found to exceed the allowed maximum pressure and therefore would be rejected whereas it might pass in another country.
The spread in measurement results so far has been attributed to the fact that each proof house uses their own test barrel, procured from any of a number of manufacturers who apply the C. I. P. specifications for its geometrical dimensions. The maximum pressure and the bullet velocity are affected by mechanical tolerances and by the effects of mechanical wear on test barrels. However, there is no specific evidence that it is only the barrels that are to blame for the spread of measurement results.
To improve the situation, C. I. P. decided to launch a comparison where each participant uses a new barrel with nominally identical dimensions. Each barrel, including the cartridge chamber, is forged by a single manufacturer around the identical hammering pin, so that the barrels should be basically identical, without variations due to the manufacturing process, in particular from using cutting tools. Each participant will be following a detailed set of rules for preparing and firing the ammunition. This comparison should reveal whether there are other factors contributing to the spread of measurement results that previously might have been dominated by barrel-to-barrel variation. The results of this comparison are not publicly available yet.
To support the interpretation of future results, in a first step it is important to establish the base line, i. e., the level of reproducibility within the same proof house. When two or more of the nominally identical barrels are used by the same team to measure the pressure of an ammunition lot there are fewer parameters that could differ from one run to the next, compared to measurements taking place in different proof houses.
Here we report on such a series of measurement runs aimed at establishing a reproducibility baseline. We have used three barrels of caliber .30-06 Spring. (a common hunting caliber), forged on the same hammering pin, and performed several runs using parameters as identical as we can make them, including having the same persons performing the same process steps. In the end, we arrive at a variability that is much lower than the spread of measurement values observed in previous measurement comparisons. In addition, we point out the influence of ammunition storage conditions, a parameter that has not been controlled in previous comparisons although it can lead to significant deviations of measurement results between different proof houses.

Measurement setup
The pressure pulse inside a cartridge typically is a few milliseconds long and has a peak amplitude in the range of several hundred MPa ( Fig. 1) [2,3]. C. I. P. has developed a set of rules [4] on how to measure the maximum pressure, with a target uncertainty of 3 % (coverage factor k = 1). Basically, a hole is drilled in the side of the cartridge case (shell) so that the combustion gases can reach a piezoelectric pressure sensor via a small channel drilled through the side wall of the cartridge chamber (Fig. 2). Immediately after drilling the case, the hole is covered with a single layer of self-adhesive Kapton tape to protect the propellant until firing. Because the cartridge is hand-loaded into the test barrel the tape does not interfere with the functioning of the barrel. During firing, the hot combustion gases quickly perforate this tape and reach the pressure sensor. A charge amplifier translates the sensor signal into a voltage which is recorded and its maximum as a function of time is determined. Simultaneously, a pair of optical screens (light barriers) located a few meters downstream from the barrel measures the bullet velocity via the time needed to travel the distance between the optical screens.
Physikalisch-Technische Bundesanstalt (PTB) uses an underground shooting range where civilian ammunition can be fired and the pressure as well as the bullet velocity measured. A formal evaluation of the measurement uncertainty for the pressure measurement chain has been completed recently [5]. It amounts to 0.34 % (one standard deviation, i. e., coverage factor k = 1 [6]) for two commonly used pressure sensor models. This value does not include effects due to deviations of the barrel from specification or within manufacturing tolerances, nor due to the variability of the ammunition itself.

Experimental procedure
For the internal comparison three newly forged barrels (A, B, C) in a total of 8 runs of 49 shots each were used ( Table 1). Runs 1-3 were performed with a freshly brokenin barrel and were preceded by an additional 20 priming shots from each barrel. For runs 4-8, only barrel B was used.
Each run consisted of firing two shots with undrilled ammunition to warm up the barrel. Data taking began by firing 10 undrilled shots and measuring the bullet velocity. For all undrilled shots, the sensor was replaced by a "dummy" metal plug that also filled the small bore hole in the side of the barrel.
Next, the sensor was inserted and 32 shots with drilled ammunition were fired, where both velocity and pressure were recorded. Finally, another 5 undrilled shots were fired to monitor the velocity again. The time interval between two undrilled shots was less than 30 s, and between two drilled shots it was less than one minute. The extra time for drilled shots is needed to ensure that the cartridge is rotated into the correct orientation to line up the hole in the case with the channel leading to the pressure sensors.
The pressure was measured with a piezoelectric sensor, in most cases a Kistler model 6215 sensor (the same one for each run), with the exception of run 3 where an HPI model GP6 sensor was used. Both sensors had not been used before, except for calibration at the manufacturer's facility during the normal delivery process.
The charge signal of the pressure sensor is amplified with a commercial charge amplifier. The amplifier output is recorded with a transient recorder at high bandwidth (40 MHz). A digital low-pass filter (second-order Bessel filter with 20 kHz corner frequency) is applied and the maximum value of the filtered signal registered as the result of the pressure measurement for this cartridge.
All ammunition was taken from the same lot and stored at PTB in the dark at 21°C (actively stabilized) and 35 %-65 % relative humidity (not stabilized) for a period of 18-24 months since delivery (storage conditions I). For runs 5 and 7, the ammunition of the same manufacturing  lot had been stored at the manufacturer's site (storage conditions II; details not available) and sent to PTB only a few days before the respective shooting run.
In the morning of the day before each run, the required number of cartridges was taken from local storage, 32 of them drilled and sealed according to C. I. P. rules, and then both drilled and undrilled cartridges placed in a climate chamber at 21°C and (60±5) % relative humidity for a conditioning time of at least 24 hours.
For each drilled shot, the maximum pressure was determined from the pressure curve ( Figure 1). The histogram in Figure 3 illustrates the typical shot-to-shot scatter of the maximum pressure, in this example with a standard deviation of 7.8 MPa (2.2 %). The arithmetic mean, p max , is depicted by the red dot above the histogram, including an error bar corresponding to the standard error of the mean. For comparison, the blue error bar (without data point) gives an indication of the 0.34 % measurement uncertainty of the pressure measurement chain [5]. The latter is much smaller than the variation between individual shots of the ammunition lot and comparable to the standard error of the mean. Therefore, the number of 32 drilled shots is a suitable compromise between taking ever more data and the resulting statistical uncertainty.
The measurement uncertainty of the pressure measurement chain is dominated by the calibration error of the sensor itself: it contributes about 90 % to the total uncertainty. Therefore, the errors due to the measurement chain are highly correlated in the experiments discussed here. Since we are interested only in the differences between runs, in the following we can restrict ourselves to considering only the statistical uncertainty due to the shot-to-shot scatter.

Barrel-to-barrel variability
The arithmetic mean of the maximum pressure of the 32 shots in each run is depicted as p max in Figure 4. The error bars show the empirical standard error of the mean. The first three data points correspond to the barrel-tobarrel variability. The relative difference between highest and lowest p max is only 1.8 % ( Table 2). This value is much lower than the spread observed in previous comparisons among different laboratories. The only known difference between the measurement runs reported here is that different barrels were used, apart from the unavoidable fact that each run is shot with a different sample of 49 cartridges drawn from the manufacturing lot. We can therefore consider the 1.8 % spread as a baseline for measurement comparisons between different laboratories; currently, better agreement than this number is not to be expected.
However, this is not necessarily the best possible limit. A χ 2 test gives a probability p = 0.95 for the weighted average of the three p max values for the three barrels (runs 1, 2, 3). In this sense, the three data points are not compatible within the stated (statistical) uncertainties. When the data point for barrel B (run 2) is replaced by the weighted mean of runs 2, 4, 6, and 8, the inconsistency for the weighted mean p max of the three barrels is even more pronounced (p = 0.99), although the barrel-to-barrel variability reduces to 0.99 %. This shows that there is an additional, unidentified parameter that influences the experimental value of p max . It would be interesting to see whether the (unknown) residual geometrical variation in the test barrels, which are nominally identical, is so large that it affects p max to this degree. Measurements of the geometry of about 15 test barrels manufactured on the same hammering pin as our barrels A, B, and C have been performed, but the results have not been disclosed yet [7]. Alternatively, it could be a sampling effect with respect to the vari- ability or long-term drift during manufacturing of the ammunition lot. But already now the spread is about ten-fold smaller than in previous C. I. P. ammunition pressure comparisons. So the decision to use hammered barrels and a more detailed procedure was a good one.

Reproducibility
Runs 2, 4, 6, and 8 were shot with barrel B, so the only nominal difference between them is that the wear of barrel and sensor increase from run to run. However, neither for the pressure nor for the bullet velocity a trend within each run or in the averages can be observed, so wear does not play a role here. The spread of maximum pressure values, i. e., between runs 2 and 4, is only 0.8 % (Fig. 4). The weighted average of the four runs passes the χ 2 test (p = 0.45). Therefore, this value of 0.8 % sets a baseline for the variability under identical conditions. It is consistent with the measurement uncertainty of the pressure measurement chain.
The fact that the reproducibility is better than the barrel-to-barrel variability supports the conclusion made in the previous section that the remaining spread is at least partially due to an unknown parameter connected to the geometry of the barrels, even for barrels manufactured on the same hammering pin.

Ammunition storage conditions
It is well-known that the performance of ammunition depends on the environmental conditions to which it was exposed during and after manufacturing. This could contribute to the spread of values in a measurement comparison between different laboratories. When the ammunition is sent to the participating laboratories, typically it is stored there for weeks or months before the actual shooting takes place. These storage conditions are not standardized. Part of the shooting procedure is the conditioning of the cartridges under defined thermal and moisture conditions, but this conditioning of typically one day in length cannot undo whatever happened in the months before.
To get an idea about the magnitude of the influence of storage conditions, two additional runs with barrel B were performed, using ammunition from the same manufacturing lot but which had been stored at the manufacturer's site until a few days before the shooting at PTB (storage conditions II in Table 1). What exactly constitutes the difference compared to storage at PTB is not clear at the moment and will need to be investigated further in a dedicated (and necessarily long-term) study.
Nevertheless, already within the present preliminary study one can see a clear effect (Fig. 5). The two additional runs were interspersed within the series as runs 5 and 7. One finds a significantly higher p max for ammunition stored under conditions II than under conditions I (χ 2 = 80.9, p ≈ 1, for one degree of freedom). Apart from storage conditions, there is no known difference between the runs. The exception is that for later runs the barrel and sensor had experienced more shots. However, within each run there is no trend towards higher or lower velocities or pressures. The same goes for the averages for each run, as evidenced by the zig-zag pattern of p max in runs 4-5-6-7-8 (Fig. 5).
An interesting observation is that the correlation between pressure and bullet velocity has a different slope for the ammunition samples stored under conditions I and II (Fig. 6). Also, the scatter between individual shots is smaller both for p max and for v for the ammunition sample stored under conditions II. Given that the ammunition lot has a size of 120 000 cartridges one could speculate that the different behavior of the samples taken for runs 5 and 7 from the other samples might (at least in part) be due to some variability across the lot, perhaps some long-term drift during manufacturing conditions, and not (only) due to the storage conditions.

Bullet velocity
For an individual shot, p max and bullet velocity are correlated although the correlation is not perfect (Fig. 6). The velocity of the bullet was measured for drilled and for undrilled cartridges. The averages show the same general dependence on barrel and ammunition storage conditions as the p max values, so they are not discussed here any further. The mean values for drilled cartridges are included in Table 2.
No difference between runs with different barrels or different storage conditions is observed in the so-called loss of velocity, i. e., the difference between the average velocity for drilled and undrilled cartridges (Fig. 7). A small loss is unavoidable, due to the loss of gas into the channel leading to the pressure sensors. When the drilled hole is not sealed properly after drilling (see Section 2) there might be additional gas leakage from the case, leading to lower p max and lower bullet velocity. A small amount of leakage is unavoidable and leads to the soot deposits around the hole in the case (Fig. 2).
Here, the loss of velocity is smaller than 1 % and therefore fully within the usual range. In summary, the loss of

Conclusion
We have compared bullet velocity and maximum pressure for three different test barrels shot under nominally identical conditions. In-house, we find a reproducibility of mean pressure that is consistent with the measurement uncertainty. With 1.8 % the span of barrel-to-barrel variability between highest and lowest mean maximum pressure is about ten-fold smaller than that observed in previous measurement comparisons performed by C. I. P. The span of the run-to-run variability for the same barrel is even lower, at 0.8 %. This establishes a baseline for the reproducibility of such measurements, when compared between different institutions.
In addition, we have obtained quantitative evidence for the effect of ammunition storage conditions (or a sampling effect in large lots) on measurement results, with a pressure difference of 3.3 % and a velocity difference of 1.2 % between ammunition stored at two different sites for about 18 months but shot from the same barrel. This influence might not be surprising for the practitioner but it has never been taken into account when designing a C. I. P. measurement comparison. Clearly, this parameter must be considered in future comparisons. It might also have implications for quality assurance systems relying on reference ammunition because climatic conditions during transport and storage need to be taken into account.
It is to be expected that the quantitative results on storage effects are specific for the chemical and mechanical composition of the propellant used in the ammunition lot and for the way it is packaged in the case. Therefore, the quantitative differences will depend on the actual type and lot of ammunition. This makes it even more important to control the storage parameters, maybe even during transport.