The feasibility of using a low-cost depth camera for 3D scanning in mass customization

Abstract By combining additive manufacturing with 3D measurement techniques, tailored production work-flows that include the digitizing of existing components, computer-aided design, and tool-free manufacturing of the customized parts can be envisioned, potentially reducing the costs of mass customization. The introduction of affordable depth cameras has greatly increased the consumer availability of 3D measuring. We present the application of an affordable depth camera for the 3D digitizing of existing components, the utilization of the produced data in the design process, and finally, the production of the designed component with additive manufacturing. The capabilities of the affordable depth camera system are evaluated by comparing it with photogrammetric 3D reconstruction, revealing issues in smaller geometric details and sharp edges.


Introduction
Mass production remains the dominant way of producing consumer goods. The reason is the significant economies of scale that arise when producing large quantities using dedicated tooling and machinery. Dedicated equipment also implies that changes to the product become laborious and expensive. Producers may resort to modular designs where different interchangeable parts are used in the final assembly to create product variants. If the assemblies are made to order, it is possible for the consumer to choose from a list of options. Well-known examples of this approach are cars and computers. This selection of readymade options is often called mass customization, but it should be noted that the components of the assembly are already produced and cannot be changed. Variations are only possible in the assembly, and there they are limited by the availability of components. Genuinely tailored production is expensive as the design and tooling phases use a lot of resources [1]. This has, to date, kept the volumes of tailored production fairly low.
Additive manufacturing (AM) requires no tooling and therefore design changes do not lead to additional costs in the manufacturing process itself [2]. While there are costs related to generating new toolpaths and tweaking process parameters for new designs, they remain low compared to the cost of tooling. Further, the emergence of affordable AM equipment, utilizing fused filament fabrication (FFF), has made AM highly accessible [3]. Affordable systems have been found to produce results comparable with those of professional-grade systems [4]. However, designing new parts for each customer is time-consuming. Hence, customized products may be priced outside the reach of the masses, even with affordable AM. Wider realization of mass-customized manufacturing requires further automation of the design process. Parametric computer-aided design (CAD) is a good solution for stand-alone products, but reverse engineering is needed if the part interfaces with third-party products. Three-dimensional scanning allows reverting existing shapes into digital form and subsequently manipulating them to give the needed geometry data for production by AM [5]. Not only do AM and 3D scanning potentially make mass customization faster, but the required equipment and software have come down to a price level that makes it possible for consumers and small businesses to produce parts for themselves or for others in small volumes [6].
The 3D measuring of objects' geometry has been accomplished with a variety of methods, including imagebased 3D reconstruction [7], time-of-flight ranging with laser light, or triangulation using either laser or visible light (see, e.g., [8][9][10]). Triangulating measurement with a visible light-projected pattern is often called structured light [10]. The use of 3D scanning has been studied over a wide area of applications, including archaeology and cultural heritage preservation [9][10][11], sports [12], forensics [13], and healthcare [14]. A depth camera involves active 3D imaging sensor technology, characteristically capable of high measurement rates, producing several tens of images with depth information per second. Typically, depth cameras utilize either the time-of-flight principle with an imaging sensor [15] or the structured light principle with a projected light pattern [16]. Following the introduction of Microsoft Kinect, depth cameras became available for consumers. This also stimulated a large amount of research on their application in robotics [17,18], indoor mapping [19], and 3D object scanning [20].
After Kinect, mobile depth cameras (such as Structure Sensor [21]) and fully integrated systems (such as Google Tango; see, e.g., [22]) have been released on the market. These devices operate in conjunction with smartphones or tablet computers, thus increasing their mobility. Often, they are accompanied by dedicated software running on the smart mobile device, creating a highly mobile, easyto-use system. To some extent they are already targeted towards consumers, unlike most commercial 3D scanners, which are more applied for specific professional purposes due to their price (see, e.g., [20]). In the relevant literature, experiments performed with these affordable systems [22][23][24] have been presented, but actual case examples of applying these systems are yet to be reported.
Our aim is to combine affordable 3D scanning, mesh modeling, and AM in order to facilitate efficient customization. The process is tested in a presented example case, customizing an adapter component. We present the use of use of an affordable depth camera sensor, mounted on a tablet computer for the 3D digitizing of existing components. The results of the 3D scanning are applied in mesh modeling, producing the geometry of the customized component that is finally produced with AM. To further evaluate the quality of data obtained with the affordable 3D scanning instrument, a comparison against a dense photogrammetric point cloud is presented.
The presented example case concerns customization of sea chat plotter mounting adapters for jet-skis. Jet-ski riders frequently want to retrofit sea chart plotters on their jet skis. The mounting is usually done by cutting an open-ing in the fairing or glove compartment hatch, which are typically high-gloss injection-molded polycarbonate parts. The sea chart plotters have prismatic shapes with a flat screen whereas the fairings on jet skis are typically complex double-curvature shapes. Therefore, an adapter is needed between the two. There are several jet-ski manufacturers with multiple models that may differ between the model years as well. Similarly, there are multiple chartplotter manufacturers with various models and various screen sizes. Consequently, the number of possible combinations is large while the required production volumes are relatively low. Nowadays, the adapters are made manually by craftsmen, but the labor costs lead to a high-priced product. Investing in expensive tooling for mass production, such as injection molding, is not economically feasible. In contrast, the manufacturing costs using AM are not dependent on the number of product variants. However, AM requires a 3D model of the product. The model needs to take into account the shape of the chart plotter and the shape of the jet-ski fairing. The chart plotter is easily modelled by taking a few measurements of the mounting surface but the complex curvature of the fairing requires a technology like 3D scanning for accurate reverse engineering.
The adapters are niché products that do not interest the jet-ski manufacturers or the chart-plotter manufacturers. However, there is a demand in the market for such a product and the approach described here is used by RD Physics Ltd. to design, produce, and sell chart-plotter adapters. It shows that the technology allows agile startups to enter the market with a small investment in equipment. Large production facilities are no longer needed to start selling a product like the chart-plotter adapter. Only a small investment is required to get started with AM [6].

The Structure depth camera sensor
The 3D digitizing of the jet-ski fairing component for reverse engineering was performed with the Structure sensor [21] attached to an iPad tablet computer ( Figure 1). The Structure Sensor is a compact depth camera with a range of 0.4 to 3.5 meters and an accuracy ranging from 0.5 mm (at 40 cm) to 30 mm (at 3 m) [21]. The manufacturer's software was used for the scanning (by applying the "Product scanning mode") and for limiting the scanned area as tightly as possible to the component being digitized. The component was mounted on a small pedestal to simplify  separation from the floor surface. As the part itself contained relatively few features, natural rocks were placed surrounding to object to provide additional image and geometric match points.
The scanning was first attempted with the object left as it was. As the Structure Sensor was unable to reliably pick up the surface of the artifact, it was lightly coated with white chalk powder prior to scanning. The same coating and scan setup ( Figure 2) was also used for the photogrammetric reconstruction described later. The scan was processed in the mobile device with the the Structure sensor software. The resulting 3D mesh surface contained a total of 24,489 vertices and 46,763 faces.

The application of parametric CAD and mesh modeling for design
To facilitate an efficient design process, a mesh-based workflow was revised for customizing the adapter component for mounting the chart-plotter unit using the 3D scanned surface of the fairing. First, a template mesh of the adapter component was produced via manual CAD modeling, based on the dimensions of the flange in a 7" Simrad chart plotter. The flange geometry is simply extruded in three dimensions to generate a solid volume on which the plotter can be attached. Secondly, the geometry of the fairing is obtained, as described, via 3D scanning. After this, the design of the customized component is performed by a fusion of the template mesh model and the 3D scan of the fairing surface using mesh modeling in Autodesk MeshMixer [24] (Figure 3). The intersecting geometry is generated using the Boolean difference function and the resulting mesh is saved. Finally, the edited mesh is utilized in Simplify3D [26] for toolpath generation, after which the AM can be performed to produce the final component.

Photogrammetric reconstruction
In addition to 3D scanning, photogrammetric reconstruction of the fairing component was performed by utilizing a conventional digital single-lens reflex camera, a Nikon D810 camera with an AF-S Micro NIKKOR 60 mm f/2.8G ED lens. The Nikon D810 has a 36.3 megapixel sensor with pixel size of 4.88 µm. An aperture of f/32 was selected for a sufficient depth of field. For optimal image quality, the ISO sensitivity value of 100 was chosen. A relatively small aperture causes a long exposure time. We used 2.5 s shutter speed and a tripod to avoid motion blur. To capture the highest quality image, we captured 14-bit NEF images, which needed to be converted into the JPG format for post-processing. The lateral chromatic aberration and vignetting were corrected in the conversion process. The camera was pre-calibrated in order to obtain a highaccuracy reconstruction.
Agisoft Photoscan (version 1.3.4) was applied for the point cloud reconstruction of the top surface of the fairing component, using a total of 26 images. When the image alignment accuracy was set at its highest, the root-meansquare error in re-projection was 0.319 pixels, with a maximum error of 5.64 pixels. The scale was determined by measuring a handle component of the part using calibers. With the "ultra high" quality and "aggressive" depth filter-

Results
With the presented methods, the stated design case was carried out successfully. In addition, dense reference data was obtained for the fairing component, facilitating performance evaluation of the affordable 3D scanning system in the presented case.

Mass customization via 3D scanning, mesh modeling, and AM
The affordable depth camera was applied for the 3D digitizing of the plastic fairing component of the jet ski in order to form a mesh surface model. The presented modeling  process allowed the direct use of the mesh produced with the iPad software of the Structure Sensor system, applying the mesh of the fairing surface to trim the pre-made template model ( Figure 5). The resulting mesh representation of the adapter component's geometry was utilized in toolpath computation, after which the final component was produced with AM. The completed piece is shown with the original fairing and the chart-plotter unit ( Figure 6). Manufacturing was performed using FFF with an Ultimaker 2+ [27].

An evaluation of the geometric accuracy of a consumer-grade 3D scanner
The geometric accuracy of the mesh model obtained with the Structure Sensor was assessed by comparing it against a photogrammetric 3D point cloud. CloudCompare (version 2.8.1) was used in processing the data sets and performing the analysis. The comparison was performed by first segmenting both data sets to remove partially scanned surrounding structures and then roughly registering the data sets that were to be compared by manually orienting them. After this, iterative closest point (ICP) was ap- plied for fine-scale registration. After registration, a meshto-cloud function was applied for evaluating the distances between the Structure Sensor mesh and the point cloud. In analysis (Figure 7), a mean distance of 0.92 mm was found between the data sets, with the standard deviation being 2.64 mm.
The results show how the strongly simplified Structure Sensor mesh tends to produce corner-rounding artifacts, which are visible in both comparisons. The smoothing al- gorithm also appears to inflict deviations in smooth surfaces near corner regions. On flatter surfaces, the strongly smoothed mesh is better able to represent the actual shape. The strong corner rounding is also apparent when looking at co-registered cross sections (Figure 8).

Discussion
The presented case displays the potential of the emerging affordable and easy-to-use 3D sensors for performing 3D digitizing for mass customization. Both the equipment and software applied are, in terms of costs, available for smaller engineering and design companies, or even enthusiastic consumers. However, some requirements still apply when using them in real-life cases. Firstly, the material of the component being scanned proved, as such, difficult to 3D digitize. This was solved by applying a removable coating, but this still presents an extra step in the process. Secondly, the smooth, fairly flat top surface of the component being scanned was geometrically simple to represent as a mesh surface model and did not require a high level of detail from the scanning system. Had the component contained intricate details, the requirements for the detail level of the 3D scanner and the modeling workflow would have been much greater. Most likely, this would have prevented the use of the affordable 3D scanning system due to limitations in the detail level obtained. In such a case, the use of macro photogrammetry (for reference data) or a more capable 3D scanning instrument would be required. Thirdly, even though the technology is highly available for both consumers and professional users, the need of know-how regarding the requirements of 3D digitizing has not completely disappeared. In the presented case, this is visible in the coating that needed to be used and the scanning setup that was constructed. It is difficult to estimate whether a user with no previous 3D scanning experience would have coped with the same issues. In the presented case, the scanning was performed by users with considerable previous experience with 3D scanning and photogrammetry.
Looking at the performance comparison, the Structure Sensor's low resolution mesh showed deviations from the photogrammetric point cloud. The deviations were, in most cases, below 1 mm, correlating with the accuracy stated by the manufacturer. A larger issue was encountered near the small details and sharp corners of the scanned piece, where the low mesh resolution combined with the smoothing of artifacts caused more extreme deviations. These also explain the high standard deviation of 2.64 mm. This indicates that the sensor should ideally not be applied for digitizing small artifacts with sharp details.
The utilization of mesh models was a central enabler in the presented process. As the toolpath generation for AM is most commonly accomplished from a mesh, a higherlevel CAD representation is not required for AM. This allowed the design of the customized adapter component to be carried out as a Boolean operation of two mesh models, in turn enabling the utilization of a mesh produced directly with the Structure Sensor software. This avoided the more complex reverse engineering process that is associated with producing a CAD representation (see, e.g., [28]) from 3D digitized data, which requires dedicated software tools (see, e.g., [29]).
Several applications can be envisioned for the presented combination of affordable 3D-scanning, mesh modeling and AM. Similar adapter components might be manufactured for other marine vehicles, or for the automotive sector for installing third-party devices on vehicles. The combination of reverse-engineering and affordable oneoff manufacturing might also find use in the manufacturing industry for producing assembly jigs and machining supports. For example, the automotive industry has been utilizing AM for producing assembly aids to enhance productivity [30]. Individually tailored support structures that have to conform to existing, complex geometries might also be applicable in the medical sector. An example of utilizing 3D scanning with additive manufacturing for producing individual arm splits is presented in [31].
As mentioned, the currently known drawbacks for the proposed solution include the limitations of geometric accuracy and surface material compatibility encountered with affordable 3D scanners based on depth camera technology. These systems might not be applicable with smaller components, pieces that contain a high amount of small details, or objects with high gloss surfaces. In addition to system performance, the operator experience has an influence on the quality and completeness of models obtained with hand held 3D scanners [20].
Further, as the pieces are manufactured with FFF, the limitations of existing FFF machines apply. While in many cases their build accuracy exceeds that of affordable 3Dscanning [5,32], they are limited in the size of the build envelope, minimum wall thickness, and their ability to support overhanging geometry in pieces to be manufactured [5]. It is also worth to acknowledge, that the orientation of manufactured piece is significant in FFF, as the dimensional accuracy [32], mechanical strength [33] and surface roughness [34] have been found to vary by build orientation. Clearly, there are also know-how requirements associated with utilizing FFF. On a practical level, affordable machines have also been reported to suffer from low reliability [35].
The presented work conforms with existing research literature concerning the use of affordable 3D-sensing techniques, where such systems have been found to be potentially applicable for various purposes [19,23]. The combination of near consumer 3D measuring, simple modeling workflows, and AM has potential for mass customization in distributed manufacturing [3] and for empowering users and hobbyists to carry out tasks that have traditionally been associated with professional product development, binding it to notions of open innovation [36], the involvement of the user in product development [37], and the maker movement [38]. If applied on a wider scale, these developments may affect manufacturing [3] and ecommerce [39] in the future. A typology for potential future users of consumer 3D printing includes the designers of completely new pieces and users producing pieces that connect to existing products [40]. Especially the later one represents a significant future prospect for integration of 3D scanning and AM, as 3D scanning has the potential to reduce the amount of work in designing such pieces. Further, the design of new pieces may also occur by mixing geometries of existing artifacts [41], potentially benefiting from 3D scanning.
As hobbyists have adopted AM as fabrication method, the need for 3D models has become apparent. Major industrial players have clearly acknowledged this potential market, with Autodesk having acquired the building instructions website, Instructables [42] and 3D printer manufacturer Stratasys the CAD community website GrabCAD [43]. The 3D models used for AM have to be designed with CAD software or downloaded from a repository (like GrabCAD). The development of consumer-grade 3D scanners enables a third way of obtaining the 3D model, as shown. This option has not yet gained popularity compared to the other two routes, but is a potential future development direction. As existing consumer depth cameras have shown potential for digitizing indoor environments [44], the development of AM and integration with 3D scanning may open new opportunities for customization in other sectors, such as construction [45], as well.

Conclusions
AM techniques allow for cost-effective mass customization, through significantly reduced tooling costs. The combined application of 3D reconstruction and AM is potentially beneficial for mass customization scenarios where the geometry of existing components has to be followed. We have shown a case where this combination proved to be feasible in producing a customized component.
With affordable depth cameras, 3D sensing has become available for consumers and hobbyists. The results presented here describe the performance of an inexpensive 3D scanner. This information can be used to evaluate whether or not such devices are sufficient for a particular application. In the presented case, a highly affordable (retail value: approx. $400) 3D scanner was applied for the 3D digitizing of a geometrically complex plastic part. The resulting mesh model of the component's top side, featuring a complex dual curvature shape, was used in designing an adapter component, which was finally manufactured with AM technology.
When comparing the geometric reconstruction results against a photogrammetric point cloud, the performance limitations of the affordable, integrated depth camera system were found to be most prominent in regard to sharp corners and small geometric details. For smooth surfaces with limited detail, the deviations remained below 1 mm.
The presented combination of 3D scanning with an affordable instrument, mesh modeling, and AM proved feasible for the customization of components with affordable tools.
Acknowledgement: This research project was supported by the Academy of Finland, the Centre of Excellence in Laser Scanning Research (CoE-LaSR) (No. 272195, 307362). The Strategic Research Council of the Academy of Finland is acknowledged for financial support for the project, "Competence-Based Growth Through Integrated Disruptive Technologies of 3D Digitalization, Robotics, Geospatial Information and Image Processing/Computing -Point Cloud Ecosystem (No. 293389, 314312)," and the Business Finland is acknowledged for financial support for the innovation project "VARPU" (7031/31/2016). The European Regional Development Fund is acknowledged for the support of the "Creve 2.0" project (S21338).