Force estimation from 4D OCT data in a human tumor xenograft mouse model

Experimental setup

For data acquisition, the following experimental setup shown in Figure 1 was designed. A robot (H-820.D1, Physik Instrumente) for positioning the OCT field of view (FOV) relative to the tumor was employed. Note, the position of a volume does not change relative to a world reference system. Rather, the mouse which is fixed with tape to a heated bed (37∘C) was driven. The heated bed can be easily mounted to the robot with a 3D printed adapter and prevents cooling of the narcotized mouse. A high-speed OCT imaging system with an A-scan rate of approximately 1.5 MHz was used. An A-scan was defined as a one-dimensional resolved depth signal. By moving an A-Scan along both lateral axes an OCT volume can be acquired. Each volume includes 32 × 32 scanlines in both lateral directions and 430 px along the depth dimension. The physical size of each volume is approximately 2.5×2.5×3.5 mm in air and the temporal resolution was set to 100 Hz. As a surgical tool, a needle with a diameter of 2 mm attached to a force sensor (Nano 43, ATI) for ground-truth annotation was employed. The needle can be forwarded with a stepper motor along the needle axis which represents a typical surgical pushing task [6].

Figure 1:

Experimental setup for data acquisition. Left: A OCT scan head (A) for volume acquisition was employed. A robot (B) drives a heated bed (E) on which the mouse is fixed. Tumor tissue is deformed with a needle (C) which is driven by a stepper motor (C) along the needle axis. The force sensor (D) is mounted between the stepper motor and needle. Right: The skin was carefully dissected and the anesthetized mouse was fixed with tape to the heated bed (left image). The right image indicates the approximate position of the field of view (FOV).

Xenograft mouse model

Experiments were conducted on pathogen-free balb/c severe combined immunodeficient (SCID) mice (Charles River, Wilmington, MA, USA). They were housed in individually ventilated cages and provided with sterile water and food ad libitum. For injection, 1×106 viable human HT29 colon cancer cells in 200 μm cell culture medium were injected subcutaneously into the right flank. Experiments were performed on mice if the primary tumors exceeded 1.2 cm or ulcerated the mouse skin. All experiments were approved by the local licensing authority (Freie und Hansestadt Hamburg, Behörde für Gesundheit und Verbraucherschutz, Amt für Verbraucherschutz, project N037/2019) and supervised by the institutional animal welfare officer.

Data acquisition and datasets

Each mouse was anesthetized and the skin above the subcutaneous tumor was carefully removed with a scalpel. Next, the mouse was fixed to the heated bed which can be easily mounted to the robot. For each experiment, the robot positions the FOV on the tumor (Figure 1, right). Next, the surface was detected by forwarding the needle along the needle shaft direction until a force of 0.02 N was registered. The tumor was palpated by moving and retracting the needle with a distance of 2 mm while continuously OCT volumes were acquired. For data variation, we performed palpation at five different velocities ranging from 0.3 mm/s to 0.7 mm/s. Tissue deformation on perfused tissue was compared to experiments performed on dead tissue. We refer to acquired data as Ante-Mortem (AM) datasets and Post-Mortem (PM) datasets, respectively. In total 10 AM and 10 PM datasets were acquired from five mice.

For deep learning model training and evaluation, a 10-fold cross-validation (CV) scheme was employed. Each fold contains approximately 6,000 vol and represents one experiment with the five different velocities and a different location on the tumor. Iteratively, we leave out one fold for validation, one fold for testing and train the deep learning model on all other folds. The final performance is expressed as the mean across all test folds.

Deep learning architectures

A sequence of 3D OCT volumes represents 4D data that needed to be processed. For this purpose, we employed 4D spatio-temporal CNNs that performed simultaneous spatial and temporal processing. Due to their high-dimensional nature, 4D CNNs are very parameter-intensive which might lead to a risk of overfitting. Therefore, a more efficient variant that uses factorized convolutions was employed [11]. Here, spatial and temporal processing were decomposed by using separate kernels. Thus, a fully 4D convolutional kernel of size k∈ℝt×h×d×w×c was split into a spatial kernel ks∈ℝ1×h×d×w×c and a temporal kernel kt∈ℝt×1×1×1×c where t is the temporal kernel size, h, w, and d are spatial kernel sizes and c is the feature channel dimension. This decomposition led to a reduced number of trainable parameters, however, at the same time, representational power was reduced as the decomposed kernel can only represent separable kernels. To ensure proper gradient propagation throughout the network, both CNNs were built on the ResNet principle [12] where skip connections enable a reliable gradient flow throughout network training. The two architectures ResNet4D and fResNet4D are shown in Figure 2. In each CV iteration, we trained the models for 50 epochs using Adam optimizer and a starting learning rate of lr=10−4 and batch size of b=64. The learning rate was halved every 20 epochs. Hyperparameters such as the number of layers, learning rate and batch size were chosen based on validation performance.

Figure 2:

The two 4D CNN architectures employed. The red boxes represent ResNet blocks with 4D convolutions.