Embedded inertial sensor for tracking projectile impact on granular media

Due to the opacity of most granular materials, it is often desirable to have three dimensional (3D) particle tracking techniques beyond optical imaging to explore granular dynamics. Using inertial measurement units (IMU) embedded in a projectile, we obtain the trajectory of projectile impacting on a granular medium under microgravity using tri-axial acceleration and angular velocity data. In addition to the standard algorithm for reconstruction, we emphasize solutions to various sources of error to determine projectile trajectory accurately.


Introduction
Along with the development of MEMS (microelectromechanical system) technology, inertial sensors capable of monitoring tri-axial acceleration and angular velocities nowadays become more compact, cost effective and readily available to us. They are used, for example, in smartphones, virtual reality (VR) devices, interactive game controllers, robotics, animation movie production, autonomous driving, as well as in motion tracking systems [1]. More specifically, a typical IMU device with three MEMS sensors (accelerometer, gyroscope, magnetometer) can be packed within 4 × 4 × 1mm 3 , consumes < 4 mW power, and costs < 10 US dollars (e.g., Bosch BMX160). Along with the development of hardware, there has also been a substantial progress in commercial as well as open source software packages (e.g., Matlab Sensor Fusion and Tracking Toolbox) available for the aforementioned applications in the past decade. In a standard implementation, both gyroscope and acceleration data are combined by means of dead reckoning to obtain the sensor trajectory in a predefined coordination system (note that IMU collects data in the sensor system). This process involves step-by-step coordinate transformation and numerical integration, therefore special care has to be taken to handle error accumulation [1][2][3][4].
In the realm of granular research, being able to accurately measure position and orientation of particles inside granular media is essential. Consequently, approaches such as positron emission particle tracking (PEPT) and Xray radiography have been developed [5]. For the case of projectile impacting on a granular medium with fast dynamics, we developed a continuous wave radar tracking system to detect projectile motion with high temporal resolution [6][7][8]. For a Ping-Pong ball sized projectile, Altshulter et al. innovatively embedded an accelerometer e-mail: kh380@duke.edu to monitor the trajectory of a projectile penetrating into a granular medium under reduced gravity [9]. More recently, IMU senors were also used to detect intruder motion in hopper flow [10]. However, it is always challenging to ensure the quality of tracer trajectories from IMU sensor data. Here, we focus especially on challenges in accurately determining position and orientation of the projectile center of mass, including: (i) Unsynchronized acceleration and angular velocity sampling data [1]; (ii) Error accumulation through integration process [3]; (iii) Additional bias due to centrifugal force arising from the distance between the sensor location and the center of mass of the projectile.

Experimental set-up and calibration
As an extension of a recent investigation on granular drag [7], an experimental campaign has been carried out at Bremen Drop Tower in 2019 [11]. As shown in Fig. 1(b), we embed a circuit board (Mbientlabs, Metawear C) into a 3D printed spherical projectile of diameter 32 mm. As shown in Fig. 1(c), the experimental set-up contains a projectile launcher and a cylindrical container holding the granular sample. It is fixed to an experimental capsule with on-board computer and high speed cameras to monitor projectile motion before it submerges in the granular sample composed of expanded polypropylene particles (Neopolen P9230). To gain ∼ 4.8 s of microgravity time, the experimental capsule falls freely in the vacuumed drop tower. After microgravity condition is established, the projectile is launched mechanically via gate opening. Sensor data logging starts before the capsule launch and lasts till the capsule settles at the bottom of the drop tower, subsequently the data are transmitted to the on-board computer via on-chip wireless data transmission module. Our goal is to reconstruct the trajectory of the projectile motion from sensor data, particularly when it travels inside the granu- lar sample for further investigation on granular drag under microgravity.
As highlighted in Fig. 1(b), the IMU sensor (Bosch BMX160) deviates from the geometric center o of the projectile, the acceleration obtained in sensor reference frame a s (t) includes additional acceleration a c = ω s × ω s × r so + ω s ×r so , where ω s and r so correspond to the angular velocity of the sensor and the position vector of the sensor with respect to o, respectively. In order to obtain the bias acceleration due to a c , an estimation of r so is needed. For this purpose, a freely rotating shaft is mounted horizontally to the stand with two ball bearings to reduce frictional force, as shown in Fig. 2(a). By gently blowing air to the device, homogeneous rotation about the shaft is established. Using a s and ω s collected when the projectile rotates about its major axes, we obtain r so = (1.2mmx+7.4mmŷ+4.5mmẑ). The relative uncertainty for the calibration is within 2.0 %, which arises from the slight misalignment between the sensor and rotating axis and can be captured with sensor data collected. Subsequently, we run a test with the projectile mounted in an arbitrary direction and use r so to remove bias acceleration a c . As shown in Fig. 2(b), removing the bias acceleration due to centrifugal and Euler acceleration can effectively correct the unrealistic deviation of the raw acceleration measured. Note that the angular velocity increases systematically, leading to an enhanced drift in the raw acceleration data (blue curve) arising primarily from the centrifugal acceleration. One the other hand, the Euler acceleration contributes to the initial offset of the raw data with respect to the corrected data.
Since the magnitude of r so is in the millimeter regime, the influence of centrifugal acceleration is typically orders of magnitude smaller than gravitational acceleration if the projectile is not rotating extremely fast. Quantitatively, the centrifugal acceleration due to r so reaches 10% of gravitational acceleration if ω s ≥ 11 Hz. In the drop tower experimental campaign, we sometimes observe ω s ≥ 35 Hz. Therefore, it is recommended to correct the bias due to centrifugal acceleration.

Trajectory reconstruction
In this section, we describe how to reconstruct projectile trajectory from sensor data collected. Although both accelerometer and gyroscope are packed in the same microchip [see Fig. 1(b)], they may not be synchronized with each other by default (even though being set to the same sampling rate) [1]. To deal with this issue, we determine the actual sampling rate from the time stamps recorded along with data. Subsequently, we 'synchronize' different time lines via determining when the experimental capsule drops; that is, the time when the whole system starts to move. Figure 3(a) shows a typical acceleration change at the onset of capsule launch. It is obvious that the magnitude of acceleration drops dramatically at a certain time. As indicated by the three lines, a zeroth order followed by a first order fit is applied to the data to find the threshold (highlighted by the gray line), which corresponds to the intersection point that minimizes the standard error. The offset from the zeroth order fit (i.e., average of data), which includes system bias and gravitational acceleration, is removed from the acceleration data to highlight the  abrupt change. Using the same approach, we determine the threshold time for angular velocity change as well as granular particle motion due to capsule launch and use it as time 0.
In addition to the sensor data, side-view images are taken [see Fig. 1(a)] as an additional approach to determine the projectile position. As the capsule launches, the granular layer starts to expand slowly, leading to the detachment of some particles from the surface. By means of structure similarity index measure (SSIM) [12], this change can be quantitatively characterized [see Fig. 3(b)]. Using the mean intensities and standard deviations of consecutive images as well as the co-variance in between, SSIM measures the luminance, contrast and structure change quantitatively with unique maximum (i.e., it equals 1 only when the two images are exactly the same). Consequently, the time scale of the image sequence is synchronized with the sensor data for comparison purposes. From both sensor and image data, we learn that the system reaches a steady state before the projectile launches.
At 0.55 s, the projectile is launched from its initial position and starts to move toward the granular sample under microgravity. Due to the limited space in the projectile launcher, the projectile might collide multiple times with the container before it moves out of the gate. During this time (0.55 ∼ 0.70 s), we observe multiple spikes in the ac-  celeration data and subsequent angular velocity changes. As soon as the projectile becomes visible to the camera (i.e., moving out of the gate), no external forces are acting on it and consequently the acceleration stays close to 0. Note that it still fluctuates slightly due to occasional collisions with the surrounding granular particles.
In order to obtain the projectile trajectory, we first define a coordination system fixed to the capsule. Our task is to convert acceleration measured in the sensor coordination system to the capsule system based on the orientation of sensor obtained from its angular velocity. Here we defineẑ as opposite to the direction of gravitational acceleration. As the camera is mounted upright, the upward direction in the image also corresponds toẑ. Due to circular symmetry of the container, the other axis can be chosen arbitrarily. In practice, we choosex andŷ axes based on the initial acceleration vector measured by the sensor. Starting from the time when the projectile launches, we use quaternions to transform the projectile acceleration a to the fixed coordination system a o obtained at the same time [13,14]: where a o , a ∈ H are both quaternion with real component = 0 and the conversion factor q is defined as Thus, Eq. 1 represents rotating acceleration vector by angle φ about an axis u. The angle of rotation changes as a function of time, therefore accumulation of rotation over time is needed. Here we combine integration in both forward and backward direction to reduce noise accumulation [3]: where n, δt and ω denotes the index of the current time step, the duration of each time step, and angular velocity, respectively. We take first order approximation and use the mean angular velocity ω for the estimation of rotation step. Note that it is important to have synchronized time scale between acceleration and angular velocity because rotation of coordination system is necessary in each individual time step in order to obtain projectile acceleration in the capsule system. For more details on the quaternion approach, interested readers may refer to [1,3]. Based on the above analysis, we obtain acceleration as a function of time in the fixed capsule system a o (t). Integration over time gives rise to corresponding velocity v(t) and position s(t). The latter in the z direction is shown in Fig. 4(a). As highlighted in the inset, projectile motion obtained from the sensor in the free flying period matches that from the image analysis well. In image processing, standard Hough transformation algorithm is used for the detection of projectile position [15].
The cross-checking with other particle tracking approaches (i.e., image analysis) is necessary because integration process leads to error accumulation. This is one of the reasons why sensor fusion is widely implemented in widespread applications of IMUs. In short, the cross-checking mechanism validates the projectile trajectory for further analysis of granular drag. Finally, as shown in Fig. 4(b), the 3D trajectory of the projectile is reconstructed based on position data obtained in all directions. It shows that the projectile bounces a few times against the launcher boundary before leaving the gate, corresponding to the peaks between 0.55 s and 0.70 s in the acceleration in Fig. 3(d). It also suggests that the projectile was not launched strictly in the −z direction, as there is a small horizontal velocity component. We leave further analysis on the dynamics of the projectile upon granular drag to follow-up investigations as the focus here is on particle tracking with an IMU sensor.

Conclusions
To summarize, we show that readily available IMU sensors can be a good candidate for assembling 'smart' tracers for remote sensing purposes. Embedding this type of tracers in granular materials can help decipher granular dynamics non-invasively. Using a projectile impacting on a granular medium as an example, we describe how to extract 3D object trajectory by both accelerometer and gyroscope data and particularly how to deal with possible sources of uncertainty. More specifically, it is necessary to synchronize acceleration and gyroscope data, to calibrate for additional centrifugal and Euler acceleration. To reduce error accumulation during the step-by-step coordi-nate transformation and integration process, it is helpful to consider integration in both directions while obtaining the rotation quaternion. Last but not least, it is always helpful to have another particle tracking algorithm for crosschecking. How to fuse additional sensing technique and use, for instance, error state Kalman filter [3] to further enhance the accuracy of particle tracking is a topic for further investigation.