Enhancement of breathing simulation using individual lobe segmentation

To tackle thorax movement from CT images, we have developed a platform to simulate a customized breathing cycle, where the pulmonary movement has been considered only at the rough border of the whole lung by artificial neural networks (ANN). The goal of this work is to include additional information of the lung lobe. Thus, more ANN will be used and future simulation will be able to take into consideration the impact of tumor on lobe movement. We present a new automatic segmentation algorithm that enables the extraction of lobar contour data using sliding mask and direction estimation. These improvements enhance the overall system performance in which higher precision and more accurate treatments can be expected.


Introduction
Radiation therapy aims at targeting tumor and preserving the surrounding healthy tissues.Nowadays, new treatment techniques improve the accuracy of beam targeting, but the problem of organ movements is still an issue.In most cases, especially for tumors located in the lungs, it is difficult to control breathing and lung movement, during the treatment session.Therefore, it is required to take into account the organ movement during therapy planning, which leads to an increasing amount of CT and 4DCT.From a radiation protection point of view, the patients receive an increasing amount of "small" doses from these imaging techniques (e.g. a 4DCT is 5 to 6 times more irradiating than 3DCT).One active research perspective concerns the simulation of breathing movement.The IRMA team has previously developed a platform for customized simulation using neural networks [1].In order to improve the simulation, it is worth performing an accurate identification and consider the analysis of lung structure on a lobe-by-lobe basis.This additional lobe segmentation information will be useful not only for the early detection of pathologies but also for better precision, higher accuracy of treatment, and later to deal with the movement artifacts due to the tumor.
Pulmonary lobe segmentation is a challenging topic that attracts interest in the medical imaging community.Due to the difficulties to control the different CT scanners and acquisition protocols, the obtained images usually vary in resolution, qualities, anatomical reference, etc.The missing of some parts of the organs, the artifacts, and incomplete fissures are among the issues found especially in unhealthy patients.Many automatic and semi-automatic techniques have been published in the literature.Rikxoort et al. [2] and Zhang et al. [3] used method based on an anatomic pulmonary atlas.Kuhnigk et al. [4] and Ukil et al. [5] proposed a semi-automatic watershed transformation based on intensity and distance to vessels and airways.Ross et al. [6] combined the atlas with hessian-based particle sampling and thin plate splines.Lassen et al. [7] and Doel et al. [8] also applied the hessian analysis with the watershed transform.This paper proposes an alternative scheme for lobar segmentation using sliding mask with direction estimation.The goal is to provide relatively fast and closed representation of lung lobes in each breathing phases for the training of ANN.

Methods
Neural networks work similarly to biological neurons, in the sense that they need training before being able to interpolate data.To learn the lobe movement, learning sets have to be constituted.They consist of the movement vectors at various breathing phases for the lobe contour points.To build these learning sets, extracting the lobe contours automatically or with minimum interactivity is desirable.In our work, image processing plays an important role and has been utilized for the CT scans.The data flowchart illustrated in figure 1 describes the main operations of the algorithm.

Lung contour extraction
The raw images obtained from the CT scans generally contain irrelevant components, e.g.other organ tissues, backbone, ribs, diaphragm, or some parts of the machine.Since we consider only the lung organ, the preliminary task is therefore to separate out the complete lung and maintain the image details within it (i.e.image intensity).Digital Imaging and COmmunications in Medicine (DICOM) images are read all at once for each breathing phase of the patient.The image sequence with approximately highest volume of the lung capacity is then called to define cropping coordinates of the target area.Consequently, morphological operations (i.e.filling holes) and image subtraction are performed to ensure that the details inside the lungs are recovered.In addition, grayscale thresholding and image normalization are applied to provide better contrast and brightness for visualization.The resulting images from these procedures are shown below.These global lungs are then divided into 2 sections to ease the later manipulation.We can split the left and the right lungs by a separation line.This line is drawn on the overlapping frame, where the image slices are all plotted (see figure 3) ensuring that the line is properly defined for the whole sequence of the breathing phase.Lungs are divided into 2 sides with 5 pulmonary lobes.All these lobes are separated by thin tissues called lobar fissures.The left lung consists of an upper and lower lobe separated by oblique fissure, while the right lung consists of an upper, middle, and lower lobe separated by horizontal and oblique fissures.Beside the lung contour, we also have to define the lung fissures, which are the separation of the lung lobes.The key idea of fissure detection is to remove the redundant elements within the lungs: bronchus and blood vessels.

Bronchus elimination
The bronchus is a passage of airway in the respiratory tract.It connects to the lung directly.Bronchi are found in almost every image slices.In order to eliminate these bronchi branches, the solution is to employ 3D region growing [9][10].The segmentation process consists in merging all neighborhood voxels that satisfy a homogeneity criterion guided by an initial seed.The seed voxel is specified interactively with a mouse, pinned over the intensity threshold selection of the bronchus area.In our work, it is located on the image frame where the bronchus first appeared.The algorithm provides rough bronchus segmentation for all the lung slices.Here are examples of results.

Blood vessel removal
The blood vessels are another main component used to carry the blood between heart and pulmonary system.The pulmonary arteries transport deoxygenated blood from the heart to the lungs, while the pulmonary veins supply oxygenated blood from lungs back to the heart.The purpose of this step is to suppress the vessels in the image, so that the fissure can be better identified.Moreover, these blood vessels are not only found in the entire lungs but also have different intensity variations, making it tedious to perform region growing as in the case of bronchus.We apply dilation operation to expand the blood vessels in each frames.The result in figure 5 shows the lung with the black patches covering the vessels area.At this step, the fissures can be distinguished easily (see figure 6).

Fissure localization
The fissures are localized by two interactive points on both sides of the lungs.These 2 initial points marked in red (see figure 7) are then expanded using sliding mask.The mask is defined based on the trend of fissure orientation and displacement, which allows us to predict the next starting points of the following frames from the knowledge of the preceding frames.The example of sliding mask is shown in figure 8.For each frame, the itinerary begins at the starting point and prolongs forward and backward by direction estimation as illustrated in figure 9, until reaching the limits at the edge of the lung.Therefore, there are 4 border lines to be considered: left-end of left lung, right-end of left lung, leftend of right lung, and right-end of right lung.
Figure 10 shows the result from fissure and lung contours extraction.The contour of the right lung is drawn in blue, the green and magenta lines indicate the forward and backward fissures.Similarly, the edge of the left lung is plotted in magenta, and partial fissures connecting in yellow and blue.

Lung lobes determination
We combine fissures and lung contour information to obtain all data points of the lung lobes.However, these individual lung lobe contours are not directly usable.To be precise, their alignment is not in an order that satisfies the interpolation necessity.Therefore, it is required to apply edge detection before partitioning the lung into several lobes.We then use the rolling matrix method with priority indexes in clockwise and anti-clockwise direction.The priority index represents the searching order of neighborhood pixels.Examples of 3x3 matrix with priority indexes are represented in figure 11.Eventually, we obtain a smooth contour and well-ordered points as shown in figure 12.The computational time of this algorithm is approximately 2 minutes (excluding the pre-image processing and interpolation step).Amount of points representing lobes is 20 per frame and number of final points corresponding to the lobes for one phase is 15.These results after spline interpolation will be used in the learning algorithm.The database is accumulated from 8 to 10 breathing phases for 10 patients.

Conclusion
We have introduced a new approach for individual lung lobes segmentation.This work is mainly of interest to both radiation therapy and radiation protection.The customized simulation of lung movement can be performed independently on each lung lobe.The clue combination of fissures and primary lung contour provides adequate numbers of points, as input data to build the learning sets for ANN training.ANN will allow us to simulate lobe movements with and without tumor.Therefore, the breathing movement can also be simulated for sick patients.
In the next stage, we would like to enhance the performance of the software in both image processing and learning network aspects.The robustness of the current algorithm can be improved to suit the variation of the CT input images regardless the scanners and the patients.These will also solve human intervention issue.Last but not least, we would like to extend this work to other organs in order to simulate a customized breathing movement for the whole thorax.

Figure 1 .
Figure 1.Flowchart of the overall algorithm.

Figure 7 .
Figure 7. Example of 2 initial points of fissure.

Figure 8 .
Figure 8. Red box indicates the considered pixel, black thick line indicates the area of sliding mask for searching the next fissure point.If no candidate point is detected, the search area is extended.

Figure 9 .
Figure 9. Direction estimation scheme.Red box indicates the considered pixel.Left and right neighbor fissure points are recursively searched within the mask.

Figure 13 .Figure 14 .
Figure 13.Lung lobe representation: (a) Upper right lobe in green, (b) lower right lobe in red, (c) upper left lobe in blue, and (d) lower left lobe in black.