SET OF ALGORITHMS AND TECHNIQUES FOR ACCURATE 3D, SINGLE BEAM – SINGLE POINTING, LIDAR MEASUREMENTS FOR SLANT RANGE VISIBILITY, PLANETARY BOUNDARY LAYER HEIGHT AND WIND SPEED RETRIEVAL, ATMOSPHERIC LAYERS SPATIAL DISTRIBUTION AND CATEGORIZATION IN REAL TIME

In this work, a full set of recently developed algorithms and techniques is presented, for a single beam-single pointing lidar to be able to perform operational and independent accurate 3 Dimensional (3D) measurements, for slant range visibility, wind speed retrieval, atmospheric layers spatial distribution and categorization, as well as Planetary Boundary Layer Height (PBLH) retrieval, in real or Near Real Time (NRT).The idea behind this development was for any single lidar to be able to perform a set of accurately measured products, either mobile or stationary, with or without network connectivity with other sensors for data-information exchange. The products were determined by the needs of lidar remote scientific and commercial community, in order to be even more attractive and valuable to atmospheric scientists, meteorologists, aviation and shipping safety operators, as well as to the Space lidar community.


INTRODUCTION
Following the work of [1,2,3] and Eqs 1-6 in these references, which also apply for this work, backscattering lidar signal received power (P(λ,R)), Range-squared Corrected lidar Signal (RCS), P΄(λ,R) which is the received power after atmospheric and electronic noise BackGround (BG) correction and processing of lidar atmospheric coefficients like, aerosol extinction coefficient α aer (λ,R) and aerosol backscattering coefficient β aer (λ,R), were used in order to produce a full set of algorithms and techniques for 3D lidar measurements. The original development of the algorithms used the above mentioned variables and Eqs 1-6, produced with Klett technique [4,5], but they are also applicable using other retrieval techniques, like the Raman one [6].
The vertical, near vertical or even slant measurements that can achieve molecular values of β aer (λ,R) (and/or α aer (λ,R)), at a molecular height above the PBLH and at any wavelength, taken from [1,2,3,7], ([8] p. 57, 58), which are recorded using the backscattering signal, are able to provide a clear view of the aerosol load located along the Line Of Sight (LOS) of the lidar beam. This aerosol load recorded as elastic backscattering signal (Eqs 1-6 already mentioned and Eqs 7-9 below), was used by these algorithms in order to have successive outcomes of processing.
Moreover, the aerosol optical depth (τ) is given below: where, R΄ is the distance at which α aer (λ,R) has been measured with a range resolution of 1 bin (i.e. 7.5 m), τ total is the sum of τ Cn , which is the individual optical depth (n=1,2,3,…) of a number (n) of atmospheric layers, decided to be used by the user in his measurement.
The relationship between the two coefficients β aer (λ,R) and α aer (λ,R) also used is some cases, is the so-called lidar ratio, C(λ,R): Flight and ground safety at airports, but also shipping safety has to do, in many ways, with visibility and meteorological conditions in the area. A major flight safety issue is to obtain visibility measurements especially from the pilot's point of view and to be able to have accurate measurements of some meteorological parameters from a respected distance of few km, not only above the airport, but also in Slant Range (SR). Also, accurate PBLH retrieval together with an estimation of the atmospheric layering and distribution, through the detection and tracking of the actual structure of β aer (λ,R) and/or α aer (λ,R), RCS or P΄(λ,R) in 3D measurements, is also a main issue for today's scientific community, as no concrete technique could offer so far, solid outcomes in Real Time (RT), producing in this way a maximum contribution in present weather conditions estimation from a distance and 'help' in weather forecasting. The main difference in these algorithms is that they create a new kind of 'mathematical tools' (notional conditions) for processing, based on the backscattering lidar signal, the concordance of which is based upon the approximation of PBLH retrieval, actual layering and distribution according to the Bayes theorem ([9] pp. 1-3).
For wind measurements, the most known method is based on the Doppler effect which has been used for the retrieval of the PBLH through wind measurements [10]. A different technique, but the most correlated to the one proposed here, based on single beam-single pointing measurements, is the one mentioned in [7]; it uses the wind retrieval correlation method for backscatter lidar scanning schemes consisting of a few profiles along the LOS, which is mathematically formulated in a matrix solution form under different atmospheric conditions. This method assumes the frozen atmosphere model [7] and works with temporal cross-correlation functions that do not need to be maximized. It also applies for the case of slant scans based on the multiple-angle azimuth technique (horizontal wind retrievals) [7]. The main differences with our proposed wind algorithms and technique, is that even if we simultaneously use single beam measurements at each time, we can produce wind speed in 2 dimensions (2D), by staying in one single slant pointing angle and calculate 3D wind speed profiles by using a method similar to the 2-angle azimuth scan [7].

METHODOLOGY
The methodology used during the development of this set of algorithms was first to create some new denoising techniques like NOSUB-S/R [1] in order to reduce any noise from our backscattering signal and proceed to more accurate estimations of our variables, to be inputted for processing. In NOSUB-S/R some new filters-variables created in order to estimate the conditions where noise is taken over the actual signal returns and produces notional backscattering signal values. A subtraction occurs in this case of the actual noise, without 'harming' the valuable lidar backscattering signal, extending in this way the distance where valuable signal can contribute to our measurements and perform to the rest of the remaining valid signal, a subtraction also, of any added noise, in order to have handed for processing, actual and valid backscattering values of our signal.
Then, the way to Slant Range (SR) measurements became easier by creating LADDER or 3D Stepping Technique [1] (Fig. 1) where, we can retrieve β aer (λ,R) and/or α aer (λ,R), RCS or P΄(λ,R) by performing measurements like 'stair steps' from vertical to horizontal direction, every 1 o vertical angle, like "walking down a ladder". In this way, we take as R F at Eqs 1-6, the distance at which we abstract 1-2 slant -height bins (1 bin is equal to the spatial resolution ΔR) for every new measurement and we retain the last value of β aer (λ,R) and/or α aer (λ,R), RCS or P΄(λ,R), at which R F was previously taken, as the new calibrating values for β aer (λ,R) and/or α aer (λ,R), RCS or P΄(λ,R). The results of such a technique when used properly were most effective (Table 1) in comparison with other techniques.  [1] applied in 3D lidar data pointing vertically, up to 3000m. Where Blue is "Light Fog", Purple is "Thin Fog", Light Blue is "Light Haze" and Green is "Haze". The black dashed line denotes the PBLH retrieval with METCON PBLH, between layers change as supposed to happen. (Lower) Application of SIBESMEA with a case study of wind direction in 2D. Red and blue fill-ins are for the upwards and downwards wind direction and green and yellow fill-ins are for the east-west wind directions. The rest of the air mass missing between continuous measurements applies for 3 rd dimension wind movement, by presuming high repetition frequency of pulsed signals. The speed of the 3 rd dimension movement of the air mass could be calculated through the rate of air mass loss in that (+ or -) direction.
By solving SR measurements it was achievable to acquire SR visibility also from pilot's point of view, with NAVIS-T/P algorithm [1] and the use of Eqs 7-9. Towards this idea of SR visibility and PBLH retrieval, initially, METCON or WEATHER PHENOMENA algorithms [1] was created in order to produce equivalent layering and distribution in a 3D view from lidar's location, by means of atmospheric layering categorization mentioned in ( [8] pp. 57, 58, 144) and ( [11] pp. 178, 179) (Fig. 2, Middle).
As a next step, after being able to actually produce atmospheric layering and tracking in RT, the achievement was to create an algorithm being able to retrieve the PBLH in an area surrounding lidar's location. By using the principle of searching for a persistent appearance ( [9] pp. 1-3) of a certain atmospheric layer inside the common PBLH area according to weather forecasting and time of the year at lidar's location, like fog or other similar atmospheric layer, in terms of close values calculation of backscattering variables measured with the ones produced by fog [1], ([8] pp. 57, 58, 144), ( [11] pp. 178, 179) the algorithm acquires usually the upper limit of that atmospheric layer, as retrieved PBLH. In this effort, certain thresholds of first and second derivatives of a new kind of 'mathematical tools' (notional conditions) for processing used for VASPAT or MUSTI algorithms, VERDE or DECIS [2] and METCON PBLH [3] (Fig. 2, Middle) and succeeded a great efficiency in more than 94% of correct PBLH retrieval in more than 30 cases studied and compared with nearby radiosonde measurements, available by the Hellenic National Meteorological Service (HNMS) at the same date and time [3] (Fig. 3). To this end, a new technique was born called UNIFORM technique [3] in order for 3D PBLH retrieval becomes possible (Fig. 2, Upper).   [3] presentation of four sequential lidar data recording files every 10". The tracking of a certain atmospheric layer in Yellow color ("Light Haze" depicted with dashed line in its center) begins at ∿1500m height and travelling up to 1750m in 30", presenting a layer tracking (wind) speed of 8.3m/s at that time and then it was lost as a specific layer. Air masses movement through atmospheric layer detection and the replacement of the "free" space in between by other atmospheric layer, is depicting the air masses movement and changes around PBLH and other atmospheric mixtures with time. The blue vertical lines represent distinguishes consecutive lidar shoots.
Finally, the goal was that once 3D atmospheric layering was possible and accurate, and through that, PBLH retrieval in a wider area in RT, tracking of these layers should become possible also for a 3D valid estimation of most of weather conditions presence, with a single lidar without the actual need of the expensive and heavy Doppler equipment or experienced user. So, SIBESMEA (Fig. 2, Lower) and WIND METCON (Fig. 4) algorithms developed [3] in order to overcome this obstacle with totally successful results, making it also possible for tracking atmospheric layers separately as well as tracking and visualizing the way they are transformed to other kind of layers through their movement in the atmosphere, even in PBLH level, were valuable case studies could be performed in the future (Fig. 2, Lower).

RESULTS
All these algorithms presented a unique performance after being developed and tested under 'real' lidar measurements, using EOLE lidar data from NTUA together with radiosonde data from the Hellenic National Meteorological service (HNMS).