VALIDATION OF OPTIMAL ESTIMATION METHOD RETRIEVALS OF MIDDLE ATMOSPHERIC TEMPERATURE

OEM (Optimal Estimation Method) retrievals of temperature from lidar measurements are robust and practical (Sica and Haefele, 2015). They offer significant improvements over traditional methods. We will show a climatology of +360 nights of measurements from the Purple Crow Lidar and the improvements offered using an OEM, including the quantitative determination of the top altitude of the retrieval and the evaluation of the various systematic and random uncertainties due to measurement noise.


Introduction
Rayleigh scatter measurements from the University of Western Ontario Purple Crow Lidar (PCL) have been used to develop temperature climatologies for the mesosphere and lower thermosphere.Temperature retrievals from Rayleigh-scattering lidar measurements have been performed using the traditional method of Hauchecorne and Chanin [1] since 1980 and the Optimal Estimation Method (OEM) of Sica and Haefele [2] since 2015.The traditional method has some limitations including the removal of the top 10-15 km from the uppermost range of observation, inducing extra uncertainty when merging temperature profiles from different channels in the common range and providing only random uncertainty.An OEM technique was used to overcome these deficiencies and address a solution for each of them.This research investigates the validity of the OEM in comparison with the traditional method.

Temperature retrieval methods
The conventional method integrates temperatures downward, and requires the assumption of a "seed" pressure at the highest altitude, taken from a model.The starting point is at the highest altitude of the PCL Rayleigh channel, and the corresponding pressure is obtained from the COSPAR International Reference Atmosphere (CIRA) model.Geophysical variation in the lower thermosphere is sufficiently large to cause temperature retrievals to be unreliable for the top one or two scale heights.Uncertainties due to this pressure assumption make it prudent, without ancillary measurements, to remove the top two scale heights of temperatures from each profile.Unlike the other methods, which include estimates of random uncertainty, the OEM produces a complete uncertainty budget for all parameters in the retrieval procedure.The OEM consists of two elements: a forward model that describes the lidar measurements in the atmosphere, and the a priori profiles with corresponding error covariance matrices that constrains solutions to physically meaningful states.The lidar equation and the hydrostatic equilibrium assumption are used as a forward model [2] and the CIRA-86 model as an a priori.The optimum estimate of the state vector, x, is found by minimizing equation (1), which corresponds to choosing the most likely state from the solution probability density function [3], otherwise known as the cost function.Here x is the most probable state, y is the measurement vector, S ε is the covariance of the system's state, x a is the a priori vector, and S a is the a priori covariance.
The averaging kernel (A) represents the sensitivity of the retrieved state ( x) to the true state (x).Sum along the rows of averaging kernel matrix (Au, u is a vector with unit elements) equal the areas of the averaging kernel matrix at each level.This quantity is shown by Au from now on.Au represents the amount of information determined from the lidar measurements and how much is due to the a priori and is almost unity at accurate retrieval levels [3].

Methodology
Measurements from two PCL Rayleigh channels including Low Rayleigh (24-45 km) and High Rayleigh (35-110 km) over 360 nights of measurements from 1999 to 2013 were used to generate the temperature climatologies using the OEM and the traditional method.The OEM was used in order to calculate nightly temperature retrievals, and the temperature climatology procedure is based on the processes presented in Argall and Sica [4].Three atmospheric models including the US Standard Atmosphere, CIRA and MSIS are considered as an a priori in the OEM.An a priori should not have a trend during time, in order to avoid the a priori inducing a trend in the temperature climatology.The MSIS-00 model's temperature shows a trend due to its complexity and therefore it was not used.The US Standard Atmosphere model provides just yearly temperature profiles which can be used as an a priori; however, CIRA-86 provides monthly temperature profiles with smaller temperature variances than those of the US Standard Atmosphere model.Therefore, CIRA-86 was chosen to provide the a priori covariance matrix as well as seed pressure at highest altitude for the traditional method.In the case of the traditional method, in order to cut off the temper-ature to achieve an accurate result, two scale heights (top 15 km) from the highest possible altitude was considered.In the OEM approach, a value of 0.99 for the Au was used as a reference for cutoff height instead removing the top 15 km from the temperature retrievals.Below the 0.99 Au cutoff, the a priori has less than one percent contribution to the temperature retrieval and below this value the results are valid.The retrieved temperatures from the PCL, Low and High Rayleigh channels in the overlap range are not identical using the traditional method.The temperatures should theoretically be identical in the overlapping regions; however, in reality they are frequently different.In the past, in order to create a single temperature profile from different lidar channel measurements, their profiles had to be merged.Therefore, it was necessary to merge two temperature profiles in order to have a temperature profile for the entire PCL profile range.This issue was investigated in Jalali et.al. [5] by using different merging functions in the traditional method.The process involved combining various random uncertainties, choosing overlap ranges, and calculating uncertainties when gluing raw photocounts profiles, all of which were intensive and challenging processes.The OEM does not require merging and provides an easy way to incorporate data from multiple sources, such as combining lidar measurements from different channels.
Figures 1 and 2 show two sample PCL temperature profiles using the traditional and the OEM to retrieve temperature from PCL photocounts.In both figures, the altitude of the 15 km cutoff height from the top of profile in the case of the traditional method is close to the 0.99 value in the Au for the OEM.Approximately 91 km and 87 km are the 15 km cutoff height in Figures 1 and 2 using the traditional method, respectively.However, 91 km is the cutoff height for the OEM corresponding the value 0.99 for the Au.It is noteworthy that there is a large difference in the 15 km cutoff height for the traditional method and the Au cutoff height of 0.9 for the OEM case.This difference is about 8.8 km in Figure 1 and 13 km in Figure 2. It can be concluded that the OEM approach allows the use of higher altitudes with known uncertainty in comparison with the removal of the top 2 scale heights using the traditional method.This ability to achieve temperature retrievals at greater heights is one of the main advantages of using the OEM approach.Also, it is necessary to mention that in both figures (1 and 2), the OEM profiles begin at 30 km, while the traditional method's results start from 40 km.This is because the OEM uses both High and Low Rayleigh channel measurements to retrieve a temperature profile, unlike the traditional method which provides two different profiles for each channel and presents just the High Rayleigh channel.Figure 3 shows the difference between the composite years PCL temperature climatology using the OEM and the traditional method between 40 and 90 km.The temperature differences show that there is a good agreement between the two temperature retrieval methods between 40 to 85 km and there is almost no difference between them.

Conclusions
Sica and Haefele [2] used a new approach for retreving atmospheric temperature from lidar measurements as an improvement to the traditional method.The temperature climatologies were calculated for the PCL Rayleigh measurements in order to investigate and evaluate this new method.The OEM shows a significant improvement over the traditional method.The OEM provides a mathematical method to determine a cutoff height for temperature retrieval below which the entire temperature is valid.Another advantage of the OEM is the ability  to provide one single profile where using measurements from two channels with merging of random uncertainty.The OEM provides a complete uncertainty budget on a per profile basis for all forward model parameters.The 15 km cutoff height from the traditional method corresponds very well with the 0.97 Au cutoff.Below these lines the temperature difference is negligible and the two methods are consistent.However, the 0.99 Au cutoff (no a priori information is in the retrieval) is on average below the 15 km cutoff height.Below the 0.99 cutoff there is almost no difference between the two methods' results except for a few spots close to the 15 km cutoff height.In this paper, only the statistical uncertainty is considered for comparison of the two methods.There are only a few spots with temperature differences larger than 5 K, the causes of which could be due to the systematic uncertainties in different parameters.The advantage of the OEM is shown with the 0.9 Au contour line where the a priori contribution is only 10 percent.It can be seen that the altitude difference between the 0.9 cutoff and the 15 km cutoff height is roughly 10 km which is an improvement over the traditional method.The uncertainty due to seed pressure is included in the OEM systematic uncertainty budget [2].Seed pressure uncertainty values for Figures 1 and 2 at the 0.99 and 0.9 Au cutoffs are 0.15 K and 2 K, respectevly.In summary, the 15 km cutoff height in the traditional method corresponds approximately Au = 0.97 cutoff for the OEM and both methods are consistent up to this cutoff height.However, the OEM determines the percentage of the confidence in the results mathematically and computationally unlike the traditional method, which does not provide a methodology for determining a cutoff height and corresponding uncertainty.

Figure 1 :
Figure 1: PCL temperature profile for 10 May 1999, integrated over 7.4 hours and with coadded altitude bins to 255 m using the traditional method (left) and the OEM (right).The shaded blue and red profiles are temperature random uncertainty corresponding the traditional method and the OEM respectively.The horizontal dashed lines are the height above which the temperature profile is less than 1%(0.99)and 10 %(0.9) dependent in the choice of an a priori (right panel) and the15 km cutoff height (left panel).

Figure 3 :
Figure 3: Temperature difference between the OEM and the traditional method.Black line is 15 km cutoff height for the traditional method and blue lines are averaging kernel contours for the OEM.Using OEM the temperature climatology can be extended to almost 100 km with less than 10 % contribution from the a priori.