Using the Monte-Carlo method to analyze experimental data and produce uncertainties and covariances.

. The production of useful and high-quality nuclear data requires measurements with high precision and extensive information on uncertainties and possible correlations. Analytical treatment of uncertainty propagation can become very tedious when dealing with a high number of parameters. Even worse, the production of a covariance matrix, usually needed in the evaluation process, will require lenghty and error-prone formulas. To work around these issues, we propose using random sampling techniques in the data analysis to obtain final values, uncertainties and covariances and for analyzing the sensitivity of the results to key parameters. We demonstrate this by one full analysis, one partial analysis and an analysis of the sensitivity to branching ratios in the case of (n,n’ γ ) cross section measurements.


Context
We present this paper in the context of the improvement of the evaluated nuclear data for application. This is done in order to perform better numerical simulations to optimize and predict performances, reactor control parameters and radiation safety related quantities. These databases still present large uncertainties, preventing calculations from reaching the required precision. Their improvement requires new measurements and better theoretical descriptions of involved reactions. Our primary work is focused on the measurement of (n, xnγ) cross-section [1][2][3] However, the general idea presented in this paper can be applied in many other contexts.

Motivation
When performing data analysis of experiments, many external parameters (detector efficiencies, distance of flight, ...) are involved, in order to process the raw data. Furthermore, all the steps of the data treatment (event selection, calibration, ...) may introduce uncertainties and correlations.
The usual method for combining and computing uncertainties is to use analytical developments based on the perturbation theory (e.g.
. This method * e-mail: ghenning@iphc.cnrs.fr * * Currently, Univ. of Helsinki works well for simple cases, but with multiple parameters and sources of uncertainty, deriving the final total combined uncertainty can be long and complex. Furthermore, it strictly applies only to small deviations from the central values, which is not always the case. Implementing the formula into the analysis code becomes a tedious process where mistakes can appear, and the final uncertainty value will be wrong.
Finally, this method makes it difficult to calculate covariances, and the inclusion of some unusual form of uncertainty (asymmetric, non-Gaussian) is not directly possible.

The Monte Carlo method
To workaround all the issues presented above, we offer one possible solution: using random sampling (i.e. Monte Carlo) methods to obtain final values with their uncertainties and covariances.
The general principle is the following : Each parameter is randomly sampled according to its probability density function, all the parameters are used to compute one realization of the value of interest. This is done many times, as each computed value is stored in a stack. When all the iterations have been performed, the average value, standard deviation and (when applicable) covariance matrix are calculated from the stacked values. We note that this method allows us to turn off and on specific sources of uncertainty, by including or not the related parameters in the calculations. This makes the study of sensitivity easy, and a good way to check consistency.
The treatment of all sources of uncertainty is the same, without differentiation between the ones that come from systematic or statistical sources, as long as their probability distribution is well-defined. In fact, in this treatment, one should prefer the Type A vs. Type B split (rather than statistical, systematic) recommended by the Guide to the expression of uncertainty in measurement [4]. Particular attention should be given to choosing the probability density functions of the parameters variables. This being done, the reprocessing of data at each iteration with new parameter realizations will ensure that uncertainty from the analyzed data will be taken into account. The ability to examine the final distribution of results (and not just the central value and standard deviation)is a great benefit to check the consistency of all results given the chosen inputs.
Special attention needs to be applied to the convergence of the series. In order to produce accurate and stable final values, enough iterations are needed. The tricky part being that it is not evident a priori how many iterations is enough. One has to check the convergence of the result by inspection.
Thankfully, with modern computing infrastructures, large storage spaces and many computing units are easily available. This allows, within a reasonable time and use of computing resources, to perform a very large number of iterations.

Full Monte Carlo Analysis
The first example of applying such a technique we are presenting is the full Monte Carlo analysis. In this example, for each iteration, we will start the whole analysis pipeline (event selection, data projection, histogram fitting, ...) from scratch, with all external parameters (detector efficiency, target mass, ...) randomly sampled. The analysis is then conducted (automatically) to the end and produces one set of values (in our example, (n, n'γ) crosssections for several neutron energies) which is added to the stack. As the spectra are re-extracted and fitted at each iteration, the procedure uncertainty, as well as uncertainties coming from the data analysis, are automatically taken into account.
When the sufficient number of iterations for convergence has been achieved (in our case, 30 is enough to reach the limit uncertainty), we use the numpy package [5] for Python [6] to compute, from all the values, the central values, standard deviations and covariance matrices. Figure 1 shows an example : it is the 184 W(n, n'γ 111 keV ) cross-section obtained with a full Monte Carlo analysis [7]. Each gray line in the plot represents the result of one iteration, the black points, line and error bars are the final values.

Random sampling applied on intermediate results
In the cases of data that have already been analyzed (using a deterministic method), one can pick up the intermediate result files and, by applying random sampling methods, replay the last steps of the analysis many times. This is a great way to access covariance when the initial analysis did not. A similar method has been applied before in reference [8].
We applied this to 238 U(n, n'γ) data [9] and the Monte-Carlo method reproduces the central value from the analytical method with a very good agreement-figure 2. The obtained uncertainties are slightly different (but still well compatible with each other). In particular, the uncertainties obtained with the Monte Carlo method present more structure than those with the analytical method. This type of structure, which is related to the input data, is not accounted for by the analytical method.

Monte Carlo method for sensitivity analysis
In addition to analyzing data using the random sampling method, the Monte Carlo method can be used to study the sensitivity to parameters. By varying a parameter through random sampling, the variation in the final results provides a sensitivity coefficient from the mean values and standard deviations of the parameter and final result. We used this method to study the sensitivity of calculated (n, n'γ) cross-sections to transitions branching ratios. For this purpose, we performed, for each transition in the level scheme, one hundred calculations using TALYS-1.8 [10], where the intensity of the transition is varied around its reference value [11].
After all calculations are finished, the results are loaded into a Python [6] numpy array [5] and the module is used to calculate the relative standard deviation expected on a calculated cross-section per relative standard deviation (i.e. uncertainty) on a specific γ transition branching ratio in the level scheme.
The resulting sensitivity matrix shows that some transitions have a sensitivity of as high as 40 % to other transitions' branching ratios, as seen in figure 3. See another contribution in this conference for more details on the interpretation of the matrix [13].

Conclusion
We gave three examples of how random sampling can be used for data analysis or to study the sensitivity of calculations to parameters. It is a convenient solution to produce accurate uncertainty and covariance matrix without lengthy and error-prone uncertainty propagation. It is also a good tool to test, using calculations, the sensitivity of a value to external parameters. We believe it's a powerful technique that can be adapted in many situations and provide useful results.