NDEC: A NEA platform for nuclear data testing, veriﬁcation and benchmarking

. The selection, testing, veriﬁcation and benchmarking of evaluated nuclear data consists, in practice, in putting an evaluated ﬁle through a number of checking steps where different computational codes verify that the ﬁle and the data it contains complies with different requirements. These requirements range from format compliance to good performance in application cases, while at the same time physical constraints and the agreement with experimental data are veriﬁed. At NEA, the NDEC (Nuclear Data Evaluation Cycle) platform aims at providing, in a user friendly interface, a thorough diagnose of the quality of a submitted evaluated nuclear data ﬁle. Such diagnose is based on the results of different computational codes and routines which carry out the mentioned veriﬁcations, tests and checks. NDEC also searches synergies with other existing NEA tools and databases, such as JANIS, DICE or NDaST, including them into its working scheme. Hence, this paper presents NDEC, its current development status and its usage in the JEFF nuclear data project.


Introduction
In nuclear applications where neutrons, protons, heavy particles/ions, gamma or other ionising radiation are handle, the data which quantify the interaction of these particles/radiation with matter are usually provided by evaluations.They are the best estimations of such physical magnitudes, merging together the best from experimental data available and physic models.
Evaluations data are usually stored into a format.In the framework of this paper, we refer only to evaluations stored in ENDF-6 format [1], despite the efforts ongoing for its modernization into GND format [2].
To ensure the evaluations quality, they are tested, verified, benchmarked and validated.So, it is checked that not only they represent the best knowledge available of such data but also simulations of applications, e.g.neutron transport, provide results in agreement with experimental data.These tasks are normally performed with software codes.Thus, they could be easily automatized, facilitating their generation to evaluators and their usage to users.
In this direction, there are different projects ongoing, as summarised in Ref. [3].Here we highlight those which are open to public (upon demand): ADVANCE [4] and MyENDF [5].Both systems launch automatically a set of processing codes on files submitted, and then parse the processing code outputs to inform evaluators/users about the quality of the files, remarking relevant issues identified by the codes.
At NEA (Nuclear Energy Agency), the NDEC (Nuclear Data Evaluation Cycle) platform is under development.It aims at automatizing the tasks of testing, verification and benchmarking of nuclear data evaluations, as well as those systems mentioned before.Apart from a e-mail: carlosjavier.diez@csn.esconsidering NDEC as a new open service, it supports the JEFF (Joint Evaluated Fission and Fusion) project [6].Within JEFF, NDEC is used for the assessment of evaluations and to assist with new nuclear data library releases, as done for the JEFF-3.3beta releases.
With this paper, we present the NDEC platform, its current status and examples of usage within the JEFF project.To remark, neutron-induced evaluations are only currently considered in NDEC.

The NDEC platform
The NDEC (Nuclear Data Evaluation Cycle) platform aims at 1. helping evaluators to test, verify and benchmark nuclear data evaluations, 2. helping users by providing, from evaluations, readyto-use processed files for simulations tools.
Both goals can be achieved by the definition, implementation and automatization of sequences of nuclear data processing codes, where the processing codes performs the tasks mentioned above while processed files are generated.This implementation takes the automatization as its core, but also modularity and reproducibility are part of the NDEC characteristics.
In addition, NDEC shall provide an intelligent diagnose, which means to provide further information than those given solely by processing codes.That requires the implementation of a logic that considers the outcome of different processing codes.Also, NDEC searches synergies among other NEA tools, taking advantage of them within the nuclear data framework.
In overall, NDEC consists in three stages, that may overlap with themselves: processing, verifications and benchmarking, which are explained as follow.

Processing
It means to reconstruct and convert evaluated data stored in ENDF-6 format into different formats, considering the needs of the nuclear data users.Taking JEFF as one of the main NDEC users, the selected output formats are: • HENDF (Hybrid ENDF): A merge of the original evaluated ENDF-6 file with its PENDF (Point-wise ENDF), taking the reconstructed cross section (MF3) in the PENDF file to substitute the ones in the original ENDF files.This format is used to update JANIS [7] database.
• GENDF (Group-wise ENDF), where cross section data from the ENDF-6 file is collapsed into a group-wise energy structure using a flat neutron spectrum.• BOXER: Corresponds to a format for storing covariance data for cross sections.These files are then used with JANIS to update its database of nuclear data libraries.• ACE (A Compact ENDF): Corresponds to the format required by MCNP [8] for performing calculations such as criticality.These files are then used for benchmarking.

Verifications
They consist in checking that format is correct and main physical constraints are fulfilled, not only in the original ENDF-6 format files but also in derived files such as PENDF (Point-wise ENDF) files.This task is partially carried out with processing codes, while further verification requires comparisons between evaluated data, and also against experimental data.The codes currently used for this tasks are ENDF Checks & Utilities (C&U) codes [11], PREPRO-2012 and NJOY2012.50.For nuclear data comparison are considered Refs.[12] and [13], both of them using identification of outliers using statistical estimators.However, they are not implemented yet.

Benchmarking
The benchmarking here is carried out by comparing simulation results of experiment models against real experimental data.So, if the models and simulation codes are reliable enough, differences between calculated and experimental data shall come from the input parameters, in this case, the nuclear data.
Currently, the benchmarking is performed at library level.That means once a complete cross section library has been processed and verified, 123 cases [14] of the ICSBEP Handbook [15] are simulated and compared with the experimental data.
This stage is still under development, although it could be performed manually, as done for JEFF-3.3beta releases.

Implementation of NDEC
The implementation and development of NDEC is carried out in two ways.The first one is aimed at defining a sequence of codes, and then automatize it.So the sequences are tested and analysed, and can be run on a  desktop or cluster computer.Meanwhile the second takes the sequences already implemented and prepare them for its use in a web-server, so evaluators and users have a online platform where to submit evaluations and retrieve the results.With this latter one, the goals of NDEC can be achieved.

NDEC Proof-of-Concept
The Proof-of-Concept (PoC) of NDEC is based on defining the sequences of processing codes to automatize, so the processing, verification and benchmarking are accomplished.They are implemented in scripts, so they can be tested and analysed individually by running them on desktop or cluster computers.
Two sequences are currently implemented: • Sequence 1 to generate ACE and GENDF format files from evaluations in ENDF-6 format.It makes use of ENDF C&U codes and PREPRO-2012 for verification of the ENDF-6 file, while NJOY2012.50 is used for processing and generating the processed files.
A general overview of the sequences is presented in Fig. 1, where the big arrows indicate the order in which codes are run.The parameters and modules used for NJOY2012.50have been selected by Processing and Verification JEFF group [16].• Sequence 2 to generate HENDF and BOXER files from evaluations in ENDF-6.Again, it makes use of ENDF C&U codes for verification of the ENDF-6 file, while PREPRO2012 and NJOY2012.50are then used for processing, complementing each other, in order to get the desired formats.The general sequence flowchart is presented in Fig. 2.After each script/code is run, a parser is applied on code outputs in search for relevant messages, warnings and errors, and also specific data.It extracts relevant information, and then group them into categories according to a warning/error dictionary.This warning/error dictionary is a novel idea whose objective is to help evaluators to understand the meaning of such messages, while at the same time group them into categories, depending on their type and importance.Only a draft of the dictionary is currently implemented.
Finally, in order to make easier the results visualisation, a HTML webpage is created for each evaluation analysis (Fig. 3).It presents the results in a dynamic and interactive way with expandable lists.

NDEC online
The online version of NDEC is based in a system where evaluators and nuclear data users can submit a file to NDEC via website.Then, the files go through all the sequences implemented, and finally the results (diagnose) are provided with the code outputs, including output files.Most relevant elements and actions/interactions are presented in Fig. 4.
The file submission to the server by a user/evaluator is the main action, done on the home panel, under the "Upload" section (Fig. 5).Once the file is uploaded, it triggers the sequences implemented on the server by administrators, which are described on the "Script" panel (Fig. 6).As a sequence consists in a series of tasks that involve codes and/or scripts, they are sent to workers.Workers launch the tasks, and when they have finished, the results are uploaded back to the server.The server checks whether all the expected results are uploaded in order to check whether it has failed or not.The task status can be checked on the "Work queue" panel (see Fig. 6).If a task  is accomplished successfully, the results (output files) are presented on the home panel, under "Content" section and at the same row of the submitted file (see Fig. 5).There, users can download the results by clicking on the output file names.
Currently, 28 tasks have been implemented on the server, including the sequence 2 presented in Sect.3.

Example of usage within JEFF
The JEFF project is currently working on the JEFF-3.3nuclear data library release.That involves the selection and assessment of evaluations, mainly neutron-induced data.In addition, in order to test the evaluations, it is necessary to process the ENDF-6 evaluations into formats readable by applications.
Hence, NDEC can provide support to JEFF by • processing the files into different formats (see Sect. 2.1), • verifying the files with processing codes (see Sect. 2.2).
That also leaves evaluations ready for benchmarking.
The results from NDEC are provided per evaluation, as shown in Fig. 3, that means all the results have to be checked.However, in order to improve the checking of NDEC results, a HTML webpage has been developed.It summarises in a table all the evaluations submitted for a given release and also includes the number of warning/errors encountered during the processing and verification for every evaluation.A colour legend is applied in case errors are found (red), just warnings (yellow) or  nothing (white).An example of this summary is given in Fig. 7.Because the table is sortable, JEFF community can focus first on those evaluations with errors and then on those with warnings.
Thanks to NDEC, a JEFF release includes more data, apart from the ENDF-6 format files (evaluations): • ACE files with their XSDIR file, tested with MCNP.
• HENDF files with their corresponding INTER and BOXER files.• JANIS database with all evaluations, for their visualisation in JANIS.• NJOY input files, for ACE file generation.

Conclusions
At NEA, the NDEC (Nuclear Data Evaluation Cycle) platform is being developed.It is aimed at automatizing the processing, verification and benchmarking of nuclear data evaluations.So, it will help evaluators and users to speed up the assessment of evaluations and the processing into other formats.
Its implementation is based in two approaches: 1. NDEC PoC: sequences of processing codes are defined and automatised.Processing code outputs are parsed and categorised.It runs in desktop or cluster computers.2. NDEC online: It is a online platform where evaluators and users can submit evaluations via website.The evaluations then go through those sequences from NDEC PoC implemented on the server.
Currently, NDEC PoC has two sequences, so it does processing by converting ENDF-6 format files into different formats (see Sect.Finally, NDEC PoC is currently being used for supporting JEFF-3.3beta releases.It provides processed files ready to be used for benchmarking and JANIS visualisation, and a summary of warning/error messages found during processing which helps JEFF community to identify issues with files.

Figure 1 .
Figure 1.Sequence of processing and verification codes to generate ACE and GENDF format files.

Figure 2 .
Figure 2. Sequence of processing and verification codes to generate HENDF and BOXER format files.

ND2016Figure 3 .
Figure 3. Landing panel when opening the HTML file with the analysis results of 192 Ir from JEFF-3.3 T2.
2.1), and verification by parsing processing code outputs: warnings, errors and other relevant messages.NDEC online has only one of the sequences implemented, the generation of HENDF, BOXER and INTER files.