Current overview of ICSBEP and IRPhEP benchmark evaluation practices

Two projects sanctioned by the Organisation for Economic Co-operation and Development (OECD) Nuclear Energy Agency (NEA) have over two decades of experience developing established and comprehensive data sets in handbooks supporting criticality safety and reactor physics. The International Criticality Safety Benchmark Evaluation Project (ICSBEP) and the International Reactor Physics Experiment Evaluation Project (IRPhEP) serve as examples of quality and excellence in preserving our experimental data heritage and establishing integral benchmark standards upon which current and future modeling, validation, and safety efforts can be supported. Evaluation practices have evolved with each year of these projects to include additional benchmark experiment data, establish more comprehensive techniques for evaluation of uncertainties and biases, and encourage established high-quality peer-review efforts. This paper will summarize the current format of the handbooks, best-practices for a comprehensive benchmark evaluation, recent activities and protocol within these projects, and a look into future identified needs and activities.


History
In October 1992, the United States Department of Energy (US DOE) initiated the Criticality Safety Benchmark Evaluation Project (CSBEP), which quickly became an international effort and an official activity of the Organisation for Economic Co-operation and Development (OECD) Nuclear Energy Agency (NEA) in 1995, as the International Criticality Safety Benchmark Evaluation Project (ICSBEP). The ICSBEP was established to identify a comprehensive data set of verified criticality benchmark data, evaluate the data and quantify overall uncertainties, compile the data into a standardized format, perform calculations of each experiment with standard criticality safety codes, and formally document the work in to a single source of verified benchmark data. [1] Over two decades of international collaboration have led to the development of the well-established international standard of benchmark excellence contained within the International Handbook of Evaluated Criticality Safety Benchmark Experiments (ICSBEP Handbook). [2] The contents of the ICSBEP include benchmark specifications that have been derived from experiments performed at various nuclear facilities worldwide. These benchmark specifications are intended for use by criticality safety engineers to validate their calculation techniques and establish margins for safe use of fissile material. Many of these benchmark specifications have also been found useful for the testing of nuclear data, as demonstrated * e-mail: john.bess@inl.gov * * e-mail: tatiana.ivanova@oecd-nea.org in the most recent release of ENDF/B-VIII.1 nuclear reaction data library. [3] The capabilities and expertise developed through collaborative international benchmark efforts has since contributed to the successful development and implementation of a similar project supporting reactor physics: the International Reactor Physics Experiment Evaluation Project (IRPhEP). [4] The IRPhEP was initiated as a pilot project in 1999, by the Nuclear Science Committee (NSC) of the OECD NEA. The project was later endorsed as an official activity in June 2003. The IRPhEP was closely patterned after the ICSBEP and has been coordinated closely with the ICSBEP so as to avoid duplication of efforts and publication of conflicting information.
Reactor physics benchmarks prepared via the IRPhEP are published in the International Handbook of Evaluated Reactor Physics Benchmark Experiments (IRPhEP Handbook). [5] The IRPhEP Handbook provides benchmark specifications intended for use by reactor designers, safety analysts, and nuclear data evaluators to validate computational techniques and data.
These handbooks are available freely to member countries of the OECD NEA and countries actively participating in their respective international benchmark projects.

International Benchmark Legacy
Benchmarks are integral validation components of contemporary neutron transport and nuclear data libraries, supporting efforts in advanced modeling and simulation; analytical methods development, validation, and verifications; reactor design and licensing; education and training; criticality and reactor safety analysis; fuel cycle and related activities; range of applicability and experiment design; and nuclear data refinement. These benchmark efforts concatenate and preserve international nuclear information; recover lost data; identify areas where additional data are needed; incorporate resources from the international community to fill identified data gaps; identify discrepancies between calculations and experiments resulting from deficiencies in reported experimental data, cross section data, cross section processing codes and neutronics codes; reduce and ultimately eliminate large quantities of redundant research and processing of existing data; and improve future experiment planning, execution, and reporting. An overview of the rigorous benchmark evaluation process is depicted in Fig. 1.
This paper provides an overview of many of the best practices employed by the ICSBEP and the IRPhEP, as example benchmark programs requiring rigorous quality control practices and extensive peer-review sustaining modern and future validation practices. The expertise, practices, methodologies, and application to support and sustain integral benchmark data [6] initiated via the ICS-BEP and IRPhEP can be applied towards other areas for modernized computational validation. Ongoing activities at the OECD NEA include planned development of a benchmark handbooks supporting SINBAD (Shielding Integral Benchmark Archive and Database) experiment data, [7] the Spent Fuel Composition (SFCOMPO) database, [8] and the Expert Group on Multi-physics Experimental Data, Benchmark and Validation (EGMPEBV). [9]

International Collaboration
International collaboration is integral to all activities under the OECD NEA umbrella. These benchmark efforts represent contributions of nuclear data and experience from leading international experts. The unique facilities, capabilities, research activities, and needs of each country, both historical and present, melds together within NEA efforts to further advance safe and economical usage of nuclear energy. The availability of diverse participation guarantees blended views throughout the review process. Collaboration amongst international experts provides the means through which expert opinions and practices can be expressed and shared. Contributions of nuclear experimental data confirms the adequacy of evaluated integral benchmark data, provides understanding into recorded historical data results, as well as guides the measurement and evaluation of new experimental data.

Well Established Handbook Format
Quality benchmark evaluation has four integral aspects: • Detailed experiment description, • Evaluation of experimental data to obtain parameter values that define the model and their respective uncertainties, • Derivation of a concise description of the benchmark model, and • Sample calculation results. [10] These four aspects are augmented via the facilitation of uncertainty guides and a stringent review process. Development of uncertainty guides by participants in these international benchmark projects retains the knowledge of best practices. A comprehensive peer-review process engenders trust in the validity of benchmark data contained within the handbooks.

Detailed Experiment Description
All available information pertaining directly to the experiment should be succinctly collated within the handbook with appropriate references cited. Detailed descriptions of geometric dimensions, material properties, and measurement methods should be identified and retained; where possible, reported uncertainties and biases, known to the original experimenters, should also be recoded. All known, relevant data about the experiments are carefully studied, compared, and reported to provide a comprehensive summary regarding the evaluated experiment.
Wherever plausible, additional information gathered from the experimenters, experimental facility, logbooks, unpublished laboratory reports, internal memorandums, photographs, sketches, analytical measurements, etc. should be gathered. The purpose is to preserve all known data into a single location, saving the time and resources of future duplicating efforts, and to assess limitations in the data, such as conflicting or missing information. Information that is missing/unknown should also be clearly indicated such that the user readily recognizes the limitations of the preserved experimental data.

Extensive Experiment Evaluation
Comprehensive evaluation of the experimental parameters is often the most time intensive steps in benchmark development. This essential step is often overlooked in basic "validation" analyses consistent of comparison between rough simulated model results and raw experimental data without consideration of any reported, or unreported, biases and uncertainties, where this rough form of "verification" often supplants any actual quality validation process. Characterization of all uncertainties Reported uncertainties collected with the detailed experimental data should be restated; unreported uncertainties, however, should be evaluated using modern modeling capabilities and data. Experiments with significantly large uncertainties or lacking essential data to prepare quality benchmark specifications should be clearly indicated.
Benchmark specifications based upon incomplete and significantly inaccurate experimental data are of little use. However, the analyses are preserved on the handbooks; a poorly performed/recorded experiment may not service for validation, but it does provide insight regarding improvements to future experimental performance.

Benchmark Model Development
Primary users of the handbook often solely focus upon this product of the benchmark evaluation process. Comprehensive specifications should be provided such that any handbook user can prepare a computational model of the benchmark experiment without making any additional assumptions, barring limitations of their implemented nuclear codes and/or data. Figures and tables clearly outlining geometry, dimensions, materials, temperatures, etc. should be provided.
The benchmark model should be simplified such that it still accurately reflects the true physics of the experiment. Gross simplifications should be avoided. The effect of all simplifications should be quantified whether through use of recorded measurement data or simulations such that the benchmark measurement value can be biased to reflect changes in the model from the detailed experiment. A summarization of model simplifications and biases should be provided. An appropriate bias correction is then applied to the benchmark model measurement to compensate for the applied simplification effect. Simplifications should have a minimal impact on the benchmark model biasing, introduce little additional uncertainty to the target benchmark measurement value, and provide a benchmark model more easily simulated.

Sample Calculations and Input Decks
The use of contemporary neutronics codes and nuclear data libraries indicates current capabilities to adequately simulate the benchmark experiment. Inadequacies or limitations in software capabilities, or deficiencies in available nuclear data to which a benchmark might be sensitive, inhibits proper evaluation of the uncertainties and biases computed. Calculations performed using multiple computer codes with varying nuclear data libraries provides sufficient understanding in variability in code capabilities, nuclear data processing, and/or completeness in library coverage.  [11] this project would streamline the procedures for validating nuclear data libraries. A component of this project includes collecting and verifying the accuracy of utilized benchmark models.

Uncertainty Guide Development
The benchmark and experiment design community benefits from the development of uncertainty guides for criticality safety and reactor physics. Participants in these international projects contribute their expertise to enable the preservation of best practices, both historically and at present, for the benefit of current and future benchmark evaluation. Characterization and quantification of the typical uncertainties encountered for the varying measurements in criticality safety and reactor physics systems further contributes towards activities extended beyond typical benchmark evaluation and validation such as training, licensing, and design.
The ICSBEP Guide to the Expression of Uncertainties [12] was developed primarily to address uncertainties in the evaluation and characterization of critical experiments. The International Reactor Physics Experiments Evaluation Project (IRPhEP) Guide to the Expression of Uncertainty [13] was more recently developed to support evaluation of uncertainties in other types of measurements, such as, buckling, spectral characteristics, reactivity effects/coefficients, kinetics, and reactionrate/power distributions.

Comprehensive Review Process
As mentioned previously, the availability of international participants with varying expertise enables a more thorough and comprehensive peer-review process. The technical review process for the benchmark handbooks is more strenuous than encountered in other peer-reviewed publications such as conference proceedings and journals. The participants in each step of the quality assurance process for the ICSBEP and IRPhEP Handbooks is summarized as follows: • Evaluator(s), • Internal Reviewer(s), • Independent Reviewer(s), • Technical Review Group (TRG), • Final Subgroup Reviewer(s), if needed, and • International User Community.
Primary assessment of the benchmark experiment and parameters is performed by the evaluator(s) of said experiment. In-house verification of the analyses presented, including adherence to handbook guidelines and procedures, is then performed by the internal reviewer(s). Independent review is then enacted by an external (typically from another country) participant to again verify the analysis. Should the availability of sufficient personnel to provide internal review be deficient, then at least a second independent reviewer will be assigned to a given benchmark. Upon completion of the independent review and resolution of any outstanding review comments, the benchmark evaluation may be submitted to the annual technical review process.
Participants in the TRG gather annually to review and discuss, page-by-page, annual submissions to the benchmark handbooks. This tedious task guarantees an in-depth review and allows for in-person discussion of review comments and questions with the evaluator(s) in attendance. Any remaining outstanding review comments are recorded as action items to be completed prior to final acceptance and publication. Final acceptance of resolved action items is provided by the independent reviewer(s) plus any additional volunteer reviewer(s) assigned to a subgroup. Ultimately, once a benchmark has been published, the international users can provide feedback to the Chairman of the ICSBEP and IRPhEP regarding identified errors, comments, and/or questions such that clarifications can be issued, and revisions applied within the next handbook editions. The ICSBEP and IRPhEP Handbooks are typically updated annually.

Evolving Database Tools
With ever-increasing handbook content, it was necessary to develop and evolve database structures able to collate, organize, and ultimately facilitate their use. The Database for the International Handbook of Evaluated Criticality Safety Benchmark Experiments (DICE) was developed for the ICSBEP Handbook. [14] The DICE program contains a relational database loaded with pre-selected information from each of the benchmark evaluations contained within the handbook. A users' interface enables querying for specific parameters. DICE also includes the means of plotting for comparison of neutron spectra and sensitivity coefficients for sets of data where that information is available.
Similarly, the IRPhEP Database and Analysis Tool (IDAT) was developed for user interface with the IRPhEP Handbook. [15] This database tool is essential for the identification of validation cases from various reactor types and measurements. The database also contains calculated quantities for these reactor systems, such as, neutron flux/capture/fission spectrum data, neutron balance data, and sensitivity data, also with the capability to visualize these data. Participation There is an ever-growing need for additional participants to serve as reviewers and evaluators of new benchmark experiment data. Not only is there a need to continue the practice of preserving historical knowledge and experience, but to sufficiently analyze and assess contemporary experiment data for inclusion in these benchmark handbooks. These international projects provide networking experience, the opportunity to publish, and enable crossvalidation efforts across experiments, measurement techniques, software systems, and nuclear data libraries.

Conclusions
The criticality safety community has explicit expectations for validation. Benchmark experiments within the ICS-BEP Handbook aptly demonstrate how well this community understands and addresses calculations and their reflection of reality. The best practices of the ICSBEP have readily been adapted into the IRPhEP and continue to be adopted in to other international projects endeavoring to provide higher-quality benchmarks to sustain their validation needs. Such practices are no longer the exception, but the norm, when addressing modern and future modeling and simulation practices. Efforts to apply lessons learned via the ICSBEP and IRPhEP to similarly proposed benchmark activities, will effectively expand the availability of benchmark experiment data to support the ongoing activities of our nuclear community, including nuclear safety.