|
|
 |
ITEP/JMU Database of International Experiences: Supporting the Test and Evaluation Community
by Franciska Borry and Dieter Guelle, ITEP
Introduction
A new database has been added to the James Madison University (JMU) Lessons Learned database, with more specific content and aim. The specific content reflects the main tasks that the International Test and Evaluation Program (ITEP) for humanitarian demining has agreed on in its Memorandum of Understanding, namely to evaluate and standardise the process of equipment testing in the humanitarian demining industry. The JMU Mine Action Information Centre (MAIC) maintains the database, whereas the ITEP Secretariat is responsible for its content.
The ITEP/JMU Database of International Experiences in Support of the Test and Evaluation Community (DIETEC) was created in order to summarise test and evaluation (T&E) experiences that are referenced to the original test reports. The most demanding part was to work out the database structure and to discuss the concrete contributions currently included. There are fine dividing lines, when at all, between lessons learned, experiences and test results. A remaining question is whether it is opportune to also include technical questions related to the use of tested equipment (test results) or to strictly focus on the testing process. In this regard, we would like to invite the reader to share his/her opinion with the ITEP Secretariat.
Currently, the database only contains experiences related to the T&E process. It was found that the test reports contain an important amount of information, either general or specific, which could be useful for the T&E community as a whole. In order to make these hidden experiences more widely and easily available to the test community, ITEP decided to join up with the MAIC. Funding was provided by the U.S. Department of Defense.
Aim
The main aim of DIETEC is to collect and publish information derived from experiences in T&E of humanitarian demining equipment. The collected "experiences" are intended to highlight key areas of consideration in the T&E process, as well as specific observations related to the evaluation of humanitarian demining equipment in operational use. Moreover, it attempts to provide a common structural approach for T&E of equipment used in humanitarian demining. The database should provide useful information for a variety of T&E stakeholders, ranging from test engineers and entities involved in large-scale T&E campaigns to the individual user at the field level interested in evaluating his/her specific piece of equipment.
Definitions and Approach
The database, in its current version, provides a list of experiences. The experience may be positive or negative but should meet the following criteria:
- It should be significant in that it has real or assumed impact on the T&E operation.
- It should be valid in that it is factually and technically correct.
- It should be applicable in that it identifies a specific process or decision that reduces or limits the potential for failures and mishaps, or reinforces a positive result.
- It should be understandable for a skilled person not necessarily intimately familiar with the subject matter (adapted from the U.S. National Aeronautics and Space Administration's definition of lessons learned).
The experience may be generic or equipment-specific and can be submitted by any individual or organisation. However, a core set of experiences is extracted from publicly available resources (T&E reports on humanitarian demining equipment) and from T&E activities undertaken under the umbrella of the International Test and Evaluation Program for Humanitarian Demining Equipment.
The intention is that all experiences entered in the database will be periodically reviewed by an international panel of experts from the ITEP network and other organisations performing activities related to T&E, e.g., the United Nations or the Geneva International Centre for Humanitarian Demining (GICHD). During this process, the relevance of the experiences to T&E standards, technical notes and/or methodologies will be assessed.
An entry could become one of the following four things:
- An item added to an ITEP T&E methodology: The experience is seen as an important and/or a new way of carrying out tests. A spin-off of this could be a request for further research to be carried out to back up a new test methodology. This further research could ultimately be executed in the form of a collaborative project under the ITEP umbrella.
- An item added to the Lessons Learned/Experiences database: The experience is judged to have a significant impact on the testing method and is preferably backed up by at least two other experiences. An existing test methodology may be changed or updated to take into account this experience. An entry can also fall into this category if the experience reports on an unexpected event during testing and outlines a strategy for dealing with it.
- An item circulated to the User Community: The experience has an impact on the operational use of demining equipment/systems. It should be passed on to the User Community, for instance through a Technical Note for Mine Action (TNMA) issued by the GICHD.
- Dismissed: The experience does not fall into one of the above categories. It may be dismissed with reasons stated.
In its initial stage, the database will mainly include "experiences." After review, the database will be expanded with "lessons learned," referenced to a set of experiences. These lessons learned can then further be incorporated into standards and similar documents whenever relevant.
Approach
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Figure 1: Structure of the DIETEC database. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||
The structure of the database is illustrated in Figure 1 above. Category and sub-category fields have been selected in order to provide the user with a structured overview of the main stages and factors that should be considered during the T&E of humanitarian demining equipment. The structure may be used as a guide for users when drawing up their own test plan. The utilized categories expand on the structure given in the International Mine Action Standards (IMAS) 03.40 on Test and Evaluation of Mine Action Equipment (first edition, 01.01.2003) and include the main technical categories utilized by ITEP and the GICHD Mechanical Demining Equipment Catalogue.
Implementation
The database consists of two main categories: Generic Considerations and Equipment-Specific Considerations. Each main category has subcategories to explore the experiences in detail and to guide the user to add relevant experiences. Each subcategory can be browsed by clicking on the corresponding individual cell or by executing a detailed search. The user can add an experience to a subcategory by clicking on the corresponding individual cell or on "Add Experience." Table 1 includes two examples extracted from the database in order to illustrate the type of information provided by DIETEC.
|
||||||||||||||||||||||||||
| Table 1: Two examples of testing and evaluation experiences extracted from the DIETEC. | ||||||||||||||||||||||||||
As mentioned in the introduction, during the analysis of the reports, valuable information was also encountered that is related to technical and operational deployment of the equipment. For instance, the report The Severe Duty Vegetation Shredder Technical Testing of Capability by the U.S. Night Vision and Electronic Sensors Directorate (NVESD, online at http://www.humanitarian-demining.org/demining/pubs/clearance/svc_test_report.asp) mentions the fact that fence wire tangled up in the rotating machinery and inflicted considerable damage/disturbances during the testing. This finding could also have implications for the operational use of this type of equipment, not only in fenced minefields but also in dense vegetation where liana-like plants can have the same effect. Hours may be needed for the machinery to become operational again. Important feedback from the reader could be an indication that similar information would be worthy of being included in DIETEC.
Final Remarks
Care should be taken when using the term "lessons learned." In general, the process followed for compiling a "lesson learned" is quite complex and consists of an information-gathering and -processing chain spread over a considerable time period. For instance, the Swedish EOD and Demining Centre (SWEDEC) lessons learned project includes several phases such as the collection of an "experience report" using a standard form, which is then analysed and commented on in an "extended experience report." This stage is followed by validation of the information (i.e., Did it happen several times? Is it useful for another organisation? etc.), which leads to the implementation phase and the compilation of the "lessons learned report." A similar process is being applied by the Department of Energy (DOE) Corporate Lessons Learned Collection database, for example. However, this is not the approach followed at present by the JMU MAIC lessons learned database and is, in our opinion, at this moment in time not necessary, mainly due to the given structure of the database.
Both the JMU MAIC database and DIETEC are currently being evaluated by an international team of "experts." No final results are available yet, but preliminary results of the assessment indicate that an important percentage of the DIETEC experiences have been classified as relevant to the T&E process.
The ITEP would like to get your feedback and opinions when you are visiting the database via http://maic.jmu.edu/iteplessons/ or http://www.itep.ws/.
Contact Information
ITEP Secretariat
JRC/EC T.P. 723
IT-21020 Ispra (VA)
Italy
Franciska Borry
E-mail: Franciska.borry@jrc.it
Tel: +39 0332 789138
Dieter Guelle
E-mail: Dieter.Guelle@jrc.it
Tel: +39 0332 785576