NICU Standard 29: Usability Testing
Each new NICU shall perform usability testing (e.g., simulation-based mock-up evaluations) during project design and commissioning. Involving multidisciplinary teams that include family members will enhance physical design, process resiliency and patient safety.
Interpretation: Latent safety threats (LSTs) emerge when existing processes are translated to the new environment. Simulation-based testing and mock-up evaluations help identify LSTs, improve planned NICU design and process, and prepare staff. Evaluations during the design process differ from those during post-construction commissioning. Pre-construction design-focused evaluations use physical or virtual reality mock-ups to help guide design decisions that optimize the built environment for safe and efficient patient- and family-centered care. Commissioning evaluations just prior to move-in give greater clarity about LSTs at the interface between clinical teams and new technology infrastructure. Though facility redesign is not feasible during commissioning evaluations, identification of LSTs enables mitigation prior to exposing babies through training or workflow modification.
Multidisciplinary team participation that includes family members in simulation-based scenarios may reveal unexpected consequences of performing routine and emergent workflows in the new environment. Including family members representing the diversity served by the NICU in simulations is crucial to enhance outcomes. Diverse family collaborators provide valuable perspectives during simulations and debriefing. Consideration should be given to ensure that family members are selected and supported in participating to minimize any repeat trauma or undue burden (e.g., PTSD) from participation in the scenario enactments or process. Family members with experience serving as a member of a family advisory council and/or a design planning committee may be particularly helpful to this process and can provide richly informative staff-family interaction insights.
Mock-ups range from simple (e.g., tape on the floor) to detailed (e.g., plywood constructed walls and furnishings) to virtual reality representations of the planned space. Virtual reality mock-ups are 3D, fully immersive, photorealistic, interactive virtual environments that are experienced using a head-mounted display. A favorable return on investment has been estimated for evaluations within each mock-up type (simple, detailed, and virtual reality), ranging from $5.06 to $26.85 for every dollar invested20.
A framework describing how to plan, design, and evaluate a mock-up, along with resources and templates, is publicly available through the Health Quality Council of Alberta at www.hqca.ca/humanfactors. The framework and the guiding principles from the framework have been incorporated into the National Standards of Canada (CSA Z8000-18) and are promoted as an FGI-supported resource on the FGI website. The following principles builds on those found in the framework:
1. Simulation-based mock-up evaluations should be considered for each design phase from schematic design through commissioning. The consideration should occur during the pre-design stage.
2. Each mock-up evaluation should be thoroughly planned. The scope of evaluations should be outlined in the pre-design stage with attention to the time and costs required to build the necessary fidelity for that stage. Define a timeline incorporating each evaluation phase, the staff education plan, construction contingencies, and the projected move-in day. Gather stakeholder commitment around the timeline. Delineate clear evaluation objectives for each design phase to maximize effectiveness. Evaluation objectives might include unit configuration, room size and configuration, space and access requirements, visibility requirements, lighting requirements, and/or ceiling and floor color choices. Solicit multidisciplinary input to generate robust evaluation objectives that not only structures each evaluation, but also pays dividends in participant engagement as well as in LST reveals.
3. Building of the mock-up should align with evaluation timing and objectives. The degree to which a mock-up is completed (mock-up fidelity) can vary significantly. The mock-up should be built to an appropriate level of fidelity to enable testing of evaluation objectives during the appropriate design phase and may, for example, include real or mock-up furniture and equipment, tape-out walls and equipment, or fully constructed walls with a ceiling and functioning lights.
4. Roles and responsibilities for those involved in the evaluation should be clearly defined. This includes identifying who will be responsible for evaluation design, staging the mock-up, data collection, and data analysis as well as who will participate in the simulation-based scenario enactments. Areas of specific expertise (e.g., human factors) should be assessed to identify if individuals external to the organization are needed.
5. The simulation scenarios that are created and enacted should test evaluation objectives. Frequent, urgent, or challenging clinical situations will prompt a series of simulation tasks that draw participants towards these objectives. The mock-up space, supplies and equipment help immerse participants in the simulation. The scenarios are enacted by users of the space within the mock-up, which includes needed supplies and equipment (real or mock-ups).
6. Recommendations will be evidence-based as observed during simulation-based scenario enactments. Data collected through debriefings and video analysis identify potential issues and successes with the planned design. Identified issues should be assigned to process workgroups for correction, tracked for resolution, and disseminated to staff. Staff awareness can be enhanced through regular messaging or other simulation-based activities.