Quantitative Study
Local
Main goals
Measuring instruments
Main Results (Findings)
Shirali et al. (2018)
Southwest, Iran, Oil Refinery
To assess the content validity, construct validity (by exploratory and confirmatory factor analyses) and reliability (by Cronbach 's alpha and test-retest) of an instrument developed to measure the resilient culture in Safety of socio-technical systems.
Likert Scale (CVI - Content Validity Index, T-test, Lawshe, Crombach 's alpha)
The results of the tests conducted indicate that the instrument is valid and reliable.
Azadeh et al. (2017b)
Southern Iran, Oil Refinery
This study aims to investigate the reciprocal impacts of managerial and organizational factors and resilience engineering through a unique mathematical programming approach.
DEA (Data Involvement Analysis)
It was found that organizational factors have a greater impact on resilience than managerial factors.
It was also found that resilience factors related to learning and flexibility have a greater influence on managerial and organizational factors.
Azadeh et al. (2017a)
Terhan Provincial Oil Company
The objective of the study was to develop a verified tool for the assessment of Resilience Engineering in maintenance organizations in the oil and gas industry.
(AHP) Analytical Hierarchical Process/ algorithm K- means /DEA (Data Involvement Analysis)
The study also confirmed the close relationship between Resilience Engineering and human-related performance modeling factors, suggesting that services to promote resilience engineering will lead to such factors being in good condition so that human error can be eliminated. reduced and safety can be improved.
Shirali et al. (2016)
Process Industry in Iran
This study aims to represent a new vision for assessing Resilience Engineering factors in a process industry using a wide range of indicators. This article aimed at a quantitative assessment of RE based on nine indicators of resilience, namely buffer capacity, margin, tolerance, cross-scale interactions, learning culture, flexibility, anticipation, attention and response using PCA and numerical taxonomy (NT) in a process plant.
Principal Component Analysis/Numerical Taxonomy/Delphi/ Superman – reliability of results
The results of the analysis showed that three indicators of interactions between scales, margin and anticipation were at a level below best practice, respectively. Furthermore, the results of the unit analysis showed that the maintenance, hydrogen and control unit 3 were in the worst condition in relation to the resilience factors raised.
Azadeh et al. (2014)
Petrochemical Industry in Iran
The main objective of this study is to assess the resilience factors of a petrochemical plant, which can be expanded to other industries. It is achieved through a Fuzzy Cognitive Map (FCMs) method, which considers interactions between factors due to their calculated final weights.
Graph theory and fuzzy logic (Fuzzy Cognitive Map)
The results showed that preparation, awareness and flexibility are the most important factors among all nine RE factors. Furthermore, redundancy and teamwork play a small role among RE factors.
Rabbani et al. (2019)
Petrochemical Industry in Iran
To this end, the aim of this study is to present a new performance optimization algorithm based on graph theory, matrix approach and statistical methods for evaluating the resilience engineering culture of a given organization.
Graph theory and matrix approach
Among the 10 departments assessed in terms of resilience factors, “production process”, “quality control” and “polymer chemical operation” were considered the most effective in the resilience culture of the petrochemical plant among the departments considered by the study. The results also indicated that the “inspection”, “laboratory” and “maintenance” departments had the lowest resilience index and require the most attention. Low resilience index means weak system capacity in case of unexpected accidents or crises.
Zarrin & Azadeh (2019)
Petrochemical Plant in Northwest Iran
The aim of the study was to map the influences of resilience engineering on the health, safety, environment and ergonomics management system, using the Z number cognitive map.
The concept of Z- numbers with Fuzzy Cognitive Map (FCM) approach is integrated and a new approach called Z- number Cognitive Map is proposed.
The results showed that the principles of Resilience Engineering have a high impact on HSE management.
The results also indicated that senior management commitment has the greatest impact (0.827) on the environment, learning has the greatest impact (0.792) on health, preparation has the greatest impact (0.786) on ergonomics, and awareness has the greatest impact (0.776) on Safety.
Grecco et al. (2013)
Brazil, Radiopharmaceutical Industry
The aim of this research is to adopt a fuzzy approach Set Theory (FST) to establish a method for assessing resilience in organizations based on proactive Safety performance indicators, defined according to the principles of resilience engineering.
Fuzzy Logic using the Fuzzy Set Theory method to measure resilience factors
We consider an attendance level greater than or equal to 0.6 to be satisfactory. The result of the average assessment showed that the radiopharmaceutical packaging shipping sector presented a satisfactory learning culture, awareness of flexibility, fair culture, and preparation. However, this sector presented problems related to senior management commitment.
Patriarca et al. (2018)
Central European Region.
Based on the four categories (monitoring, response, anticipation and learning), this article aims to define a semi-quantitative analysis to measure organizational resilience in complex socio-technical systems, combining the RAG (Resilience Assessment Grid) resilience analysis grid and the process of AHP analytical hierarchy.
RAG (Resilience Assessment Grid) and AHP - Saaty Matrix
The result of the resilience analysis grid, i.e. a weighted set of probing questions, can be used in different domains as a supporting tool in a broader managerial action oriented towards Safety -II, to bring management safety at the core organization's business.
Nelson et al. (2016)
Idaho National Laboratory, USA
This work describes the analysis of operational data of the information contained in the Corrective Action Program. The methodology considers human error and organizational factors because of their large contribution to consequential events. The results include a tool developed from the data to be used to identify, predict and reduce the likelihood of significant consequential events.
MATLAB Fuzzy Logic Toolbox (version 2.1.1; The Math Works, Inc., Natick, MA, USA)
The results include a tool developed from the data to be used to identify, predict and reduce the likelihood of significant consequential events. This tool is based on the resilience curve that was constructed from the plant's operational data.
Saurin & Werle (2017)
Maternity, Rio Grande do Sul, Brazil
The aim of the study was to learn how to analyze slack resources of different natures in complex socio-technical systems, offering insights into work system design to cope with the unexpected variabilities of complex systems.
Descriptive statistics, such as means, standard deviations and coefficients of variation, were used to analyze the questionnaire data, which produced information that was directly used.
Benefits of the framework for assessing slack: (i) it allows the joint analysis of slack resources of different natures, based on a shared construct (UoS), metric (protection score) and classification scheme; (ii) allows prioritization of sources of variability, based on how effectively they are covered by slack resources ; (iii) sheds light on sources of variability and idle resources that arise from self-organization and informal work practices; this reflects a focus on the work done [16] while leaving room for imagined work, emphasizing the regulations' time off requirements; (iv) UoS identified over time and correlate this with accidents and near misses: the measured clearance is expected to be an important safety indicator.
Slack Units (UoS): each pool of similar slack resources that share the same purpose, corresponds to a UoS, for example, redundant equipment in a surgical ward, regardless of the number of extra equipment.
Next, the extent to which each UoS mitigates each source of variability must be assessed. This assessment is based on a 7-point scale.
Hollnagel et al. (2021)
Sweden, Eurocontrol
Build a basis for the resilient performance of complex socio-technical systems that manage air traffic through a qualitative method of assessing resilient performance in an organization that controls air traffic. Proposing a questionnaire and Likert scale with scores for potentials: monitor, respond, learn and anticipate.
Web-shaped chart with Likert scale scores to measure: monitoring, response, learning and anticipation.
Qualitative method based on a psychometric questionnaire that assesses the resilience potential of organizational systems that aim to promote air traffic management, using the potential for response, monitoring, anticipation and learning.
Øien & Nielsen (2012)
Goliat Field in the Barent Sea
Develop a resilience-based approach, mainly due to the ability to provide warnings for unknown (atypical) scenarios, not covered by a risk assessment. Improving the capacity for anticipation, through indicators, to avoid a major accident.
Awareness of risks
It shows that the method can be applied in any oil and gas industry. It also shows that the proposed general issue indicators can provide early warning for major accidents.
1.1.1. Number of years of knowledge of the production system.
1.1.2. Information about risks through courses, documents, HAZOP, AQRs, in the last 3 months.
1.1.3. Quality of reports of near misses and incidents.
1.1.4. Information on the quality of barriers (technical safety).
1.1.5. Information on the quality of barrier support functions (operational safety).
1.1.6. Discussion of HSE issues/status at regular meetings.
1.1.7. Risk/resilience communication at all levels of the organization.
Anticipation
1.2.1. Risk/hazard identification number (HAZID etc.).
1.2.2. Learn from your own experiences and accidents.
1.2.3. Learn from other experiences and accidents.
Herrera et al. (2014)
Arlanda Airport in Sweden
It proposed a scaling framework with the aim of identifying sets of resilience indicators. Applied the Safety II concept of safety as the presence of capability.
Resilience indicators are nothing more than instantiations of problem solutions that procedures do not cover. Problems presented during the go-around maneuver at a central air traffic management tower.
With regard to Resilience Engineering, the SCALE framework supports the identification and extraction of resilience indicators and their mapping into patterns.
Examples:
(+) Potential conflict in the go-around maneuver administered by two Air Traffic Control Officers (ACTO)
(+) ATCOs (Air Traffic Control Officers) - Prevent a potential collision between aircraft.
Peñaloza et al. (2020)
Brazil, UFRG
A systematic review of the literature aimed to identify whether RE offers a new perspective on safety performance measurement systems, and to understand how RE has been put into practice in safety performance measurement systems.
a) system of indicators must support the monitoring of daily variability;
However, there have been several studies that are moderately or strongly aligned with these guidelines, suggesting that RE has been implicitly adopted to some extent in performance measurement systems.
b) indicator systems must provide real-time feedback to those directly involved
in the execution and supervision of production activities;
c) system of indicators should facilitate learning about what is going well, as well as what is going wrong;
d) indicator system should offer insights into trade- offs management between Safety and other dimensions of the business;
e) indicator system must evolve due to the changing nature of complex socio-technical systems.
Souza et al. (2021)
UFRJ, Rio de Janeiro
It proposed applying resilience engineering concepts to understanding the organization through the analysis of tasks and cognitive activities to develop indicators by the human agents themselves working at the operational level, in addition to the initial specification of a computational tool to support this understanding.
Radial (spider) graphs with scores for the following factors to perform operational tasks, e.g .:
Provides a technological opportunity to detect deviations of work towards its safety boundaries. It allows managers to photographically analyze how activities were or are being conducted throughout the day. Radial graphs are obtained with indicators that affect the team's difficulties in achieving the task's objectives.
training and experience;
communication;
facilities and equipment;
group activities and work interfaces.
Patriarca et al. (2019)
Italy, Rome
The proposed approach aims to encourage workers' engagement in workplace safety and, more generally, to overcome psychological barriers to their participation.
Number of years of experience with NH3 production.
The method is still viewed with great skepticism, as it involves a structure for putting together games. Perhaps with cost and benefit analysis it could be implemented electronically, since assembling the structure and design of the game can be costly.
The approach was explored in a case study within the chemical industry. In particular, the safety-critical sector of ammonia production was addressed, with the aim of promoting and improving its resilience to unwanted events.
Number of hours of risky courses in the last 12 months.
Information on the quality of barriers (technical safety).
Number of barrier failures, e.g. PRV failures.
Number of exceptions handled in the last month
Number of cases of unsuccessful communication between operators during operational maneuvers.
Number of cases in which a response decision was delayed in the last three months.
Chuang et al. (2020)
China, Medical Emergency Department of a Hospital in Taiwan
Hollnagel Resilience Assessment Grid (RAG) into a customized RAG (ED-RAG) to support resilience management in a Hospital Emergency Department in Taiwan. ED-RAG (Emergency Department - RAG)
Scores from 0 to 4 for specific questions prepared by a team of researchers and employees who knew the department. With this, it was possible to define grades from 0 to 100% for the four resilience skills: potential to learn (86.11%); response potential (61.56); anticipate (56.25%); monitor (33.93).
The ED-RAG represents a snapshot of the resilience of EDs under specific conditions. It can be performed multiple times by a single hospital to monitor the directions and content of improvement that can complement conventional safety management toward resilience. Some considerations are necessary to be successful when hospitals use it. Future studies to overcome potential methodological shortcomings of the ED-RAG are needed. Among the four skills evaluated, the only one that was below was monitoring.
Sakuda & Kitamura (2020)
Fukushima Nuclear Power Plant, Japan
Used RAG to prevent degradation of resilience potential in a nuclear power plant.
Spider chart with Likert scale scores to measure monitoring, response, learning and anticipation.
It is found that fragility-based RAG can be a useful tool to facilitate awareness about possible weaknesses of nuclear power plant.
Jain et al. (2018)
University of Texas, USA
Presented a generic framework for measuring the probability of resilience of a process system based on metrics in which a Bayesian Network (BN) can be developed using an equation.
Resilience Analysis Framework (PRAF), the metrics were developed considering the performance indicators established by API RP 754 (API, 2010) and based on the resilience aspects of the Early Detections (ED) process, Error Tolerant Design (ETD), ability to recovery (recoverability) (R) and (plasticity) plasticity (P). These were categorized into technical (ED, ETD, R) and social resilience (P).
The Process Resilience Analysis Framework is
Note: uses the calculation of these factors in a Bayesian Network, however, it did not demonstrate values.
presented in the context of improving risk and safety management.
As illustrated, the key aspects of process system resilience are detection (ED), error-tolerant design (ETD), recoverability (R), and plasticity (P). In application, the use of these aspects serves to drive predictability.
Huber et al. (2012)
Institute of Nuclear Engineering, Cidade Universitária, Rio de Janeiro, RJ, Brazil
The aim of this study was to propose a framework to indicate where the organization is located, within the capabilities necessary to deal with the disturbances that may affect it (the organization's level of resilience).
Conscience;
In the experts' assessment of the air taxi company's resilience application, the method and its division of activities into three levels resulted in activities compatible with the capabilities, interests and availability of those involved, and was therefore well accepted. The combined concept of practical development (a representative, participatory, iterative and interactive method) of resilience indicators has also been well accepted and should be tested in more areas of work.
efficiency;
adaptability;
commitment.
Pflanz & Levis (2012)
Maritime Industry, US Navy Command and Control Center in Georgia, USA
This article describes a quantitative approach to assess the expected resilience of a command-and-control system. It presents guidelines for measuring organizational resilience, based on proxy measures, such as tolerance to errors, responsiveness to unexpected events and level of connectivity between elements of the system or organization.
Petri Net software presents linear algebra network modeling that demonstrates how to calculate resilience.
It presented guidelines for measuring organizational resilience based on proxy measures such as error tolerance, responsiveness to unexpected events, and level of connectivity between system elements.
Ranasinghe et al. (2020)
Newcastle University
The aim of this research is to explore RE indicators that have been identified as important in developing and evaluating resilient work environments in high-risk industries, particularly construction renovation.
Commitment from senior management;
The results show that the four commonly used indicators were: senior management commitment, awareness, learning and flexibility, all of which have a strong relationship with RE. The findings of this study are useful for interested parties in making decisions about the most important RE indicators in the context of their research or practice, as this avoids ambiguity and disparity in identifying RE indicators.
awareness
apprenticeship
flexibility
Source: Prepared by the author (2022).
Thumbnail
Thumbnail
