Effective immediately, Pipeline and Hazardous Materials Safety Administration (PHMSA) inspections will emphasize the review of operator methods for integrity management (IM) program evaluations, the agency said in an Advisory Bulletin released earlier today.  PHMSA also stated that it will carefully evaluate whether operators have meaningful metrics to identify how well they are implementing their IM programs, how well their systems are responding to their IM programs, and how well their systems’ integrity is maintained.

The Advisory Bulletin is not surprising in light of concerns that the National Transportation Safety Board (NTSB) raised in its investigation of PG&E’s September 2010 gas pipeline accident in San Bruno, California.  In its report on that investigation, the NTSB specifically called out concerns with PG&E’s self-assessment of its IM program and with PHMSA’s oversight of performance-based safety programs like IM.

Regardless, liquid and gas pipeline operators are now on notice that PHMSA will immediately be focusing on operators’ selection of meaningful metrics “that allow them to quantify, understand and improve their own performance.”  The agency stressed that an effective operator performance evaluation process should include:

• A well-defined description of the scope, objectives, and frequency of program evaluations.
• The use of periodic self-assessments, internal or external audits, management reviews, performance metrics analysis, benchmarking against other operators, or other self-critical evaluations to assess program effectiveness.
• Clear performance goals and objectives to measure the effectiveness of key integrity activities.
• Clear assignment of responsibility for implementing required actions.
• Review and follow-up of program evaluation results, findings, and recommendations, etc., by appropriate company managers.

As to having meaningful metrics, PHMSA urged operators to focus on:

• A description of the type of performance measures to be used, along with the data sources, data validation and quality assurance activities, the frequency of data collection, and any normalization factors.
• A means to update the performance measures (if needed) to assure they are providing useful information about the effectiveness of integrity management program activities.
• The use of performance metrics data to check and calibrate the operator’s risk analysis tools to assure that these best represent the performance of the operator’s specific assets.

The agency stressed that the annual performance metrics reported to PHMSA are “a small subset of the overall suite of metrics to be used” internally to evaluate IM program performance.  This larger suite of metrics includes those for measuring:

• Overall program effectiveness indicated by the number of releases, number of injuries or fatalities, volume released, etc.
• Specific threats that include both leading and lagging indicators for the important integrity threats on an operator’s systems. These include:
  • Activity Measures that monitor the surveillance and preventive activities that are in place to control risk.
  • Deterioration Measures that monitor operational and maintenance trends to indicate if the program is successful or weakening despite the risk control activities in place.
  • Failure Measures that reflect whether the program is effective in achieving the objective of improving integrity.
• Metrics that measure and provide insights into how well an operator’s processes associated with the various integrity management program elements are performing. Examples of such processes would include integrity assessment, risk analysis, the identification of preventive and mitigative measures, etc.

Finally, PHMSA put large operators on notice that they will be held to a higher standard: “While operator-level rollups of metrics are useful for small operators, a robust program for large operators should also include metrics at a more granular level. The metrics should enable operators to drill down to understand the performance of specific systems or segments within systems. This is particularly important for the threat-specific metrics mentioned previously.” 

The Administration also reminded operators both large and small of the requirement that they keep records supporting the decisions, analyses, and processes they develop and use in their evaluation of IM program effectiveness.