Data-Driven Pipeline Management System for Oil and Gas...

Data-Driven Pipeline Management System for Oil and Gas Transportation in Unconventional Plays

Dr. Huzeifa Ismail, Founder, Chirality Research Inc

Dr. Huzeifa Ismail, Founder, Chirality Research Inc

Advances in horizontal drilling have made exploration of unconventional hydrocarbon resources commercially viable. However operators who exploit this resource must also deal with the enormous challenges. One of such challenges is pipeline management. It is extensive because the liquids pipeline network in the United States now totals approximately 200,000 miles.  Not having a pipeline integrity management program can not only cause major operating expense (OPEX), but it can also be detrimental to the organization’s image if a pipeline fails or worse, leads to a spill.  In 2014, there were over 9,000 pipeline damage-related incidents. These incidences highlight how important it is to have a pipeline integrity management system in place to foresee such disasters before they become a reality. Chirality Research’s team has developed and implemented a data-driven pipeline management program that optimizes maintenance schedules which provides a fine balance between maintaining asset integrity and combating high OPEX for Oil and Gas Operators. Furthermore, they have developed dashboards with specific key performance indicators (KPIs) to run day-to-day pipeline operations and make intelligent-metric decisions with ease for modern day management.

The pipeline management program is comprised of following components:

• Line identification system

• Pipeline maintenance database

• Pipeline maintenance dashboards

Each of these components builds on each other to create an efficient system that allows operators to keep tabs on their pipelines and optimize integrity management operations to reduce OPEX.

"We empower our clients to make faster more faster more informed decisions with continuous  job site visibility, tracking, and secure collaboration"

Overview Challenges and How To Resolve It

The two critical challenges in implementing pipeline management system programs are:

• A large number of wells were developed over a vast geographical area in a very short period of time. The development of infrastructure took priority over sound record keeping and documentation. Also the massive geographical spread of wells makes it difficult conduct survey of the entire infrastructure. 

• A second problem is that the data provided by infrastructure service provider is very siloed and fragmented. Such data cannot be used for decision making for operational management. Standardization and normalization of data is critical to perform analysis and prediction of potential risks and failures. 

Line Identification System

Each pipeline must be properly identified and named in a systematic and uniformed manner from the beginning. The Pipeline Line ID system assigns a unique ID to each pipeline by tagging the starting node and ending node of a pipeline segment and associating it with its field or facility. The unique ID should identify the location, network, field, facility, or route where the pipeline is located. The pipelines should also be segmented in a manner that each segment identifies its starting node, line segment, and the ending node. Lines can be segmented based on turns, when the line goes above ground, when the line feeds into another line, and other cases. The segmentation should be relevant to the operation that is being tracked. For example, pigging is a common integrity management operation where a cylindrical pig is sent through a pipeline to remove solids such as paraffin. One simple framework would be to use Piggable lines for line segmentation. The pig launcher is the starting node and pig receiver is the ending node could be tagged as L-001 and R-001. If this line ultimately feeds into ACME Central Facility, the final pipeline ID format would be: ACME-L-001-R-001. This simple system ensures a unique ID for each pipeline. 


All the data collected by surveys and reviewing old documents for each pipeline must be entered into a pipeline database which includes general information about each line such as location, line length, and line diameter, and oil, water, and gas production values. The operator must determine relevant Key Performance Indicators (KPIs) to be tracked for the maintenance database.

For integrity management operations, KPIs can include date and time of operation, type of equipment used during operation, and results of the operation. For pipeline cleanout operations (such as pigging), the amount and consistency of solids collected would be important KPIs. Diligently tracking the KPIs and updating the database is integral to having an effective maintenance database. As a bare minimum pipeline maintenance database can be quickly built in a spreadsheet software such as Excel and Google Docs.

If the database is routinely updated after each operation with all of the KPIs, reliable analyses can be obtained. Analysts can determine each pipeline’s health and risk level. Integrity management teams can then assign priority levels to the lines and optimize integrity management operations and schedules accordingly. 


Dashboards are an effective way to summarize the information from pipeline maintenance data analysis. Weekly Pipeline Integrity Management dashboards can show which operations took place over the week, their results, points of urgency, and the operations scheduled for the upcoming week. Microsoft Power BI is an easy-to-use software which can be used to create interactive dashboards with vivid infographics.


Integrity management teams can fit this pipeline integrity management system model to any pipeline network. Graphic visualizations help managers quickly understand the data and make instant decisions. Oil and Gas Operators can diligently monitor their operations and successfully predict potential problems in the field through this system. This in turn can keep the OPEX in check and minimize the risk of disastrous accidents on the field.

Weekly Brief

Top 10 Big Data Consulting/Services Companies - 2018

Read Also

Unified Command and Operation Cyber Centers (UCOC): Beyond Remote Operation Centers

Unified Command and Operation Cyber Centers (UCOC): Beyond Remote Operation Centers

RobelloSamuel ,Chief Technical Advisor and Halliburton Fellow (WellEngg.), Halliburton
Resource Adequacy and Grid Flexibility Depend on Analytics for Energy Storage

Resource Adequacy and Grid Flexibility Depend on Analytics for Energy Storage

Sean Halloran, Vice President of Wellsite Technology at Ensign Energy Services
Change Management in Times of COVID

Change Management in Times of COVID

Hilary Ware, Senior Vice President and Chief Human Resources Officer & Amber Rabo, Director of Organizational Effectiveness and Talent Acquisition, Cheniere Energy
The Changing Role of Downstream Oil Companies in the Energy Transition

The Changing Role of Downstream Oil Companies in the Energy Transition

Mitchell Board, Global Head - Future Energies, Puma Energy

"Capturing the Power of the Sun Yields Benefits for Everyone"

Tim Seamans, Head of Commercial Solar at Direct Energy