Augmented Reality Application Development

Fig. 1. Microsoft HoloLens Application Example

Figs. 2-3 Developed applications for the leak investigation procedure and MSA build using Microsoft Guides.

Figs. 2-3 Developed applications for the leak investigation procedure and MSA build using Microsoft Guides.

Description: To develop an Augmented Reality (AR) training application for the Microsoft Ho-loLens focused on leak survey, Meter Set Assembly (MSA), and investigation in the natural gas distribution sector.

Status: NYSEARCH collaborated with an AR training vendor to build and deliver a custom leak investigation application and an MSA procedural guide on the new Microsoft Ho-loLens 2.


Augmented Reality (AR) and Virtual Reality (VR) have swept innovation into various industries in recent years. The technology provides a creative solution to add efficiency in workflow and numerous ways to optimize business operations. AR and VR implementation have proven to increase productivity within workforces across industries.

The potential benefits of this program are: 1) the ability to advance the timescales required to train gas operators, and 2) the potential for bringing the field experience through AR and real-time instruction into the classroom without the need to

wait for or recreate the situation in a live field environment. In addition, as the next generation of gas professionals are becoming more digitally based, the ‘gamification’ of training using state-of-the-art technologies would have fast take-up.


The gas industry is experiencing a tremendous amount of growth. With that growth is a surge in construction activity to expand the gas pipeline network and to replace aging infrastructure. Simultaneously, retirements of personnel have expanded the need to bring in a new, younger workforce that needs a fast-paced and high degree of training and certification by various entities. This results in a need for expanding gas operator training programs and a need to innovate on methods for training.

Today, to rise to a fully rated, high level operator, training must be gained in the classroom and in the field. Rarely are gas industry field conditions able to be simulated without actual participation in field activities. The motivation for a HoloLens test program is to bring 3D reality that depicts various tasks to gas company training programs. This accelerates the ability for the trainee to experience the conditions that are required for that task and to make the process for gaining knowledge more accurate and quicker.

Because Microsoft released the HoloLens device and advertised the ability for users in personal or commercial situations to develop their own applications, NYSEARCH investigated and reported on the possibilities of using HoloLens to address one or more gas training tasks in a hands-free, heads up fashion using Augmented Reality.

Technical Approach

A pilot program began in 2017 to explore and evaluate the HoloLens at each gas utility funding the NYSEARCH project. Along with providing a HoloLens device and training to an individual or multiple personnel who is/are assigned at the funding company, a test plan was generated for all funders to follow and keep personnel focused on a thorough evaluation of the application(s) of interest.

Following the pilot at each funding company, feedback was collected where funders gathered ideas and preferences for gas operations training tasks at the end of testing.

NYSEARCH funders agreed to develop a more robust training experience focusing on a single gas operational task and a consensus was reached to develop a gamified training of gas leak survey and investigation procedure. Furthermore, a specific gas operational procedural task was also developed using the Microsoft 365 Dynamics program.

The specific course developed uses a ‘guided discovery process’ that strongly engages the learner with the importance of their work and its critical nature ensuring the safety of our customers, preservation of assets and reputation of the company. It builds confidence while teaching the standards and process.

The belief is that AR technologies will be instrumental in closing the gap that is responsible for the shortage of skilled workers. Because AR will allow more workers to do high-skill jobs, and improve their performance in this work, industrial productivity will grow. Further, due to the young age and interest with ‘gamification’, the newest generation of gas professionals would likely welcome with little inhibition the latest generation of technology for their training and take great interest in a job and a training program that is embracing state-of-the-art technology.

Program Status

The AR vendor completed the build of the custom leak investigation application and the Microsoft Dynamics 365 Guide of an MSA. A demonstration and training session to introduce the new Microsoft HoloLens 2 device and the developed applications was successfully completed. A next phase is being designed for further customization of the developed applications and to strategize an implementation plan. The strategy for implementation also includes deploying a unique function of the Microsoft HoloLens called Remote Assist where distance learning and field calls can quickly be integrated into utility operations.

For more information Contact NYSEARCH.


For over thirty years, NYSEARCH has worked as a consortium of natural gas Local Distribution Companies (LDCs) who have a common interest and need for research and technology development and demonstration. Today, as part of the Northeast Gas Association (NGA), NYSEARCH manages over (30) projects in various stages of development for LDCs, transmission companies, federal agencies and their manufacturing and commercial partners. NYSEARCH is located in Parsippany, NJ and NGA is headquartered in Needham, Massachusetts.