April 6, 2018 | Sean MacPhedran
With the release of ARCore and ARKit putting more augmented reality content in consumers hands, the media buzz around augmented reality is continuing to increase. On the whole, the AR/VR market is expected to be worth approximately $4.7 billion by 2025, but the large numbers frequently thrown out by analysts don’t mean anything without understanding the direct value of use cases for any particular industry.
In other words, what’s in it for you?
- Integrating Design and Manufacturing: The HoloLens enables designers and manufacturers to collaborate in order to better integrate the design and manufacturing process.
- Training Manufacturing Workers: Using the HoloLens to train key manufacturing workers may result in a four-fold decrease in training time.
- Complex Assembly: The Microsoft HoloLens allows manufacturing companies to increase efficiency in assembly and maintenance/repair operations.
- Servicing and Maintenance: Field use of HoloLens, made possible by its standalone form factor and hard helmet attachment, gives workers hands-free line-of-sight access to data.
- Complex Sales: HoloLens has significant potential in helping customers visualize and tailor complex end-states during the sales process.
- Executive Oversight & Data Visualization: From exploring a future manufacturing facility to diving into live production capacity data, HoloLens can provide the 100,000 foot view, with the option to dive deep.
Microsoft’s Hololens is a mixed-reality product that allows true interaction with 3D models integrated into the physical world. The HoloLens is completely untethered from any device, is able to detect gestures, and includes an on-board voice interface with Cortana. The only similar untethered AR device is Magic Leap, which is not on the market yet.
Other AR devices, like Meta, require a cabled connection to a PC, while smaller HUD devices, like the ill-fated Google Glass, are more akin to the dashboard on a car, rather than an overlay on everything you see through the windshield.
As an augmented reality HMD, users can see 3D objects, dynamic content, and live data superimposed on the world around them. For example, an empty warehouse might be filled with a virtual at-scale aircraft that can be walked around and taken apart, and a user who is accessing vehicle diagnostics from an OBD device might see that data laid over the actual car.
While you can watch the many videos online on the subject, the best way to fully-wrap your head around the impact of these technologies working in tandem is to buy one for your company and spend some time exploring.
1. INTEGRATING DESIGN AND MANUFACTURING
The design process is inextricably linked to manufacturing, and good use of the HoloLens allows designers and manufacturers to collaborate to produce better products and streamline the manufacturing process on complex projects. A best practice in this instance involves creating 3D models of the products that can be altered and changed, so that mistakes and manufacturing issues can be detected before building begins.
For example, ThyssenKrupp, the elevator manufacturing company, is using the Hololens to assist in the design and construction of their stair lifts. The HoloLens allows easy measurement of a customer's staircase. The AI-based application then designs a staircase to suit, and integrates with the manufacturing plant to start the build process. The stair lift can also be modeled in real time and shown to the customer. ThyssenKrupp notes that integrating HoloLens into their process enables them to design and build a stair lift up to four times faster.
Boeing is using the Hololens in the manufacture of its Starliner transport module for the International Space Station. The device lets them literally see if there is clearance and space for a human to get access into areas that might need fixing — a difficult task in the necessarily small environment of spacecraft.
Stryker designs and builds surgery rooms, and the HoloLens allows all stakeholders to collaborate and visualize the room before manufacturing in order to ensure that it will meet the necessary specifications, and ultimately deliver better patient care.
In addition to modeling the surface of Mars, NASA's JPL is planning to use the HoloLens to create holograms of the new Mars rover. This means that potential manufacturing issues can be addressed, the build process can be laid out step-by-step, and engineers will have an easy means to view and manipulate a model of the rover.
HoloLens goes beyond sketches and CAD models by providing sharable, life-size, interactive, and dynamic representations of the complex, making it far easier for multiple stakeholders to see the “same part” of the elephant as they all approach a design challenge from their respective areas of expertise.
2. TRAINING MANUFACTURING WORKERS
The second most effective use case for the Hololens is training staff in the often convoluted processes involved in manufacturing and servicing complex equipment. VR is also a practical tool here. Instead of lengthy classroom demonstrations using video or 2D images, trainees are able to visualize the components from every angle, manipulate them and add or remove components in real time. Having better-trained staff ultimately results in improved products and more efficient processes when dealing with complex manufacturing projects.
Boeing has been using the HoloLens for this purpose, has integrated text guidance for the individual learning tasks, as well as a voice-over to guide the engineers through the process. Using the HoloLens in this way, Boeing predicts that it will reduce training time by 75% per trainee.
Japan Airlines is also using the Hololens to train their staff and has demoed a scalable 3D model of a jet engine that can be taken apart to access and study different elements of the engine. The HoloLens software also enables them to scale the model to life-size, and approach it from different angles to get a full view of how it operates. Japan Airlines is hoping this will help them reduce training costs for its engineers.
3. COMPLEX ASSEMBLY
Assembly can be a lengthy and complex process, with multiple parts that have to go together in a specific order for the finished product to function as it should. In traditional assembly, manuals must be consulted which results in inefficiencies. Using the HoloLens allows manufacturers to have each step overlaid on the task at hand, enabling assembly workers to know exactly which step of the process to do, and how to do it.
In one study that used a simpler heads-up display, GE’s experiments revealed a 34% increase in efficiency on a first-time use case by illustrating next-steps for the user via a heads-up manual.
Volvo is experimenting with HoloLens technology on the assembly line using the processes outlined above, in addition to using it to display key vehicle features to its customers. While elements of augmented reality have been present in manufacturing in the past (e.g. Boeing have used a tablet-based AR tool), the HoloLens allows a much faster and more accurate representation of the steps involved, with the benefit of being a lightweight wearable.
Lockheed Martin has also been exploring the potential of the HoloLens to improve the speed and quality of the manufacturing process. They see the ability to have all the possible information regarding assembly, as well as guided instruction, to be extremely advantageous.
4. SERVICING AND MAINTENANCE OF EQUIPMENT
Servicing and maintenance is often carried out by manufacturers themselves, particularly for products which are complex or have proprietary tools and equipment needed for their repair, and the HoloLens allows engineers to do this in a much more accurate and time efficient manner. Best practices for using the HoloLens for maintenance include the ability to triage requests ahead of the visit, and enable hands-free remote guidance on site.
Augmented reality enables 3D modeling of how to fix or replace parts in a step-by-step manner. Since the HoloLens also has Skype integration, subject matter experts can be contacted quickly and easily for additional support and are able to directly see the task-at-hand. This can radically reduce time and cost for repair, as it allows remote assistance to be both telepresent via video, as well as to “draw” on the real world in-view of the local maintenance operator via their tablet.
Thyssen Krupp is using the Hololens in their service department, and the ability to use the tools outlined above has reduced the average length of a service call by 4x. This is further assisted by the integration with cloud-based sensors in all of their elevators which allow engineers to have easy on-hand technical information to help them focused on servicing and maintenance. The platform will be rolled out to over 24,000 ThyssenKrupp engineers. Since manufacturing contracts often include a service agreement, being able to provide faster, better quality servicing may help improve the interaction between the manufacturing company and the customer.
NOTE: See our Webinar on integrating AI, Voice, and Augmented HMD for a more in-depth exploration.
5. COMPLEX SALES
Complex solutions are often difficult to visualize in their final form. Whether it’s the layout of a room filled with laboratory diagnostic equipment that needs to be linked in-sequence, the design of an industrial HVAC system that will integrate into a building that doesn’t exist yet, or the form factor of next-generation casino slot machines and how they’ll look in a room, HoloLens provides a tool for providing immediate interactive prototypes at-scale.
In the case of the manufacturer who must frequently present complex solutions in meetings and tradeshows, the HoloLens allows products to be virtually present in the room without the need for actual physical materials.
Whether you manufacture steel doors, motorcycle gas tanks, toasters, or propellers for ocean vessels, it’s a lot easier to bring a HoloLens than product samples to the Hilton.
In the case of a proposed custom-design solution, HoloLens can bring the solution to life without requiring a prototype, offering the ability for customers to see the end-state before anything is built, providing immediate feedback and acceptance far ahead in the cycle.
6. EXECUTIVE OVERSIGHT & DATA VISUALIZATION
Blending an HMD AR with access to data can provide unprecedented levels of executive awareness in the context of visual representation of systems, whether those systems are representations of global operations, or schematics of an assembly line overlaid with efficiency data.
To date, public case studies around data visualization for operations management are fairly limited. But as a thought experiment, imagine yourself in your boardroom, with representations of your operations laid out on the table, displaying YTD and live data, with your KPI’s on screens locked to the walls around you. This live data experience, which can be shared by those in the room with you, provides a new way of exploring the information that drives your decisions – both alone – and for the executive team as a whole.
One of the ultimate use cases for HoloLens is the development of a live “War Room” for your business that can be carried anywhere, set up in minutes, and provide access to live feeds of any sort of data.
BONUS APPROACHES: From Lorraine Bardeem, GM Studio Manager, Mixed Reality, Microsoft
“But wait, there’s more!” We are only beginning to scratch the surface of how mixed reality will impact business operations. Here’s two more ideas provided by Microsoft’s Lorraine Bardeem:
DESIGN EFFICIENT AND SAFE WAREHOUSE/FACTORY SPACE LAYOUTS
The proper use of factory and warehouse space is crucial for manufacturing companies to stay competitive. Manufacturing facilities are constantly evolving due to production needs. Each time this takes place there are expensive rounds of design iterations and installations prior to finalizing, HoloLens can reduce the number of these iterations drastically.
The ability to visualize and collaborate on the layout with team members locally or remotely expedites the entire process without disrupting current processes.
CREATING DIGITAL TWINS
The manufacturing industry is also leveraging technology to reduce downtime by investing in concepts such as building Digital Twins of machinery and processes to predict maintenance needs, enable remote support, etc. Digital Twins will provide deeper value by combining machine metadata with spatial mapping as a very powerful tool that brings efficiency, cost savings, and safety.
If you’d like to discuss these topics further, SMITH can help.
Whether you’re exploring an evolution in your approach to selling by moving to pure ecommerce or a hybrid approach with a tool like Mirakl, looking for an edge in operations optimization with augmented reality, or ready to deploy intelligent replenishment for your customers, we can help.