AR in maintenance

Co-author: Noel Shi , Weite Dai, Jieming Hu, Darius Rosa-Deham 

1. Introduction

Hiring a technician to fix something or arranging a service can be expensive and time-consuming, especially under some unexpected circumstances. This AR maintenance application will guide users to conduct some easy day-to-day maintenance tasks that can be fixed within a minute and at a low cost.

2. Description of the application

The purpose of this application is to help passenger car owners to do some basic maintenance. Car owners can use this application as a guide to complete some maintenance that can be done without the help of a technician. 

Different kinds of maintenance are needed for passenger cars from time to time. However, it is also difficult to choose a good car maintenance service provider, because the service quality varies (Brito et al. 2007). In fact, many of the maintenance procedures and tasks can be completed easily by the car owners (Turo 2022). The so-called basic service, which include changing the engine oil or the filters, are some of the most popular and well-known easy maintenance. But not all car owners are familiar with the structure of a car and, therefore, fear of conducting the maintenance. 

Another important scenario that requires self-maintenance is when something unexpected happen. The most common situation is tyre puncture on the road. In this case, it is normally difficult for people to look for help. Although a good driver should always be prepared for anything that might happen on the road, it is a tough procedure for people who met this the first time.

In the above cases, car owner might be able to find information and instructions from the car manual or online websites, but there is lack of virtual or augmented materials. This information is often just some simple steps to follow – normally with text, which is very user-unfriendly. People without any experience will not know how those steps correspond to reality and which parts is what. Therefore, this AR application is an excellent approach to guide inexperienced car owners to handle those maintenance issue.

From another perspective, the use of internet technology is expanding quickly and is practically becoming a necessity. The idea of employing technology in daily life is not new, but because of technological breakthroughs, technology is now influencing practically every element of a person's life. This of course also includes car-related aspects.

When it comes to the connection between cars and smartphones, people first think of the application of mobile phone navigation in this industry. In fact, with the help of other technologies, smartphones can do much more in this area. Aloul (et al. 2015) investigated a technology that use smartphones to quickly and automatically identify and report car accidents. With Augmented Reality, it may bring more possibilities. Bhorkar (2017) states that it may be further enhanced to become realistic and user-friendly by merging navigation with augmented reality. Smartphone with AR application using for car maintenance training has also been reported (Kim & Moon 2013). 

Therefore, this application will be designed to implement AR technology with mobile phones in car maintenance. User can choose specific car model to match the virtual application instruction with a real car model, so it will give a more user-friendly human interaction with the device. By offering a new affordance within the augmented reality environment, this application offers multiple maintenance options to meet various requirement in different situation which could be tough without AR technology.

3. Description of the interface solution

Software, hardware, and usability must all be examined in deciding the requirements of this application. To start, the software used must have strong visual capabilities that can illustrate the physical car interactions in an easy, consistent format as well as simple interactions which give the user control over the instructions. Hardware needs to be portable and accessible so it can be used whenever needed in case of an emergency. As for usability, there needs to be real life interaction capability, so the maintenance issue can be worked on during the use. 

At the initial stage, two interface technologies have been considered for the implementation of this application - Virtual Reality (VR) and Augmented Reality (AR). 

Although VR is a phenomenal technology, it does not suit the requirements of this application. VR is extremely visually and physically stimulating, especially with its human-like interactions that involve the entire body. But they are not essential for this application, as the main interaction is directly with the car, only simple application interaction is needed. 

On top of this, the drawbacks of using VR are detrimental. These drawbacks include inability to interact with the real world, cost of device, and portability (Meta, 2020; Rubin & Grey, 2020). Inability to interact with the real world is due to the headset seizing vision making the user unable to see the car, it also causes safety risks, as cars are usually outside or in garages which may have hazards. Device accessibility is also a major issue, as VR devices are far from affordable to most, especially when considering the applications occasional use. Another issue is portability, making it unusable in an emergency. To summaries, VR technology is unapplicable for this project due to its conflict with the requirements.

AR is a better fit for this project as it meets the application requirements, and its disadvantages are merely hinderances. Starting with the advantages, AR is portable, can involve real world interaction, effective at simple visual communication, capable of simple and effective interaction (Krost, 2019). The portability of AR devices comes from its ability to be used with any smartphone with a camera, making it highly accessible, since more than 86% of Australians have a smartphone (Granwal 2021). 

Another main component of this application is physical car interaction, since AR takes place on a separated screen, real world interaction is available during the use. The level of visual communication within AR is applicable for this project, as it can produce a clear and accurate image of necessary car parts and interactions. As for the in-app interactions, only simple buttons are needed to move through steps, making canvas buttons perfect. 

Disadvantages for AR are not extreme: Setting up the device to be continuously pointed at the target can be slightly difficult, but always possible with something to hold up the device (e.g. tripods). Smart phone screen size may also cause issues to those with poor eyesight. And lastly a device with a camera is a necessity for application use. Overall, despite the slight issues that may arise when using AR with this application, it satisfies all application requirements. 

4. Interaction Design

The technology selected for this application - AR, can well fulfill the user requirements that the traditional maintenance guidance do not have (e.g. video, paper manual). Through a unique user experience, intuitive and dynamic presentations, users can enjoy the process of maintaining their cars themselves. Below are an example of the storyboard.

Interactions that have been designed for this application include:

· Select type of maintenance. On the main screen of the application, there are several maintenance options for users to choose from, including tire, engine oil, coolant, and other services. On selection of the service, 3D models will zoom out and a closer view of the target item will be shown on the screen.

Although this interaction is triggered by clicking on the screen, the strength of using AR still supports a better user experience in this use case by providing a zoom-out view of the target object. A more intuitive action is provided by AR to help user to understand where the maintenance should be conducted.

· Switch from steps of maintenance. After selection of the maintenance, there will be many steps of it. Users can switch steps in the service. For example, when the user goes to the next step, he can return to the previous step as well. When the user returns to the previous step, the target 3D model will be presented again.

The strength of using AR to support this is that a dynamic movement in the real world can trigger interaction with the target item on the canvas. A unique experience is offered, instead of just reading text or image information from other resources, users can interact with the models to better understand what they should do for the maintenance service.

· Check recommendations for further maintenance. For each maintenance service, there will be a button to alert the user with a recommendation. The recommendations include the maintenance cycle and issues that need to be looked after by a technician. On press of the target item on the screen, general recommendations will be shown, and the user can hide the information by pressing the button again.

The portability of AR allows users to easily access this application. According to different maintenance services, AR can combine real-world interaction of targeting the item with useful manual information. This is a good implementation of visualization sensor data over time and space.

· Drag tools to target object. After selection of the maintenance, except for a closer view of the target item, tools will appear around the target item. Users can interact with this tool by dragging it around the screen and if it is dropped on the target item. An animated movement will appear, telling the user how to conduct the maintenance. For example, user can drag a wrench to the tire, then screw will be detached which is shown as an animation.

5. Initial technical Development 

This AR maintenance application is developed by using Unity (version 2022.1.7f1), mainly targeted at Android phone devices and/or webcam. In the initial stage of the technical development, this AR application will be developed on Vuforia engine (by using image target). At a later stage, the group will assess the feasibility of using VR-core to implement this application. 

By using this application, users will be able to interact with the application by pressing the buttons located on the screen of the mobile phone. Users will be able to select the maintenance job, once selected, a detailed step-by-step guide will be shown on the screen to guide users to complete a maintenance issue. Buttons will be shown on the world canvas and the guide will be shown in animations. 

The group will assess the feasibility of using virtual buttons and include videos or audio in the application in the next stage. The potential of including product marketing (such as selling engine oil) or making appointments for a service will be considered as well. 

 6. Initial 3D Models 

 Some of the 3D models that will be/have been used in this project include:

• A tyre from 3D car model

Main purpose: 

     - To demonstrate a prototype of how to use AR technology to teach or guide people to fix some items;

     - To show an example of how to use the AR maintenance application to replace a flat tyre.

Source:  https://assetstore.unity.com/packages/3d/vehicles/land/realistic-mobile-car-demo-173467

Author: Pro 3D models

• Audi engine model

Main purpose: 

     - To demonstrate a prototype of how to use AR technology to teach or guide people to fix some items;

     - To showcase an example of changing engine oil and adding coolant for a car.

Source: https://www.cgtrader.com/free-3d-models/vehicle/vehicle-part/audi-tfsi-engine-scanned

Author: F4LCO

Besides of the above internet-sourced models, the group will also develop a few simple models for illustration purposes. For example, nuts (by using cylinders and cubes 3D object).

 7. Conclusion

By using AR technology, this application will allow users to conduct some easy maintenance without a high cost and without taking much time. Traditionally, when a car requires service or under some unexpected circumstances, car owners will normally have to read through the paper manuals or to search for a troubleshooting from the internet, which will normally be in written format. This AR maintenance application will allow users to select different types of maintenance. Once selected, a detailed step-by-step guide will be shown on the screen, AR animation will be displayed by highlighting each component to the specific issue, in order to assist users to complete the maintenance task.

References

Aloul, F, Zualkernan, I, Abu-Salma, R, Al-Ali, H & Al-Merri, M 2015, ‘iBump: Smartphone application to detect car accidents’, Computers & Electrical Engineering, vol. 43, pp. 66-75.  

Audi Engine model. https://www.cgtrader.com/free-3d-models/vehicle/vehicle-part/audi-tfsi-engine-sc...  

Bhorkar, G 2017, ‘A Survey of Augmented Reality Navigation’, Cornell University, viewed 9 October 2022, <https: arxiv.org="" abs="" 1708.05006="">. </https:> 

Brito, EPZ, Aguilar, RLB & Brito, LAL 2007, ‘Customer choice of a car maintenance service provider: A model to identify the service attributes that determine choice’, International Journal of Operations & Production Management, vol. 27, no. 5, pp. 461-481.  

Car (tyre) model. https://assetstore.unity.com/packages/3d/vehicles/land/realistic-mobile-car-demo...  

Granwal, L 2021, Smartphone penetration rate as share of the population in Australia in 2017, Statista, viewed 7 October 2022, <https: www.statista.com="" statistics="" 321477="" smartphone-user-penetration-in-australia="">. </https:> 

Kim, YD & Moon, IY 2013, ‘E-Training Content Delivery Networking System for Augmented Reality Car Maintenance Training Application’, International Journal of Multimedia and Ubiquitous Engineering, vol. 8, no. 2, pp. 69-80.  

Krost, C. (2019) 3 capabilities of augmented reality, Chelsea Krost. Available at: https://chelseakrost.com/3-capabilities-of-augmented-reality/ (Accessed: October 7, 2022).  

Meta (2020) Meta quest 2: Advanced all-in-one VR headset, Meta. Meta. Available at: https://www.meta.com/au/quest/products/quest-2/ (Accessed: October 7, 2022).  

Rubin, P. and Grey, J. (2020) <span class="NormalTextRun SCXW191127202 BCX4" <the="" wired="" guide="" to="" virtual="" reality<="" span="">>span class="NormalTextRun SCXW191127202 BCX4">, Wired. Wired. Available at: </span>https://www.wired.com/story/wired-guide-to-virtual-reality/ (Accessed: October 7, 2022). 

Turo 2022, 7 DIY car maintenance projects you can do at home, viewed 8 October 2022, <https://turo.com/blog/gearheads/7-diy-car-maintenance-projects-you-can-do-at-hom >.