Assignment 5 Report - AR in maintenance

2 years ago by noelshi

Share this post:

Share on Bluesky Share on Twitter Share on Facebook

1. Introduction

Hiring a technician to fix something or arranging a service can be expensive and time-consuming, especially under some unexpected or emergency circumstances. This AR maintenance application will guide users to conduct some easy day-to-day maintenance tasks that can be fixed within a minute and at a low cost.

The main update of this version of the application from the design phase (Assignment 4) is that all detailed steps of the maintenance process have been filled in and updated with interactions and animations.

2. Technical Development

In order to demonstrate how AR can be used in maintenance, this project has been developed by using Unity (version 2022.1.7f1), targeting at Android mobile devices and/or webcam.

Although AR core is considered to be a more suitable technology for this application, so that users will be able to use the phone camera to point to the real object and the 3D models will be shown on the real object to guide users on the process. For simplicity and illustration purpose, Vuforia technology has been adopted for this project. By pointing to a pre-set image target, UI buttons will be shown on the screen for users to select different types of maintenance and to move along with the steps.

2.1 Changing tyres

Once the users have pointed their camera to the correct component – a flat tyre (image target), a step-by-step guide will be shown to guide and teach users how to replace a flat tyre. Users will be able to choose different types of maintenance, move to the next step, move to the previous step or return to the main menu by pressing the UI buttons located on the screen. If the user select a maintenance type but points to the wrong object, an error message will be shown on the screen. Users will also be shown with a recommendation after the maintenance, for example, to find a nearest tyre shop.

Key scripts that have been utilised for this function mainly include the built-in scripts for UI buttons (On Click function). Once a button is clicked, the components that are irrelevant will be deactivated and those relevant will be activated and shown on the screen. Animations have also been included in each step to guide the user to complete the maintenance. For example, arrows point to the right components and locations.

2.2 Changing engine oil

When a user points the camera to the target object (the engine), a window with options will appear and the user can select the changing engine oil service. Steps of the service will be triggered by clicking the next or back buttons. At the last step of each service, a service button will appear, leading the user to a service website.

Key functions for this section include built-in scripts for UI buttons, e.g. Onclick function and setActive function. Once a section is selected, then animation for the related object will be activated and the UI components will be activated as well. For example, when changing engine oil button is clicked, components from previous scene will be deactivated and Menu, title, next-button, and back-button of current scene will be shown. The animation of objects will be activated too. A URL script is set as the component of an UI button to open the URL written in the script. It will be called once the user clicks the button.

2.3 Adding coolant

When the user points the camera to the coolant tank, a window with options will appear and the user can select the adding coolant service, which is mostly the same as the previous type of maintenance. Steps of the service will be triggered by clicking the next or back button. At the last step, a service button will appear, leading the user to a service website.

Key functions for this section include built-in scripts for UI buttons. For example, when the next button is clicked, components from the previous step will be deactivated and Menu, objects and other buttons will be activated. And specifically, when it is the last step, every 3D object on the scene will be deactivated and a service button will appear. An URL script is set as the component of the service button to open the URL written in the script. It will be called once the user clicks the button.

3. 3D Content

Car tyres from 3D car model	Author: Pro 3D models Source: https://assetstore.unity.com/packages/3d/vehicles/land/realistic-mobile-car-demo-173467 Purpose of use: to demonstrate the maintenance of how to change a flat tyre and how the AR technology can be used to guide people to complete a maintenance task
Audi engine model	Author: F4LCO Source: https://www.cgtrader.com/free-3d-models/vehicle/vehicle-part/audi-tfsi-engine-scanned Purpose of use: to demonstrate and guide users to complete a maintenance task (such as changing engine oil or adding coolant), to demonstrate how AR technology can be used in maintenance.
Jack screw	Author: mcbuzz Source: https://www.turbosquid.com/3d-models/jack-screw-max-free/1019854 Purpose of use: to demonstrate part of the steps to complete the maintenance task for replacing a flat tyre
Arrow	Author: Peter Iliev Source: https://www.turbosquid.com/3d-models/free-arrow-3d-model/582495 Purpose of use: to guide users and point to the specific components of the maintenance steps
Nuts (screws) model	Author: Noel Shi Purpose of use: to represent a nuts or screws that need to be installed or removed as part of the process for replacing a flat tyre
Tank Cap	Author: reramil Source: https://sketchfab.com/3d-models/sketchfab-lenticular-bottle-cap-aad7aa46a7c94f18b94c0f839ca79afb Purpose of use: Used as the lip of tanks like oil tank and coolant tank. To perform the movement of removing and adding the lids.
Oil Bottle	Author: Kwon_Hyuk Source: https://sketchfab.com/3d-models/20l-oil-bottle-86bedfb7ff3e4682bebbdf38ebe933b4 Purpose of use: Used to present the engine oil and perform the movement of filling the engine oil into the engine. It makes the guidance clearer with this object.
Funnel	Author: Dape Source: https://3dexport.com/free-3dmodel-funnel-369174.htm Purpose of use: Used to present the funnel that is usually needed when adding liquid into tanks. To represent the movement of placing a funnel at the top of a tank.
Coolant Bottle	Author: Francesco Coldesina Source: https://sketchfab.com/3d-models/bottle-a-a25540fc03ba400faf012d4f68b7ba41 Purpose of use: Used to present the coolant bottle and represent the movement of pouring coolant to the coolant tank. It makes the guidance clearer with this object.

4. Usability Testing

4.1 Design and Plan

In order to find out the effectiveness and functionality of the application, both qualitative and quantitative methods have been used for the usability testing. A combination of observation and interviews have been used to obtain usability testing data for the purpose of quantitative analysis.

Users (participants) will be instructed and guided to start the testing, while the tester will record observation notes throughout the process, including time taken to complete each task, the mistake that users have made and so on. After that, a semi-structured interview will be conducted to collect users' feedback and recommendations for improvement.

For the purpose of the usability testing, the details of requirements are listed in Table 1.

In order to test the application, a list of tasks have been designed and shown in Table 2.

Usability task matrix is shown in Table 3 to match the task to the requirements.

The questionnaires are developed in five-level Likert scale and open-ended question. Example of the observation note sheet and questionnaires are shown in Table 4 and Table 5.

4.2 Recruitment

The participants were recruited from users with smart phone experiences. There was no bias when the participants were recruited.

4.3 Protocol of the testing

Testing of the application was done face-to-face within a quiet testing room at the University of Tasmania. Participants were given the following information about the application:

“This is an Augmented Reality application for general car maintenance. So far, the application has tutorials for changing a tyre, changing engine oil and adding coolant. Desired use for this application is for at home or emergency car maintenance that can be done without a mechanic.”

Followed by instructions on application operation:

Select maintenance type
Point camera at target image
Follow instruction
Move to next instruction
Repeat until all instructions are completed
Move backwards through instructions if needed

And finally, the following instructions on the interactive features:

UI selection of maintenance type
UI selection of instruction stage buttons

Testing is closely observed, and usability information was recorded on the experimenters note sheet.

After application testing was completed, they were asked to complete a survey that records their gender, knowledge level of Augmented Reality and their experience level with car maintenance.

4.4 Report of the findings

After the testing were completed, the raw data of result was combined to the Table 7 and Table 8 below. The original data can be found in Appendix A.

4.5 Analyses of the findings

It can be seen from table 7 that all participants were able to successfully complete all tasks. While 73% of tasks completed without help and 27% of tasks were completed with help. During the test, the participants had difficulty on 27% of tasks using specific aspect of the application. Those difficulties mostly happened on lack of interface indicators/navigation, including:

Switching maintenance type: No exit button or indicator telling user to click the menu button to quit the current maintenance type and switch to another one.
Hard to see instruction texts: Instruction texts blended into background so users can’t see the texts clearly.
App launch screen: when users launched the app first time, it only shows a camera screen but nothing else to indicate users when to start take action.
Camera stability: Holding the camera up to the image target is difficult while multitasking
Further action/help needed: After maintenance completion, user want to have more help such as the spare tyre safety issue and if problem occurs.

According to the table 8, 60% rating this application is easy to use and 100% rating it has clear instructions.

Overall, all participants have completed the testing task. There were some difficulties on specific aspect which need to be improved in the next stage.

5. Addressing the Results of the Usability Testing

From the findings of last stage, most of the feedbacks from users were focusing to the interface indicator or navigation.

5.1 Result Analysis

Firstly, errors/difficulties based on the themes and other elements are summarised into Table 9.

5.2 Identified issues

Issue 1: [during operation steps screen] – No extra exit button on the screen, or indicator to guide user to click main menu for quit.

Issue 2: [on changing tyre step 2] - The description on the bottom is hard to read as the texts blended into background.

Issue 3: [on app home page] - After the app launched, the screen is empty and no indicator so users don’t know when to start take action

Issue 4: [completion screen] – No further help if problem occurs (e.g. the spare tyre is space-saver size)

5.3 Updates to the application

The project group have updated the application based on the feedback collected from the usability test.

For issue 4 - no further recommendation is provided, the project group have implemented improvement to add new action buttons, so users can get more help after the maintenance is completed. For example, once users have completed the whole process to replace the flat tyre, a “find a tyre shop” button will be shown and once clicked, users will be re-direct to google map to assist users to find the nearest tyre shop. The changes are shown as below:

For other type of maintenance, in case user encounter problem and need to seek further help, the application has also provided a notice and a button to re-direct users to book a service with a technician. Once users have clicked on the button, a webpage will be opened and users will be able to book a service online.

Reference

SurveyPlanet (no date) SurveyPlanet. Available at: https://surveyplanet.com/ (Accessed: October 23, 2022).

Unity Documentation (n.d.). Application: OpenURL. Accessed on 27 October 2022. <https://docs.unity3d.com/ScriptReference/Application.OpenURL.html>

<https: 3dexport.com="" free-3dmodel-funnel-369174.htm=""></https:>

Dape, ‘Funnel’, https://3dexport.com/free-3dmodel-funnel-369174.htm

F4LCO, ‘Audi TFSI engine - scanned Free 3D model’, https://www.cgtrader.com/free-3d-models/vehicle/vehicle-part/audi-tfsi-engine-sc...

Francesco Coldesina, ‘Bottle A’, https://sketchfab.com/3d-models/bottle-a-a25540fc03ba400faf012d4f68b7ba41<https: sketchfab.com="" 3d-models="" bottle-a-a25540fc03ba400faf012d4f68b7ba41="">.</https:>

Kwon_Hyuk, ‘20L OIL bottle’, https://sketchfab.com/3d-models/20l-oil-bottle-86bedfb7ff3e4682bebbdf38ebe933b4.

Mcbuzz, ‘Jack screw’, https://www.turbosquid.com/3d-models/jack-screw-max-free/1019854<https: www.turbosquid.com="" 3d-models="" jack-screw-max-free="" 1019854="">.</https:>

Peter Iliev, ‘3D Arrows’, https://www.turbosquid.com/3d-models/free-arrow-3d-model/582495 <https: www.turbosquid.com="" 3d-models="" free-arrow-3d-model="" 582495="">.</https:>

Pro 3D models, ‘Realistic Mobile Car Demo’, https://assetstore.unity.com/packages/3d/vehicles/land/realistic-mobile-car-demo....

Reramil, ‘Sketchfab Lenticular bottle cap’, https://sketchfab.com/3d-models/sketchfab-lenticular-bottle-cap-aad7aa46a7c94f18... <https: sketchfab.com="" 3d-models="" sketchfab-lenticular-bottle-cap-aad7aa46a7c94f18b94c0f839ca79afb="">.</https:>