To achieve a final breakthrough, augmented reality needs smart content and genuine value added applications for consumers and professionals. Key enablers for this are 3D models of urban environments and buildings, big data and artificial intelligence.
Augmented Reality or AR broke through among consumers with the Pokemon Go application in the summer of 2016. It has been possible already for some time to present various services and tourist information via mobile phone AR visualisation. AR hit applications in the last Christmas season included Stockmann’s augmented Christmas window and a virtual Santa Claus roaming the aisles of the department store.
Augmented reality is expected to become part of our everyday lives in the coming years, as data glasses become lighter. However, it will also require sufficient motivation to use data glasses – smart content and context awareness, which enable useful information to be displayed to the user before she even asks for it.
A key source of content for the future mixed reality applications will be 3D models of cities and buildings, and the related big data which AI will filter to meet user needs. In this blog, I will focus on urban AR applications, an area in which VTT has been active for some time. We are also developing AR applications for other fields such as manufacturing industry.
Urban applications
In city planning, AR enables project stakeholders to see how a designed building will look in its actual location. VTT brought AR into the decision-making process at the beginning of the decade. In the picture/video below, Helsinki City Planning Committee is using AR tablets to evaluate plans for the Clarion hotel in 2012. AR has also been used to present building plans to citizens and residents, who can then review plans from their own homes, for instance.
Less conventionally, we have have enabled historical photographs augmented in urban environments, together with organisations such as the Helsinki and Lahti City Museums. The video below shows how the app works on site. In the future, this could work on the basis, say, user created own content (photographs, videos and comments), which would enable whole new kinds of applications when combined with artificial intelligence.
New opportunities for augmented reality are provided by the 3D city models that for example the City of Helsinki just released as open data in November 2017. Besides visualisation, 3D city models can be used as content repositories for AR applications and for the accurate camera tracking required by AR.
New forms of Mixed Reality (MR) presentations are being created, as generalisations of mobile outdoor AR. An example of this is augmentation of 360 videos recorded using autonomous drones. These augmented videos can be broadcast in real time and immersively as ‘remote experiences’ for multiple users at events such as planning meetings.
Inside buildings
BIM-based design enables the 3D presentation of building models and the related metadata. BIMs are today created for most new construction sites. However, they are seldom utilized after the design phase, and printed 2D drawings are still mostly used as reference at construction sites.
VTT was the first in the world to demonstrate AR methods on a construction site, already in 2010. Currently VTT, alongside Fira Oy and Sweco Rakennetekniikka Oy, is implementing AR and BIM-based planning in renovation; aims include reducing the duration of plumbing renovations from months to weeks. These methods are being piloted at a real renovation site in Hernesaari in Helsinki.
Big data on buildings comes from various data sources connected to the Internet, such as IoT sensors, process monitoring and simulation systems, automation systems and digital maintenance manuals, etc. When linked to a 3D model of a building, these form a so-called digital twin, in which the data sources have coordinates ready for use in AR applications.
Besides just visualisation, AR enables interaction with and reporting to back-end systems, by clicking at the building’s devices on the touchscreen, or by pointing at them in the data glasses’ view. Applications include building maintenance and monitoring of industrial processes. VTT is currently implementing an AR maintenance system integrated with Granlund Manager facility management system, at VTT Espoo in Otaniemi. As an example of industrial applications, the image below shows an AR view of equipment and processes on a factory shop floor.
Augmented reality is already being used in consumer entertainment and advertising, and – in the professional arena – for the visualisation of building models for stakeholders. Next-generation professional applications will include lifecycle applications for industry and buildings, such as servicing and maintenance. In the near future, AI and big data will facilitate a new level of consumer appications in urban environments. With strong multidisciplinary research expertise, VTT has excellent opportunities to be a trendsetter in this development process.
