ECCE App Challenge 2019 – How We Made an AR App in 5 Days
The App Challenge project that Jaydeep discusses below can be viewed from his team’s profile on the ECCE GitHub website: https://esricanada-ce.github.io/appchallenge/2019/teams/uw/SpectaculAR/
ECCE App Challenge 2019 – How We Made an AR App in 5 days
This post is co-written by my teammate Manpreet Chahal.
Since 2015, the Esri Canada Centres of Excellence (ECCE) has been hosting an annual App Challenge in Canada for participating schools who are designated as an Esri Canada GIS Centre of Excellence. Each school is allowed roughly up to 4 teams with 4 members each. Year after year, that number of participating teams has been rising from 10 teams in 2015, to 24 teams in 2019. It is no easy feat by ECCE to convince students to take a week out of their year to participate in the App Challenge. Some students are drawn in by the possibility of winning the grand prize of getting an all expense paid trip to the Esri User Conference in San Diego, California. While some are drawn in by the sheer challenge to push their geospatial creativity to their limits and create a product they can be proud of. For us, I’d like to think we fall into the latter category of GIS students where we are comfortable in committing 12 hour work sessions in the GIS labs just to make sure that our scripts work perfectly, or that we can add one more feature onto our apps.
Both Manpreet and I had participated in the ECCE App Challenge last year while being in separate teams at different schools. I was in my first year of my Masters at the University of Waterloo (UWaterloo), while Manpreet was completing his 4th year of undergrad at McMaster University. My app was about visualizing and analysing the economic and environmental impact of sea level rise in the City of Vancouver in 3D, while Manpreet’s app was for parents to find the most optimally located day-care centres for their child in the City of Toronto. Both of us had made excellent apps, but unfortunately they weren’t enough to be the winner for the 2018 cohort.
This year, where I’m now in my second year of my Masters, and Manpreet joined UWaterloo for his Masters as well, we banded together to create an app that none of the ECCE judges would have seen so far. We knew we had to create something spectacular. Using new tech such as 3D GIS, or focusing on a niche problem wasn’t enough. We had to make something that had not been attempted before, thus we chose to focus all of our efforts towards solving a sustainable development problem using Augmented Reality (AR).
The ECCE App Challenge does not happen at the same time for all participants. The one week opportunity to create an app for submission is staggered for all the schools into multiple weeks, where some schools get to work on their submission at a different week than other schools. UWaterloo was the first school to participate this year, and our week was assigned to our Reading Week of February 15th-22nd. Although we had one week to make an app submission for the 2019 theme of “Sustainable Development”, in reality, we had a lot less time than that because we individually had other commitments already made during that reading week. Realistically, we only had 5 days to make our app. The following is a summary of the challenges we faced, the progress we made over each day, and the final app we made. We hope you like our app just as much as we enjoyed making it.
The Challenge
Although ECCE provides each team with 7 days to complete the App Challenge, our group only had about five days to complete our app because of time constraints each of us were facing. Our team consisted of four group members and below were each of the challenges we faced:
- Jaydeep had to present a short paper he wrote for the Spatial Knowledge and Information Conference in Banff. While the app was due on Friday Feb 22, he had to leave Waterloo on Wednesday night to catch his flight for the next morning to Calgary.
- Manpreet was a Teaching Assistant to an Undergraduate course called GP487: Management Issues in GIS Systems. He had assignments to mark that were due to be given back to the students on the week following the app challenge.
- Juan had to submit a paper for a conference that was due at the same night as the ECCE app challenge.
- Fraser had to work on his Masters thesis to give to his supervisor during the earlier days of the App Challenge week and was available for the last four days.
In the short time of almost five days, and through our own time constraints, we were able to persevere and submit an AR app about sustainable development that we are extremely proud of making.
Day 1 – Exploring the Theme
On Day 1, we did our initial research on the topic of Sustainable Urban Development/Sustainable Urban Communities that was chosen by ECCE for the App Challenge. We decided to focus our research on the United Nations’ Sustainable Development Goals (SDGs). We began to look at the SDGs, which we found really related to addressing the topic at hand. In addition, we also did research on a new sustainable urban development initiative known as Sidewalk Toronto. We determined that this initiative would be a great example to use to showcase our app. Sidewalk Toronto is a joint-initiative between Sidewalk Labs, Alphabet Inc.’s urban innovation organization, and Waterfront Toronto, a tri-government agency in downtown Toronto. The goal of Sidewalk Toronto is to create a new kind of mixed-use, complete community on Toronto’s Eastern Waterfront, beginning with the creation of Quayside. We wanted to give a focus on user feedback for our app. We found out that Sidewalk Labs provides more face-to-face meetings and a few online options for citizens to participate in the planning of the development project. Hence, we wanted our app to give users an option to provide feedback on what type of developments would be happening in the Sidewalk Toronto project.
In terms of the AR technology, we tried to build an app using Google’s ARCore examples. We managed to pick up a simple example from their website and configure it to include 3D data that we wanted. We included an open source 3D model of the CN Tower in Toronto, Canada into a basic example of Google’s ARCore and created the demo in the video below.
3D CN Tower AR Demo on Android’s Google ARCore
Day 2 – Exploring AR Technology Options
On Day 2, we tried to build more onto the Google ARCore’s demo code but quickly realized that none of us were experts in native Android development. We were thinking of either having an extremely basic example of AR and sustainable development, or we find a compatible solution that brings together AR, a mobile platform, and ArcGIS libraries. We started to research into what Esri had showcased in the past related to AR and found the perfect example. Hence we directly asked the organizers of the ECCE App Challenge that we wanted to build something like the demo shown in the video below, and if we could get access to their code, tutorial, or any documentation that would allow us to build an AR app using ArcGIS compatible software.
Day 3 – Working with ArcGIS Runtime SDK AR/VR Beta Code
On Day 3, we got extremely lucky because the organizers of the ECCE App Challenge were generous enough to give us access to beta-version of their ArcGIS Runtime SDK which was being developed to include functionality for AR/VR apps. Although the tutorial app that the beta-version code came with was extremely basic, it was running on top of ArcGIS Runtime SDK which I had experience of from building my last year’s ECCE App Challenge submission completely from ArcGIS API for Javascript. From this point onward, we went full speed into developing the other features of the app that would make it into a complete package rather than only having an AR Component.
The video below is what the tutorial app of the ArcGIS Runtime SDK AR/VR beta-version looked like. The only thing that the app did was open a scene somewhere in San Diego, California, USA, but it was still amazing to see that it was running on ArcGIS code that I was familiar with, and it was running on top of the Google’s ARCore library (meaning that it calculates the position of the camera based on the telemetry data of the cell phone as well as the main camera of the phone for extremely accurate positionality of the objects in AR).
ArcGIS Runtime SDK AR/VR Demo App for San Diego Buildings
Day 4 – Develop the Core App Features
Knowing that we were able to get the AR part of the app to work exactly as we wanted, the next step was to create a mock-up of what the Sidewalk Toronto could look like, and if we can bring it into our app in an AR Scene. Additionally, we developed some other key features of our app:
- App navigation
- Contact information
- User account information
- Project page
- Project Overview
- Project Dashboard (ArcGIS Online Operations Dashboard)
- Project User Feedback
- Standard 3D Mode (ArcGIS Online Web App)
- AR Mode (ArcGIS Runtime SDK AR/VR)
- Host all the 3D data on ArcGIS Online
Day 5 – Polishing the Product & Pitch/Demo Videos
On Day 5, we looked at polishing our final product. We made sure that the AR functionality was working on our app. We tested the app on multiple mobile devices to make sure that the AR display was functioning correctly. In addition, we made sure that our Esri 3D web map showing the same display as our AR display was also functioning correctly. We worked more on the design of the app with the app logo, start screen, menu options, dashboard, and feedback form. The dashboard was designed in a way that the user would be able to see where in the City of Toronto, all app users were providing feedback on our app. It would also provide statistics on the number of users in each group that were participating in providing feedback on our app. The feedback form was designed in a way that it provided multiple choice questions to users to rank their preference of the types of development that would happen in Sidewalk Toronto in relation to addressing sustainable Transportation, Recreation, and Public/Residential Development.
Lastly and most importantly, we also created our pitch video, showcasing the capabilities of AR in our app. Our pitch video was designed as a sales pitch of our app being useful to various municipalities, companies, and organizations. We also created a demo video, which went through greater detail to show how a user would use our app from opening it on their mobile device, to seeing the different options on the start screen, menu, dashboard, and feedback form, to using the Esri 3D technology for displaying Sidewalk Toronto in a 3D space, and to using the AR component for visualizing Sidewalk Toronto in a real-time physical space.
Conclusion
Every year that we have participated in the app challenge we keep raising the bar for ourselves in terms of what we can accomplish. This year we really believe we went above and beyond to create an app that is an AR mobile app, that includes three separate ArcGIS products working together in perfect unison, as well as tackling an important sustainable development issue facing Toronto, and other cities across Canada that are pursuing new developments.
We hope that you enjoy our app just as much as we enjoyed making it!
Located below are links to our app submission for the ECCE 2019 App Challenge.
Team: SpectaculAR
Pitch Video: https://www.youtube.com/watch?v=2KRp0xgTSDA
Demo Video: https://www.youtube.com/watch?v=BTxz8_ssLAE
Sincerely,
Jaydeep Mistry & Manpreet Chahal