Big Data and the Power of ArcGIS Pro
As a MSc student at the University of Victoria, the research I am conducting feeds my passions: GIS, remote sensing, and machine learning/data mining. My research involves analyzing automatic identification systems (AIS) vector data. AIS is an automated and autonomous system that allows the transmission of vessel position data and vessel identification information via VHF to nearby vessels, and terrestrial or satellite receivers. Positional AIS messages contain a set of coordinates, are transmitted between every 2 seconds to every 6 minutes, and can be represented spatially as points or trajectories. With thousands of marine vessels sailing in our oceans this results in millions of data points per day. Storing and managing this multitude of data is difficult and requires powerful databases. Fortunately, ArcGIS Pro has functionality that lends itself to working with these large data sets when compared to ArcMap or other GIS software.
Figure 1 contains AIS positional messages, represented as points, for the month of August, 2019. These data were collected using a terrestrial receiver on the eastern point of Saturna Island, near the Southern Gulf Islands of BC, indicated in yellow. This receiver is approximately 9m above sea level, is meant to cover Boundary Pass (south of the receiver), and will typically receive messages from up 60 nautical miles. Within 25 km of Boundary Pass there are 2,357,633 AIS message points. Displaying, processing, and manipulating this massive data set tends to be quite difficult for ArcMap, but the ArcGIS Pro’s ability to work with these data is phenomenal.
ArcGIS Pro is able to display all the data points within a few seconds compared to minutes in other GIS software. Considering visualization of large data sets is quite difficult for other GIS software, any type of processing or manipulation takes quite a lot longer than ArcGIS Pro. For example, converting these points to a raster using a kernel density tool took 3 minutes in ArcGIS Pro and over 10 minutes in other GIS software.
With the growth of big data science, data mining, and machine learning, it is comforting to know that, as my data set grows, Esri has provided the optimal platform to analyze big spatial data. It allows me to marry spatial and data analyses through ready-made tools or by creating my own programs, and it provides a simplistic way to connect to my database. Overall, the advanced functionality of ArcGIS Pro supersedes other software when working with BIG DATA.