Prioritization-based subsampling quality assessment methodology for mobility-related information systems

Mobility-related information systems, such as on-street parking information (OSPI) systems have become more popular in the original equipment manufacturer (OEM) industry over the last decade. However, there is a lack of methods to assess their quality at a large scale. This paper introduces a data-driven methodology to measure the true quality by fleet data prioritization-based subsampling strategies (PSSs). It is applied to the use case of OSPI using parking events (PE), but is applicable to other mobility-related information systems utilizing their respective fleet data. PSSs are defined based on neighbourhoods and time periods. Each PSS generates a unique set of spatio-temporally important areas at different quadkey zoom levels over 168 week-hours, called slices. The importance weight in each slice depends on the volume of PE within them. The algorithm for each PSS automatically selects important areas and time frames that are vital to be observed. Sample prediction models are used for the benefits assessment of the methodology by comparing it against non-prioritized randomized selection of ground truth. It is proven that the methodology can lessen the effort of ground truth collection, while maintaining the amount of information necessary to assess the true quality of a prediction model.