Video quality evaluation in the cloud

Video quality evaluation with subjective testing is both time consuming and expensive. An interesting new approach to traditional testing is the so-called crowdsourcing, moving the testing effort into the internet. The QualityCrowd framework allows codec independent, crowd-based video quality assessment with a simple web interface, usable with common web browsers. However, due to its codec independent approach, the framework can pose high bandwidth requirements on the coordinating server. We therefore propose in this contribution a cloud-based extension of the QualityCrowd framework in order to perform subjective quality evaluation as a cloud application. Moreover, this allows us to access an even larger pool of potential participants due to the improved connectivity. We compare the results from an online subjective test using this framework with the results from a test in a standardized environment. This comparison shows that QualityCrowd delivers equivalent results within the acceptable inter-lab correlation.