Scaling virtual agent-based testing for cross-platform analysis of algorithmic content curation

The use of virtual agents (i.e., software programmes emulating browsing behaviour) offers new possibilities for studying algorithmic content curation. Compared with human participants, deployment of virtual agents induces fewer costs and can be conducted in a fully controlled research environment that facilitates the investigation of the effects of specific factors (e.g., location or browser type) on content curation. Yet, so far the majority of agent-based studies rely on a small number of agents deployed for a single platform that limits the potential for comparative studies.

In our presentation, we introduce a new approach for scaling agent-based testing that uses distributed cloud infrastructure to simultaneously deploy a large number of virtual agents and then track their interactions with online platforms in a controlled environment. Because of its scalability, this approach enables new possibilities for conducting the analysis of algorithmic content curation across multiple platforms, in particular when such curation is
subjected to randomness (e.g., through the reshuffling of results by information retrieval algorithms for optimization purposes). We examine the advantages and drawbacks of this approach using a large-scale analysis of algorithmic curation by multiple search engines on a selection of topics varying from the US presidential elections to COVID-19.