Dangerous Imaginations: The New Epistemic Threat from Synthetic Media

Imagination takes place partly outside the mind, through technological aids. This fact is not new in itself, but new technologies are changing the nature of imagination. Synthetic media generators, such as DALL·E, and synthetic media items, such as deepfakes (realistic images and videos algorithmically generated to depict people doing or saying things they never actually did or said), challenge our fundamental epistemic standards. But the nature of the new epistemic threat they pose remains elusive, since fictional or distorted representations of reality are at least as old as photography itself. Existing characterizations in social epistemology of the threat posed by deepfakes struggle to pinpoint what exactly is novel about it. This paper has two aims: to characterize the new epistemic threat posed by synthetic media, and to identify the conditions under which a synthetic media item is epistemically malignant or toxic. We argue that the epistemic threat posed by deepfakes in particular, and by synthetic media more broadly, lies in the unprecedented practicable possibility afforded to ordinary computer users to cheaply and effortlessly create fictional worlds that are indistinguishable from the real world and share them widely. A synthetic media item is epistemically toxic or malignant in a given context when a person acquainted with the context is liable to confuse the fictional world depicted in it with the real world or with credible representations thereof on matters of moral or epistemic significance.