THE NEW SHIBBOLETH

'Say now Shibboleth'; and he said 'Sibboleth'; for he could not frame to pronounce it right
Book of Judges, 12:6

In the biblical story of the Shibboleth (literally, an ear of grain in Hebrew, and some say a rushing stream), a single mispronounced word separated insider from outsider. Since then, it has named any indicator that identifies someone as one of us or one of them, as fully human being or less of one. The Shibboleth has always defined what it means to be considered human.

Today, a new version of this test is playing out between humans and AI. We have entered an age of suspicion, in which the origin of texts is increasingly scrutinized and readers search for signs that might reveal whether a text was written by a human or generated by a machine. Yet research consistently shows that most users across fields fail to reliably make this distinction. 

The New Shibboleth Project investigates this emerging culture of suspicion and its consequences. The Turing test, which has long shaped how we think about the boundary between human and machine, was fundamentally a test of performance: can a machine imitate a human well enough to fool a judge? The Shibboleth is a test of belonging: does the speaker display the signs that mark them as one of us, humans? The question is no longer whether machines can pass a Turing test. Now we are asking what kinds of cultural tensions the constant search for the new Shibboleth creates and what it reveals about conceptions of what is distinctly human. 

The project has three purposes:

1. Mapping the critical thought and cultural representation of the Shibboleth across history, and providing a theoretical framework for understanding the Shibboleth in the case of nonhuman entities.

2. Work in progress: a timeline of the shifting indicators that users recognize as AI’s Shibboleths since large language models went public. Analyzing what they reveal about perceptions of what makes language human. 

This is a rare moment in the history of language. We intend to understand it.