Language Learning Models (LLMs) are all the rage these days. Tools like ChatGPT and others can easily be used to create bogus club reviews that do not offer new and relevant information to this community. But how can TUSCL adjudicators weed them out? Here are 7 tell-tale signs that AI was used to generate a review:
-
The Emdash. The emdash is an elongated single dash that is often used highlight and separate a subordinate clause — just like this. Most users composing a review in a browser will use a double dash -- like this -- or a single dash to achieve the same effect. There's no emdash key and it is difficult to produce an emdash in most browsers and operating systems. For example a Windows user would have to type Alt + 0151 to produce it. The emdash is a hallmark stylistic quirk of ChatGPT, which uses them ubiquitously. Though it's a strong indicator, an emdash isn't a guarantee a review is AI generated, as some tools such as MS-Word will automatically convert a double dash to an emdash.
-
Ice Cold Grammar. Human writers, even good ones, frequently make minor grammatical mistakes such as your in lieu of you're or it's where its is correct. We write fragments and run-on sentences. An article devoid of any grammatical flourishes should raise suspicions. One gray area we do encounter is reviews that were written by a human and polished by AI. I don't think we should find such reviews objectionable so long as the content is good.
-
Not X, but Y. This form of statement is employed by humans frequently so one needs to be careful, however, LLMs absolutely love this form and rely on it heavily because the it produces clear, well-structured text. For example, The club was not too crowded, but it was a festive and lively atmosphere. One or two of these in an article shouldn't raise eyebrows, but if half the content takes the form of Not X but Y, be suspicious. Other forms the AI loves include On one hand..., but on the other... and AI also loves to say In conclusion...
-
Conclusions in bold. LLMs love to put important or concluding thoughts in bold. Human writers do this too, so it's not a slam dunk. However, in conjunction with other indicators bold text conclusions may indicate the use of an LLM.
-
PG Content. All the major LLMs have guardrails that prevent users from generating explicit language and content. If the review reads like a USA Today article, it may have been generated by an LLM.
-
Redundancy and AI-flavored Word Salad. Repetition of the same idea with synonyms that produces word-salad is another tell-tale sign of LLM-generated content. A human might write something like, The club was crowded, but had a fun party vibe. An LLM might state the same idea as Although the venue was densely populated, it maintained an energetic and enjoyable party atmosphere.
-
A mile wide and inches deep. LLMs are great at generating volumes of snappy well-structured text. They do not know jack shit about most strip clubs (at least until @Founder trains up a model on the vast data he has.) They can't describe the dancers, they won't describe with detail the experience of getting a dance. The biggest tell tale sign of an AI-generated review is that it doesn't offer any new and relevant information of the club. The second biggest clue is that the review gets salient details of the club wrong. The lack of relevant and accurate information in conjunction with spotless grammar is probably the biggest indicator that someone used an LLM to generate the review. The best way to weed out AI-generated reviews is to ensure strict adherence to the review guidelines. Does the review accurately describe the PEOPLE, PLACE, and PRICES? If not, reject it!
-
Zero engagement from the OP. Unfortunately, this tell-tale sign only rears it's head after a review has been published. TUSCLers who use AI to generate reviews will rarely comment in the own reviews, and when they do they won't sound the same!
Here's an example of a recent review were you can see almost all of these indicators in action: tuscl.net. Do you have other good examples of AI-generated reviews that slipped the adjudication net on TUSCL? Do you have other tricks for spotting AI-generated content? Post them below!
Here's a few more: