Guest post by Tracy Chou yes, that’s elvis. he’s the coda of my very own chatgpt horror story … i share this as a cautionary tale about generative ai and the ways that it supercharges incompe…
But the article author wasn’t interfacing with chatgpt, she was interfacing with a human paid to help with the things the article author did not know. The wedding planner was a supposed expert in this interaction, but instead simply sent back regurgitated chatgpt slop.
Is this the fault of the wedding planner? Yes. Is it the fault of chatgpt? Also yes.
They’re not capable of actual intelligence or providing anything that would remotely mislead a subject matter expert. You’re not going to convince a skilled software developer that your LLM slop is competent code.
But they’re damn good at looking the part to convince people who don’t know the subject that they’re real.
I think we should require professionals to disclose whether or not they use AI.
Imagine you’re an author and you pay an editor $3000 and all they do is run your manuscript through ChatGPT. One, they didn’t provide any value because you could have done the same thing for free; and two, if they didn’t disclose the use of AI, you wouldnt even know your novel had been fed into one and might be used by the AI for training.
I think we should require professionals not to use the thing currently termed AI.
Or if you think it’s unreasonable to ask them not to contribute to a frivolous and destructive fad or don’t think the environmental or social impacts are bad enough to implement a ban like this, at least maybe we should require professionals not to use LLMs for technical information
Yea yea words.
Trust but verify.
what does this have to do with the article
Here’s a better idea - treat anything from ChatGPT as a lie, even if it offers sources
But the article author wasn’t interfacing with chatgpt, she was interfacing with a human paid to help with the things the article author did not know. The wedding planner was a supposed expert in this interaction, but instead simply sent back regurgitated chatgpt slop.
Is this the fault of the wedding planner? Yes. Is it the fault of chatgpt? Also yes.
Scams are LLM’s best use case.
They’re not capable of actual intelligence or providing anything that would remotely mislead a subject matter expert. You’re not going to convince a skilled software developer that your LLM slop is competent code.
But they’re damn good at looking the part to convince people who don’t know the subject that they’re real.
I think we should require professionals to disclose whether or not they use AI.
Imagine you’re an author and you pay an editor $3000 and all they do is run your manuscript through ChatGPT. One, they didn’t provide any value because you could have done the same thing for free; and two, if they didn’t disclose the use of AI, you wouldnt even know your novel had been fed into one and might be used by the AI for training.
I think we should require professionals not to use the thing currently termed AI.
Or if you think it’s unreasonable to ask them not to contribute to a frivolous and destructive fad or don’t think the environmental or social impacts are bad enough to implement a ban like this, at least maybe we should require professionals not to use LLMs for technical information