Image by Emily Zheng and DALL-E 2
As an agency that works with companies at the forefront of technology, we endeavor to do the same. When generative AI gained more steam (and headlines) in the last couple of months, we didn’t shy away from it. Instead, we invited ChatGPT to play—and learned some pretty surprising things.
1. It can deny your request! (And we got scolded, too)
When my sister, an academic, joined me in working from home one day, we jokingly asked ChatGPT to write a sarcastic thank you note for receipt of a (very small) grant. ChatGPT was not pleased—not only did it deny our request due to its inappropriate nature, but we got a mini lecture about how we should be grateful for funding. Lesson learned: aspiring comics can forget AI. Turns out we aren’t alone in our experience—there are several Reddit threads (like this one and this one) that recount ChatGPT’s dismissal of ridiculous but largely harmless queries, followed by a brief morality lesson.
Additional bummer: even if ChatGPT is cool with your query, you can’t rely on its availability. The free version is increasingly unavailable due to high demand, so we suggest saving your burning questions for a Saturday night.
2. It can’t create a poem that doesn’t rhyme—even when you specify that
We tried so many times, but alas, ChatGPT simply can’t conceive of a poem without rhymes (hey, that rhymed)—even when explicitly asked. This was perhaps the biggest sign that generative AI still has a way to go in bending to our wills.
3. It can offer a quick explanation, but you still need to do some work
We love ChatGPT for its quick overviews and definitions. Instead of spending 20 minutes sifting through Google search results to learn what referential integrity means in the database world, ChatGPT breaks it down in an understandable way—in less than a minute. However, the AI can’t discern between a reliable and an unreliable source and can supply incorrect or inaccurate information. A great example is when ChatGPT was asked to review Conan O’Brien’s podcast; the AI reviewed it as a memoir and said it included topics such as O’Brien’s divorce (he has never been divorced). Not catching these things can be a huge risk for brands, and in the case of Google, it was a $100B risk.
4. Input and output are limited
What you give will determine what you get, so practice strategically worded queries. Because we can’t feed ChatGPT all the sources we’d use to inform new content, we end up getting only a few paragraphs that sound good when you read them, but ultimately, say nothing of consequence. This input limitation often results in copy that omits the “but how/but why” aspect, which is crucial for effective marketing content.
At 2A, we comb through a wealth of materials—such as research reports, press releases, blogs, and interviews—to build out content. We absorb them like pieces of a puzzle and put them together as a strong piece of content that helps our clients meet a specific goal. An additional input limitation is that the platform can only draw information from 2021 and earlier, so it knows nothing of what’s happened since 2022.
5. It’s a great starting point, but won’t get you to the finish line
ChatGPT is valuable for high-level brainstorms, general outlines, and inspiration for social media copy. But to create a stellar product, it’s best to limit ChatGPT to the role of springboard and then dive in with human talent and experience. This is especially true for marketing assets such as case studies, which are about highly unique experiences that integrate effective ingredients like real-life quotes.
Want to pen a personal essay, reflective blog post, or investigative report? ChatGPT can’t help you there. Everything we asked ChatGPT to produce required a fair amount of tweaking and additions, so use it for its bits and pieces but not as something that will give you a final product. Keep an eye out for embedded bias and other gaffs that could ruin your reputation—something this creator experienced at the height of popularity. In short, think of generative AI like a 5-year-old: it can say insightful things, but don’t leave it home alone.
TL;DR
We are embracing ChatGPT for what it is currently good at, which is its ability to assist and accelerate our own creative process. As content creators, we were admittedly not crazy about the idea at the start—but we know things change and we plan to be along for the ride.