How to get the best from AI with your prompts

LLM output is improved by a respectfully worded prompt. The only caveat is that it’s also possible to be overly polite. The secret – as with many things in life, including human-to-human communication – is to find the right balance.

Welcome to the weird and wonderful world of prompt engineering. With generative AI now used to create entire marketing campaigns, including Coca-Cola’s 2024 Christmas ad, there’s a very real premium on crafting prompts. Every word counts, quite literally.

‘Do not hallucinate’

LLMs will also perform better if you offer to tip them $20. And even better still if you raise that offer to $200. But the secret to getting the best out of AIs isn’t just about adding zeroes, explains Joe Crawforth, head of research and development at Jaywing.

“That isn’t just like an exponential growth thing, though,” he says. “It’s not going to just do better and better. It meets its limit.”

Similarly strangely, a release of prompt templates from Apple last year revealed that Apple Intelligence is primed to answer user requests with the instruction: ‘Do not hallucinate.’

To write a good prompt? “Be direct, clear, and concise, and add context,” says Crawforth. “Because context is king at the end of the day.”

He adds that it’s important not to take the example of Apple telling its AI to avoid hallucination as an indication that LLMs are self-aware. “The newer models have been trained on data where papers have talked about hallucinations, so it understands the concept of hallucination in terms of generative AI,” he explains.

“They have a conceptual understanding of ‘don’t hallucinate’… What you’re trying to do is force it to look at the context that you’re giving it instead,” he adds.

Briefing machines

Jon Wiliams, founder and CEO of The Liberty Guild, wrote for The Drum last year about hiring a prompt engineer. Since then, he says, AI use has been incorporated into the day-to-day work of most employees across the agency. “The way that you interact and the way that you get what you want is a core skill, or should be a core skill, of everybody moving forward now,” he explains.

“In the old days, we would receive a brief from a client and we would respond to that. And in many ways, a prompt is actually just a brief to an LLM. What you’re doing is you’re telling it what you want and what you need and how you’d like it served up. So, all we’re doing is briefing the machine. And the words that you use are incredibly important in the same way that for me, the words I use in every sentence have been incredibly important throughout my entire career, so it’s kind of no change there.”

Macklin Andrick, senior creative technologist at George P Johnson, says he uses tools such as Midjourney, ChatGPT and AI code editor Cursor in his work for clients such as IBM, Salesforce and Google to “support ideation, research and prototyping.”

“While ‘prompt engineering’ can sometimes sound like a complex science, at its core, it’s about asking for what I need in a way that the AI can understand,” he explains. “For me, the process is often iterative – trial and error plays a big role. I know the quality of the AI’s output is only as good as the context I provide.

“If the results feel off, I clear the slate and start fresh. Garbage in, garbage out. In some cases, I’ll even ask the AI itself to improve my prompt or generate a better one – it becomes a game of experimenting with sequences of words or combining prompts with reference images to inch closer to the desired result.”

Andrick adds that AI rarely delivers a finished product. “It’s like working with a half-carved sculpture – there’s always a need to refine, reshape and add the final details. The real craft lies in editing: continuously iterating and chiseling the output until it aligns with the vision.”

Magnetic words

Lucas Stanley, a cinematographer at Jellyfish, used AI to create the agency’s video Christmas card as well as for other projects. When trying to get the best from generative AI, he says, a socio-linguistic approach helps. “If you want to get really good at prompting, the lens through which you have to see prompting is the same lens through which models are trained,” he explains.

Generative AIs are built on datasets of human-made content, be that words, images or videos. Associations are established, unique to each tool, that inform its output. Getting to know each tool and thinking laterally to work around blocks that might arise in achieving the desired creative is key for Stanley.

“They are associations which are in the general public’s mind in popular media which manifest within AI,” he explains, adding: “There are loads of names for these associations. The one that we use within Jellyfish and that I use with clients is magnetic prompts. Some people call them master prompts.”

He came up against one such ‘magnetic prompt’ in a spec ad he made recently. Stanley wanted clothes to dance magically in midair, with nobody wearing them. However, the AI he used seemed unable to originate images of clothes free from humans, trained as it had been on a dataset that presumably only contained clothes that moved when on human bodies.

“You’ve got this giant magnetic association prompt next to it that is if clothes are moving you need people in them,” he explains. “So you need to put another prompt over here that pulls you just enough outside of that well and lets you sit just outside of it. And we were butting our heads against that for ages.”

After trying phrases including ‘clothes without people,’ ‘the invisible man’ and ‘caught on a gust of wind,’ the phrase that led the AI to create the desired video of moving, unworn clothes was ‘coathanger.’ “[It was] a strong enough prompt over here for it to realize that clothes can move or hang without people in,” Stanley explains. “The point being that that is not something that you learn. What it is is a very creative socio-linguistic approach.”

A good place to start, for the uninitiated, he says, could be using a prompt builder, which assists in the task of creating the right prompt. Despite the effort it took him to arrive at ‘coathanger,’ the proof is in the final product. It took Stanley 25 hours and around 500 generations to create the minute-long spec ad – something that would’ve taken a team weeks using traditional production methods.