Quick Answer
- Most "magic phrase" prompting advice from 2023 has lost its edge, models were trained on those exact tricks
- What still holds up, according to actual research, is structural: showing examples, requesting a specific format, and asking the model to check its own work
- One technique alone, few-shot prompting, has taken real tasks from 0% to 90% accuracy in documented case studies
- One popular technique, role prompting ("you are an expert..."), has been shown to have little to no effect on actual correctness, only tone
Why Most Prompting Advice Doesn't Hold Up Anymore
The Prompt Report, described as the most comprehensive study of prompting techniques done to date, benchmarked six major prompting methods against the MMLU dataset using ChatGPT. The finding worth remembering: small, structural tweaks, format and example similarity in particular, improved accuracy by up to 90%, while some popular techniques performed far worse than their reputation suggests.
Sander Schulhoff, who runs Learn Prompting and the world's largest AI red-teaming competition, HackAPrompt, has tested this at scale with over 600,000 collected prompts in collaboration with OpenAI and Anthropic. His conclusion lines up with the formal research: some techniques deliver dramatic, measurable gains. Others, despite being widely repeated advice, do close to nothing for accuracy.
What the Research Actually Shows
- Few-shot prompting has taken a real medical-coding task from complete failure to near-perfect output, just by adding a handful of example input-output pairs, per Schulhoff's testing
- Few-shot prompting improved a clinical text extraction task from an F1 score of 0.288 to 0.676, more than double, in a peer-reviewed study
- A histopathology classification task jumped from 40% to 90% accuracy using 20-shot prompting in another peer-reviewed evaluation
- Combining self-consistency with chain-of-thought prompting produced accuracy gains of 17.9% on GSM8K, 11.0% on SVAMP, and 12.2% on AQuA over baseline chain-of-thought alone, across a systematic academic survey
- Role prompting ("you are a senior engineer") was found to have little to no effect on correctness, despite being one of the most commonly repeated pieces of prompting advice, its main measurable effect is on tone and writing style, not accuracy
The 10 Techniques Worth Using
Backed by strong evidence
-
Give one or two examples of what "good" looks like. Few-shot prompting is the single most consistently effective technique in the research above, and it's the easiest to actually do
-
Ask for the answer, then a self-critique of that answer. Self-criticism and self-reflection techniques are shown to meaningfully improve output quality across studies
-
For anything involving reasoning or math, ask for the work to be shown, then checked twice. Chain-of-thought combined with self-consistency delivers real, measured gains on reasoning-heavy tasks specifically
-
Specify the exact output format you want. Format consistency was one of the two biggest levers found in The Prompt Report's benchmark
-
Match your examples closely to the actual task. Example similarity was the other biggest lever, mismatched examples measurably hurt performance
Genuinely useful, less dramatically proven
-
"Ask me clarifying questions before answering." Reduces the single most common cause of a bad answer, a vague question
-
"Tell me when you're not sure, don't just sound confident." Doesn't eliminate errors, but surfaces the ones worth double-checking
-
"Give me the bottom line first, then the details." Respects your time and lets you judge whether you need the full explanation at all
-
Ask for the answer ranked by confidence. Turns a flat list of options into something you can actually prioritize
Weaker than its reputation suggests
- Giving it a role or persona ("explain this as a doctor would") still has a place, but set expectations correctly: it shapes tone and audience fit, not correctness. Don't rely on it to fix an answer that's actually wrong
Chaining a Few of These Together
These work even better combined than alone. A realistic example:
"Here are two examples of the format I want. Ask me clarifying questions before answering. Once you have what you need, give me the bottom line first, then the details as a table. Flag anything you're not fully certain about."
Where These Won't Save You
- They don't fix a genuinely unclear question. Structure helps the model organize a good answer, it can't manufacture information you never provided
- They don't replace fact-checking. Asking for confidence levels reduces false certainty, it doesn't guarantee the answer is actually correct
- Some techniques matter less on reasoning models. Explicit step-by-step requests add less value on models that already reason internally by default, the structural techniques above still hold up regardless
Sources & Official References
- Sander Schulhoff, Learn Prompting and HackAPrompt, via Lenny's Newsletter: lennysnewsletter.com
- A Systematic Survey of Prompt Engineering in Large Language Models, arXiv: arxiv.org/html/2402.07927v2
- The Prompt Engineering Report Distilled, quantitative technique comparisons: arxiv.org/pdf/2509.11295
Figures above come from peer-reviewed studies and named industry research as cited. Results vary by model, task, and domain, treat these as strong directional evidence rather than guaranteed outcomes for your specific use case.
