Mastering Prompt Engineering or the Art of AI whispering
March 05, 2023
Generative AI systems that respond to plain language inputs are a game-changer. It means anyone can interact with AI without needing programming skills. Users can generate complex outputs like code, images, music, or text just by giving clear instructions. This saves time and effort while also unleashing new possibilities for creative expression and problem-solving.
In this post we will explore the following topics:
- The importance of prompt engineering
- Practical examples of prompt engineering
- Resources and tools to get started with prompt engineering
The importance of Prompt Engineering
Anyone who has used generative AI systems knows that getting the best results for a task requires writing very precise instructions. This process is called prompt engineering, and it’s a crucial skill for effective use of generative AI. Crafting effective prompts involves understanding the AI model’s capabilities and limitations, selecting the appropriate input format, and providing precise instructions.
The hottest new programming language is English— Andrej Karpathy (@karpathy) January 24, 2023
Mastering prompt engineering leads to more accurate and relevant outputs from generative AI systems. Tesla’s former chief of AI Andrej Karpathy describes it as a kind of “large language model (LLM) psychologist”.
Jaon Allen won the Colorado State Fair’s annual art competition last year, he used a combination of multiple Midjourney outputs (AI art generator tool) and manual editing on Photoshop to create “Théâtre D’opéra Spatial”
In this interview 1, he describes his process and how he spent hours searching for the perfect prompt to create his work.
“I’m not sharing my prompt ever, it’s because the prompt is valuable. I think that it proves there’s skill and time and creativity involved in creating the prompt.” — Jason Allen
Prompt engineering is starting to be recognized as a valuable skill, the superpower to get AI to generate the outputs you want. Companies are starting to hire Prompt Engineers 2 and new businesses are emerging around it.
Practical examples of Prompt Engineering
Designing a prompt
A prompt may consist of one or a combination of the following components 3 :
A specific task you want the model to perform:
It allows to add external information or additional context, which guides the model to generate better responses. Example:
Twitter is a social media platform where users can post short messages called "tweets". Tweets can be positive or negative, and we would like to be able to classify tweets as positive or negative
The input can be the question that we want to answer, examples of answers or an image to edit in the context of image generators.
Indicates the desired type or format of the output 4.
Format everything in Markdown,
output this in a table
Examples of prompting strategies
Few-shot prompts are prompts that include a few demonstrations of the task, which improves the performance 5.
An advanced technique to interact with an AI Chatbot is to include a priming prompt. This allows you for example to give a personality to the model. For example if you want it to act as an advertiser 6 :
I want you to act as an advertiser. You will create a campaign to promote a product or service of your choice. You will choose a target audience, develop key messages and slogans, select the media channels for promotion, and decide on any additional activities needed to reach your goals. My first request is "I need help creating an advertising campaign for a new type of energy drink targeting young adults aged 18-30."
You can find some additional examples here
Chain of thought
This type of prompting tells the model to explain its reasoning before giving the final answer. For instance, using
Let's think step by stepallows to improve the permormance in reasoning tasks 7.
Performing some complex tasks using a single run of a Large Language Model (LLM) can be challenging. To solve this problem, we can chain multiple prompts to achieve better results 8.
If you have attempted to generate images with DALL-E, Midjourney, or Stable Diffusion, you may have experienced disappointing results. This is likely due to an incomplete or insufficient prompt. Here are some examples of what your prompt may be missing to achieve fabulous art:
featured/trending on X
However, creating art with just words can be challenging, but Prompt engineers can now count on ControlNet 9 which is a new flexible method for adding an additional input, such as a sketch or human pose, to guide the generation of new samples in AI models.
Resources & Tools
Prompt-Engineering-Guide: This Github repository is a gold mine for AI whisperers, it contains a large list of courses, papers, datasets and tools to get you started with prompt engineering.
Best practices for prompt engineering by Open AI.
learnprompting.org: A free, open source and user friendly manual for prompt engineering.
AIPRM: AIPRM is a prompt template collection for ChatGPT, you can choose your task (SEO, SaaS, Marketing, Art, Programming..) and access a curated list of prompts. I would like to thank my colleague Léo for sharing this.
PromptHero: One of the largest prompt libraries for AI generated art.
Generative AI has enormous potential, and as the tools and models improve, they will become more user-friendly and effective, unlocking fresh opportunities for prompt engineering while potentially streamlining the process.
Prompt engineering will evolve alongside AI tools, leading to a new breed of prompt engineers who specialize in fields like advertising, law, and art, as well as specialized tools. As the practice becomes more widespread, it will transform existing jobs and empower professionals to achieve more.
Adding Conditional Control to Text-to-Image Diffusion Models