Basics of Prompt Engineering

Adapted from the Prompting Guide by Graverman#0804

Updated 7-Dec 22

If you’re new to the world of AI art, prompt engineering may look daunting, but at its most basic, it’s just using words to paint a picture of what you want to create. In this guide we’ll discuss a few different elements of a basic prompt, providing newcomers with a solid foundation upon which to build anything they can imagine.

Prompt Elements

1. Core Prompt

The simplest way of describing the central theme, subject, or figure in your prompt, for instance;

1. Panda
2. A warrior with a sword
3. Skeleton

In simple prompts, this is often the center around which the rest of the prompt is built. Below are examples of 1-3:

As a beginner, it’s tempting to use a prompt this simple and just call it a day. While core prompts on their own often work relatively well with newer models at default DreamStudio settings, the image quality may suffer in earlier models and at non-default settings.

Additionally, while conceptually consistent with the prompt, these images are fairly generic. Their prompts could benefit from a lot more specificity, which is tied directly to…

2. Style

Style is a crucial part of the prompt. The AI model, when failing to recognize a requested style, usually defaults to one most common in related images.

For example, given the core prompt of “landscape,” the model would likely generate landscapes that were realistic or in the style of an oil painting.

Having a well-chosen style together with an effective core prompt is sometimes enough to create a fully-realized concept; after the core prompt, the choice of style influences your final image the most in a simple prompt.

The most commonly used styles include:

1. Realistic
2. Oil painting
3. Pencil drawing
4. Concept art

There are a number of ways to invoke a style in your prompts.

To take an example from above, the following are ways you might format a prompt for a realistic image:

• a photo of [core prompt]
• a photograph of [core prompt]
• [core prompt], hyperrealistic
• [core prompt], realistic

You can, of course, combine these modifiers to pursue greater realism, but a little often goes a long way.

Here’s our panda from Section 1 in each of the prompts above:

Referring back to our style examples, for an image in the style of an oil painting, adding something like “an oil painting of [core prompt]” to your prompt works well..

This sometimes results in the image showing an oil painting in a frame, to fix this you can just re-run the prompt or use negative weighting (discussed below).

“An oil painting of a panda”

For an image in the style of a pencil drawing, one easy approach is to add “a pencil drawing of” to your core prompt or make your prompt “[core prompt] pencil drawing”.

“a pencil drawing of a panda”

The same applies to landscape art.

“A landscape painting of a panda habitat”

3. Artist

To make your style more specific, or the image more coherent, you can use artists’ names in your prompt. For instance, if you want a very abstract image, you can add “in the style of Pablo Picasso” or just simply, “Picasso”.

Below are lists of non-living artists (subdivided by style) that can be used, but doing some art history research of your own is encouraged–you’ll learn a lot about what elements contribute to pieces that move you, and you’ll discover a lot of incredible art and artists whose work you might never have come across otherwise.

Portrait Artists

• John Singer Sargent
• Edgar Degas
• Paul Cézanne
• Jan van Eyck

Oil Painters

• Leonardo DaVinci
• Vincent Van Gogh
• Johannes Vermeer
• Rembrandt

Pen/Pencil Illustrators

• Albrecht Dürer
• Leonardo da Vinci
• Michelangelo
• Jean-Auguste-Dominique Ingres

Landscape Artists

• Thomas Moran
• Claude Monet
• Alfred Bierstadt
• Frederic Edwin Church

Note: Mixing the artists’ names can lead to interesting-looking art unlike anything any artist invoked ever made.

“An oil painting of a panda by Leonardo da Vinci and Frederic Edwin Church”

4. Finishing touches

Finishing touches are the additional elements added to your prompt to really make it look how you envision it. This is the part that some people take to extremes.

In relatively simple prompts, finishing touches might be adding “trending on artstation” for a polished, artistic flair or “Unreal Engine” for more realistic lighting. In more advanced prompts, things get way more complicated, but that’s beyond the scope of this guide!

You can add anything you want for a finishing touch, but here are some examples that work well:

Highly-detailed, surrealism, trending on artstation, triadic color scheme, smooth, sharp focus, matte, elegant, illustration, digital paint, dark, gloomy, octane render, 8k, 4k, washed-out colors, sharp, dramatic lighting, beautiful, post-processing, picture of the day, ambient lighting, epic composition

“An oil painting of a panda by Leonardo da Vinci and Frederic Edwin Church, highly-detailed, dramatic lighting”

Advanced Topic: Prompt Weighting [New in DreamStudio!]

Using prompt weighting, users can prompt the model to have more or less of certain elements in a composition, such as certain colors, objects or properties. Starting with a standard prompt and then refining the overall image with prompt weighting to increase or decrease compositional elements gives users greater control over image synthesis.

For example:

Side-by-side comparison of a prompt in DreamStudio without a negative prompt (left), and with a negative prompt (right). In this case the negative prompt is used to tell the model to limit the prominence of trees, bushes, leaves or greenery while maintaining the same input prompt.

A weight of “1” is full strength. A weight of “-1” is full negative strength. To reduce a prompt’s influence, use decimals.

For example, the prompt “A professional color photograph of a bearded man on the sidewalk, fujifilm : 1 | centered : 1” will often yield centered items that aren’t bearded men on sidewalks:

To mitigate this while retaining the “centered” element we’re after, we can make the “centered” prompt weigh less than the “bearded man” prompt. We do this by using a decimal smaller than 1.

A professional color photograph of a bearded man on the sidewalk, fujifilm : 1 | centered : .5

If you want even less weight on “centered” relative to “bearded man,” you can use an even smaller decimal.

“A professional color photograph of a bearded man on the sidewalk, fujifilm : 1 | centered : .1”

To turn off prompt weighting, begin a prompt with “||”.

To check how your weights are being interpreted by DreamStudio, click the    icon above the prompting text box. A pop-up will appear describing how your prompt is currently weighted.

 Negative Prompting :

Negative prompts are the opposites of a prompt, and allows the user to tell the model what not to generate. Negative prompts often eliminate unwanted details like mangled hands or too many fingers or out of focus and blurry images.

You can easily give negative prompts a try in DreamStudio right now by appending “| <negative prompt>: -1.0” to the prompt. For instance, appending “| disfigured, ugly:-1.0, too many fingers:-1.0” occasionally fixes the issue of generating too many fingers.

Conclusion

Prompt engineering allows you to better control what your images will look like. If done right, it improves image quality and composition significantly.

From the Promt.engineer team: “Thanks Graverman and DreamStudio!”