What “Prompt to Image” Really Involves

Prompt to image means handing a text description to a generative model and getting a picture back. Words in, image out. The mistake almost everyone makes is assuming the sentence carries most of the weight. On Stable Diffusion-class models it usually does not. The checkpoint, the sampler, the step count, and — if you run locally — the VRAM you have left decide the result at least as much as the phrasing does.

If you are not running any of this locally, most of the controls above are not yours to touch — a hosted tool sets the sampler, the step count and the memory budget, and hands you two levers instead: which model or style preset to pick, and the wording itself. That is not a lesser path, only a narrower one. Choose the engine first, because a photorealistic model will not return anime however you phrase it, and an illustration preset will not hand back a photograph. Then spend the effort on the sentence — name concrete objects rather than moods, and write actions as participles.

The nine, in the order they actually bite

1. Check the checkpoint before rewording.
2. Check the checkpoint's own published settings.
3. Try the participle form of the verb first.
4. Keep weights inside 0.4–1.6.
5. Watch the term order.
6. Keep AND terms off equal weight.
7. Raise one related attribute at a time.
8. Change the sampler only after the rest is ruled out.
9. Suspect the upscaler on large local frames.

The model sets the ceiling, not the prompt

When the style comes out wrong, rewriting the prompt will not save it. Ask a photorealistic checkpoint for anime and you get anime-ish realism no matter how you word the request. Switch checkpoints. This is the most common misdiagnosis in the whole workflow: an hour spent rewording, aimed at a model that cannot produce that look at all.

Checkpoint authors publish recommended settings, and those settings are not decoration. The AbyssOrangeMix model card on Hugging Face is a plain example: it names orangemix.vae.pt for the VAE, calls DPM++ SDE Karras good, gives Clip skip 1 or 2, starts illustration runs at 20 steps, and on AbyssOrangeMix3 puts CFG 8 inside a 6 to 12 range. Step outside that envelope and you are evaluating the checkpoint at settings it was never tuned for. An image that reads grey and washed out is frequently just a missing VAE, which is why the card names one.

Verb form changes the picture

Prompt parsers tokenize, and English morphology is not free. Describing a figure with run has produced no running pose in reported tests — the subject simply stood there, and in one frame disappeared entirely. Changing that single token to running produced the pose immediately. Same model, same settings, one suffix apart.

So when an action will not appear, reach for the participle before reaching for heavier weighting. It is cheaper, and in practice it works more often than stacking brackets does.

Weight syntax, with the actual numbers

In AUTOMATIC1111 WebUI, parentheses multiply a term by roughly 1.1 and square brackets divide by roughly the same; nesting compounds, so ((term)) lands near 1.21. Explicit values go inside parentheses with a colon — (term:1.3). NovelAI runs a different set of marks — curly braces do the strengthening there, and the multiplier per level is not the same as WebUI's, so the numbers above should not be carried across.

Stay between 0.4 and 1.6. Below 0.4 the token gets ignored; above 1.6 the model over-fits the concept and the frame starts to break. Order matters as well, because weight decays left to right: put the landscape tag first and your subject shrinks.

That band is checkable here rather than quoted. The parser this site ships is tools/weight-syntax.js in the library behind the tool on the front page, and it prints the weight behind every term. It also shows the trap that catches people who already know the syntax: the brackets that carry the number do not protect it from the brackets around them.

That band is the short version. How to weight a prompt, and how to write a negative goes through the marks themselves, how bracket depth compounds in both directions, and why a negative takes the same syntax in a field of its own rather than a stronger mark.

Two failure modes are worth knowing before you hit them. Joining concepts with AND at equal weight tends to fuse them — a prompt asking for cabbage and meat returned cabbage wrapped around meat. Comma separation or plain adjacency almost never does that. And pushing one element too high contaminates its neighbours: a test prompt that raised several hair colours at once produced a frame where the whole composition fell apart.

Samplers are not interchangeable

Reported tests out to 40 steps against an Euler baseline separate them cleanly:

• Simple results — Euler or Heun. Heun tolerates fewer steps, which saves time.
• Speed and variety — DPM++ 2M Karras or UniPC, 20–30 steps.
• Quality, convergence be damned — DPM++ SDE Karras at 8–12 steps, or DDIM at 10–15.
• Reproducible output — avoid ancestral samplers (Euler a, DPM2 a, DPM++ 2S a, DPM++ 2S a Karras). They add noise at every step and never settle. Fine for one-offs, wrong for anything you need to regenerate identically.

Among the older methods, PLMS converged worst; LMS Karras was only slightly better. Second-order solvers cost roughly double per step, because they evaluate the U-Net twice. DPM adaptive converges best of all and runs slowest of all.

If a checkpoint looks wrong on Euler, switching to the DPM++ family at 20–25 steps is the first thing worth trying.

On a 12 GB card, the upscaler is the bottleneck

VRAM, not prompt quality, is where local setups actually stall. The useful habit is measuring before rendering. A plugin called VRAM Estimator — space-nuko/a1111-stable-diffusion-webui-vram-estimator — benchmarks your card once, and on a 3060 12 GB that benchmark ran for about fifteen minutes, during which you cannot generate. After it, predicted usage is shown beside the size fields, so you see the collision before the render dies.

The three figures below are what that plugin reported on a 3060 12 GB in a June 2023 write-up of it on Zhihu — not measurements taken here, and not a general rule for other cards:

• 768×512, batch of 4 → 46.09% used, completes.
• Same image with Hires.fix and 4x-UltraSharp → over 100%, and it did fail.
• Changing only the upscaler → 75.90%, completes.

At 1536×1024 the same write-up reports a starker gap. Latent upscaling runs 7 images in a single pass on 12 GB. 4x-UltraSharp at identical size manages 3. The sharper upscaler genuinely looks better, so this is a trade, not a free win.

Two more things that have cost people whole evenings. Use round numbers — 1024 beats 1000, because non-standard dimensions force extra conversion work. And on ComfyUI, if a setup that used to run four LoRAs suddenly dies with none loaded after an update, check cross-attention and the CUDA allocator; reverting cross-attention to xFormers and the allocator to the built-in async one cleared it in at least one reported case.

Below 12 GB, the browser WebUI is often simply the wrong tool. Launch flags help — --xformers --no-half-vae --medvram-sdxl for 8–16 GB, --xformers --no-half-vae --lowvram under 8 GB — and xFormers is the usual first move, because attention is where the memory goes. Quantized GGUF builds push the floor lower still: 6 GB VRAM with 16 GB system RAM is workable, and 8 GB cards do better on the Q4_K_S variant than on full precision.

Nothing in this section was measured on this site: the tool here reads images, it does not render them, so it has no VRAM to report. The 3060 figures and the upscaler comparison are the June 2023 write-up's, named above; the launch flags, the round-number advice and the quantisation note are general practice for small cards rather than figures anyone here produced.

One thing worth stating plainly: the only figures on this page this site produced itself are the five in the weight table above, printed by the parser it ships. Everything else is reported from somewhere else — the checkpoint's settings from its model card, the VRAM figures from the June 2023 write-up, and the participle and sampler observations from tests run elsewhere, marked as reported. The tool here reads images and writes prompts; it does not render them, so it has no way to measure a sampler, a step count or a card.

Where that leaves the prompt

The prompt is the last lever, not the first. Work the nine above in the order they are given. Rewriting the sentence before doing any of that is the slow route to a worse picture.