Rod Howard Résumé

ColorPages · Built to learn

Designing Around the Limits of Generative AI

The ColorPages scene editor: the Wizard page from Costume Night on the canvas, the repair tools beside it, and six candidate drawings under it
Scene editor
The ColorPages Studio: four books in progress, each with its lead page, my notes on pages, and how many scenes are left to judge
Studio
The ColorPages character library: twenty characters, each listed with the packs they appear in
Characters

Generating a drawing was easy. Keeping characters consistent across a book took more. I designed and built ColorPages around selecting and repairing drawings, preserving what worked instead of starting over.

Role
Design and build, start to finish
Scope
AI authoring workspace · Local image generation · Print production
Scale
11,049 renders → 389 finished pages

The core tension: idealized AI vs. human agency

Generative AI suggests a simple workflow: describe what you want and get the finished result. Building a real product around it exposed a different reality.

I chose local image models for privacy, speed, and fine-tuning control, but they came with three recurring constraints:

01The local model bottleneck
Fine-grained LoRA training and no marginal API cost came with higher variance, character drift, and weak spatial editing.
02The “almost-right” trap
Most failures were small. A page could be 90% right but still fail because of an extra limb, bad hair texture, or stray artifact.
03Coupled mutation
Changing a prompt to fix one detail often regenerated the entire composition, destroying parts that already worked.

3.5%389 of 11,049 renders became finished pages.

Judging candidates, not generating them, became the bottleneck. I designed ColorPages around reducing that attention cost.

Get the character right

Character consistency evolved from words, to references, to trained identity

A book about my nephew fails if the child on page twelve is not recognizably the child on page one.

I did not solve that in one move. Each approach exposed the limitation that led to the next.

Attempt 1 · Prompt-based canon

Describe the character

It helped, but identity still drifted when pose or viewing angle changed. Read attempt 1Close

Every character started with a written identity canon carried into each prompt. Over time I tightened the rules around which traits belonged to the character versus the illustration style.

The canon says who. The style says how.

It helped, but identity still drifted when pose or viewing angle changed. Negative instructions were unreliable too. Telling the model what not to draw could reinforce the unwanted trait instead.

What I learned: prose was useful reinforcement, but too weak to be the primary source of identity.

Attempt 2 · Pinned reference sheets

Show the character

The reference sheet solved one problem but surfaced another: the character should persist while expression changes scene by scene. Read attempt 2Close

I moved the stable identity into an approved full-body reference sheet that stayed pinned across scenes.

That worked much better, but introduced a new failure with similar characters. When two toddlers appeared together, traits could bleed between them because the system was treating the cast too generically.

I changed the generation logic to bind traits to each figure individually, and limited that behavior to multi-character pages so it would not override a scene's own instructions.

What changed: identity now had a visual source of truth instead of depending on the prompt to reconstruct it.

Attempt 3 · Expression boards

Separate identity from expression

The reference system worked, but every generation still had to carry and reconcile those images. Read attempt 3Close

The reference sheet solved one problem but surfaced another: the character should persist while expression changes scene by scene.

Character Studio evolved from a casting portrait, to a full-body sheet, to an expression board built from the locked identity.

I also tried localized face editing. That failed because the mask preserved the very mouth shape I was trying to change. Editing the whole reference proved more reliable.

But even a correct expression on the reference was not enough:

41% requested grin appeared when the prompt only pointed to the expression reference
18 / 18 when the prompt also explicitly named the expression

That produced one of the system's lasting rules:

Shape language for identity. Emotion words for expression.

Attempt 4 · Character LoRAs Where it landed

Train the identity

The reference system worked, but every generation still had to carry and reconcile those images.

The next step was putting stable identity into trained weights for Trey and Jasmine. Once that recipe worked, solo pages no longer needed a pinned character sheet.

And importantly, the previous layers did not disappear.

The LoRA recipe still includes the written identity canon. Characters without LoRAs still use pinned sheets. Expression wording still controls emotion. Finished-page repairs still go through the edit model.

12 → 5 median renders per finished character page
76% → 29% pages requiring manual repair

Fig. 01Persistent identity, per-scene expression

PersistentThe child, Trey, on every page, drawn by his trained model.

Per sceneThe expression

Trey's face on the Musician page, happy
happy
Trey's face on the Surfer page, excited
excited
Trey's face on the Soccer Player page, proud
proud
Trey's face on the Astronomer page, surprised
surprised
Trey's face on the Painter page, calm
calm
Trey's face on the Sprinter page, determined
determined

Per sceneThe pose

Trey as deep-sea diver, swimming, curious; a picked page from When I Grow Up
swimming
Trey as builder, kneeling, focused; a picked page from When I Grow Up
kneeling
Trey as fisher, seated in a boat, surprised; a picked page from When I Grow Up
seated in a boat
Trey as figure skater, gliding on one foot, calm; a picked page from When I Grow Up: Sports
gliding on one foot
Trey as hockey player, skating with a stick, focused; a picked page from When I Grow Up: Sports
skating with a stick
Trey as police officer, holding up a sign, brave; a picked page from When I Grow Up
holding up a sign
Twelve picked pages from two books. Each face is what that page's prompt asked for; each pose is what its scene needed.

Keep what works

Fix only what is actually broken

My first response to a defect was to edit the prompt and generate the page again.

That is still the right move when the prompt is the problem. But often the prompt was already right and the model had simply produced something weird: an extra line, a bad hand, the wrong mouth, or a misplaced object.

Regenerating the whole page meant putting everything that already worked back at risk.

So I built repair tools around the size and type of the defect.

  1. Erase or ink. Remove an unwanted mark or draw in a missing line without touching the rest of the page.
  2. Move, cut, copy, and paste. Reposition successful artwork deterministically instead of asking the model to recreate the composition.
  3. Targeted redraw. Brush over the area that actually needs new drawing, then give the model a new prompt that affects only that region.
  4. Code-rendered overlays. Handle repeatable elements like lettering outside the image model when precision matters more than generation.

The model still had a role. I just stopped giving it more of the page than the repair required.

1 in 3 canvas edits became the finished page: 120 of 359
1 in 12 targeted AI redraws became the finished page: 46 of 563

43% of finished pages were repaired rather than regenerated: 166 of 389.

Fig. 02Five tools, five real repairs

Tennis Player as first drawn, with a second tennis ball in the top-right corner
The picked page, with the second ball erased
beforeafter · erase

A second ball the model added. Gone, and nothing else moves.

queued

Every panel ends on the page I picked. Press a tool to see what it fixed.

Encode systemic memory

When a failure repeats, the system should remember it

Some defects were random. Others kept happening.

A fuel gauge made the difference impossible to ignore. The model failed to render it correctly in 45 consecutive attempts. At that point, another prompt tweak was no longer useful experimentation. I had learned something about the model that the system itself did not yet know.

Fig. 03One gauge, asked for six times

The brief asked for“one gauge on Rumble's hood with the needle pointing straight up in the middle”

Render 209: a clock, in the windshield
A clock, in the windshield
Render 210: three dials, as roof lights
Three dials, as roof lights
Render 217: a dial, on the roof
A dial, on the roof
Render 218: a clock on the cape, a needle on the hood
A clock on the cape, a needle on the hood
Render 219: a dial, on the roof
A dial, on the roof
Render 220: a dial, in the windshield
A dial, in the windshield
My redraw of render 218: the clock gone, the needle still on the hood
My fix of 218, in two redraws: “remove the clock”, then “the hood of the car should be empty”. The needle stayed.

0

correct gauges in the ~45 Pit Stop Heroes renders I reviewed on Aug 6

Every failure here cost a full render to discover.

Once a failure becomes predictable, the system should stop asking the human to rediscover it.

I started encoding recurring failure modes into validation rules and reusable authoring skills that run before generation.

One recurring example was negative phrasing. Instructions like “no signs” often caused the model to add signs instead. Once that pattern was known, ColorPages could rewrite the brief before spending another render discovering the same failure again.

Fig. 04One brief, caught before it rendered

As I wrote it

…leave the bottom fifth of the page completely empty — no words, no letters, no numbers and no signs anywhere in that band

4 errors

Rewritten

…the whole scene drawn in the top four fifths of the page with the bottom fifth left as one clear open white band

0 errors

To the image model, “no signs” reads as add “signs”.

The same clause appeared in all 18 briefs in the book. None rendered with it.

Prevention introduced its own risk.

One validation rule began rejecting valid basketball scenes because its scope was too broad. I narrowed the rule instead of allowing one known failure mode to constrain unrelated creative work.

The goal was not to eliminate failure.

It was to make sure each failure could teach the system something once.

Turn the method into a workflow

Offloading technical overhead without giving up creative control

As ColorPages accumulated more rules, character logic, and model-specific constraints, planning a book became its own technical job.

I wanted to think in stories, characters, and scenes, not remember which phrasing a model rejected, how many figures a composition could support, or which generation route a character required.

So I built the pack planner around a domain-specific agent harness that applies those skills and rules in the background.

I can start with something rough like “Trey goes to a pit stop” or “add some mythical creatures.” The harness checks that idea against the developing book, known model constraints, and accumulated authoring rules, then turns it into a viable plan.

The first planner proved the method

The first version already worked as a real planning system. It could read the live pack, inspect what was already decided, generate candidate pages, prune weak options, budget the compositions, and stage a proposal for review.

But the planning logic still lived inside a long conversational transcript. The method was working, yet the structure of the decisions was not clear enough.

Fig. 05One session in the first planner

One planning session in the first planner, scrolling from my request through the gates, candidate tables, composition budgets and lint pass to the Create card
One session, from my first message to Create.

The next step was to give the method a better shape

I evolved that first planner into a gated planning flow: one decision at a time, with the developing book visible throughout.

The gates made the underlying method legible. Instead of burying the logic inside one long session, the interface shows what has already been decided, what question is open now, and what the final Create action will commit.

Fig. 06One planning session, replayed

Six gates, one asked twice, eight messages from me, and a Create button only I can press.

The harness can stage the plan, but nothing enters production until I press Create.

Decide what not to automate

Testing the boundary between assistance and authority

After building the harness, I tested whether ColorPages should go one step further and predict which generated image I would choose as the final page.

Across 57 scenes, I compared 18 image measurements and seven scoring approaches against selections I had already made.

29.8% Best automated scorer. Top-three match with my selection.
23.7% Random ordering. Top-three match with my selection.

The best approach improved on random ordering by only 6.1 percentage points. That was not enough evidence to delegate final selection.

The system had earned authority over technical constraints, known failure modes, and planning rules. It had not earned authority over editorial judgment.

So I automated compliance, not taste.

Conclusion

ColorPages started as an experiment with local image-generation models. The harder product problem became designing around what those models could not reliably do.

Character identity became an explicit constraint. Small corrections became deterministic tools. Repeated failures became system memory. That knowledge became an agentic workflow with a deliberate boundary around execution.

Next case study

Home Assistant·Notification systems·IoT

Deciding when a smart home should interrupt.

Read case study