Introduction
A generated image of a garden is easy to admire. A garden design that tells you which plants to buy, how many of each to order, and where to place them in your actual garden is something you can act on.
That distinction drove how we built Tuindirigent’s AI garden design features. The goal was never to produce attractive pictures in isolation. The goal was to build a connected experience: one where the AI-generated visual, the recommended plant list, and the accompanying planting guidance are aligned, matched to real catalog products, and presented in a way that supports a clearer path from inspiration to practical next steps.
This post explains how we engineered that experience, what consistency challenges we discovered along the way, and how we solved them.
Why AI Garden Design Needs More Than Image Generation
Calling an image generation API and displaying the result is a single step. Building a product around it requires considerably more.
Consider what a useful AI garden design feature actually needs to provide:
-
A visual that reflects the user’s preferences, their garden’s dimensions, and their local growing conditions.
-
A plant list derived from that visual, with each plant verifiable against a real catalog and available to purchase.
-
Enough structured data to generate a practical layout plan that corresponds to the visual.
-
Reliable behavior when the AI returns unexpected output, a catalog match fails, or a generation step takes longer than expected.
-
A frontend that presents the result clearly and lets the user move forward.
None of those requirements are met by image generation alone. The AI model is one component in a larger system, and the system’s design determines whether the feature is genuinely useful or just visually appealing.
Feature Overview
Tuindirigent has two distinct AI garden design flows, each suited to a different starting point.
Inspire Me is built for users who have a garden photo they want to reimagine. The user uploads a photo of their existing garden and describes their preferences: garden style, color palette, maintenance level, sun exposure, and how much creative intervention they want from the AI. The system analyzes the photo, suggests appropriate plants, and produces an edited version of the original image showing what the garden could look like with those plants in place.
Build a Border is built for users who are planning from scratch. Instead of a photo, the user provides the dimensions of their border, their location or postal code, their desired style, and their preferences for color and maintenance. The system uses those inputs to select a curated set of plants, generate a photorealistic visualization of the finished border, and produce a 2D planting plan that shows where each plant should go.
Both flows return a set of recommended plants matched to catalog records, supporting a more direct path from design to ordering.
Architecture Overview
The two flows share infrastructure but differ in how they sequence their steps. Understanding the architecture of each helps explain the consistency challenge described later.
Inspire Me runs in two consecutive AI steps:
-
A Gemini text and vision model receives the garden photo alongside the user’s preferences. It analyzes the image and returns a structured JSON response containing a list of recommended plant species (in scientific naming format) and Dutch-language design notes.
-
A Gemini image generation model receives the original garden photo and the plant suggestions from step one. It produces an edited version of the photo incorporating those plants.
After both AI steps complete, the backend runs a database match: each suggested scientific name is compared against the platform’s plant catalog using PostgreSQL trigram similarity, with a filter requiring at least one in-stock affiliate product. Matched records become the plant cards the user sees alongside the generated image.
Build a Border runs a more structured pipeline:
-
A Gemini text model suggests scientific plant names based on the user’s preferences, border dimensions, and inferred soil and climate conditions for their location.
-
Each suggested name is immediately matched against the plant catalog. Only plants with confirmed in-stock affiliate products are kept. If the matched count falls short of the target species count, the system requests additional suggestions and repeats the match step, up to a bounded number of attempts.
-
Design notes are generated using only the confirmed, matched plants.
-
A photorealistic border image is generated using the matched plant list as the authoritative input, ordered by height so the image reflects a realistic back-to-front planting arrangement.
-
The generated image is sent back to a Gemini vision model, which reads the visual and maps each cell of a planting grid to the plant visible in that position. This derived grid drives the 2D planting scheme.
-
A 2D top-down planting plan is rendered programmatically from the derived grid, showing plant positions, quantities, and a legend.
-
All generated assets (photorealistic image and 2D scheme) are uploaded to cloud storage and returned to the frontend as time-limited signed URLs.
The Build a Border pipeline: catalog matching happens before image generation, and the generated image becomes the reference for the 2D planting scheme layout.
The frontend handles generation asynchronously. After submitting a request, the UI displays a loading state while it polls the backend for the completed result. Once the result is available, the generated image, plant cards, design notes, and planting scheme appear together.
The Main Challenge We Faced
The core tension in any AI design tool that connects to a real product catalog is consistency: the generated image, the plant recommendations, and any accompanying plan should all describe the same garden.
When we built the Inspire Me flow, we discovered a consistency challenge inherent to running two independent AI calls sequentially. The text model suggests a set of plants. The image model receives those names as part of a prompt and produces a visual that interprets them. But image generation models are not strict rendering engines: they produce plausible, visually coherent output that is informed by the plant names in the prompt, but is not bound by them. The resulting image might emphasize certain plants strongly, place them differently than expected, or incorporate stylistic elements that do not map directly to the suggested species.
Separately, the catalog match runs after image generation and filters out any AI-suggested name that does not meet the trigram similarity threshold or lacks an in-stock affiliate product. A user might see a generated image showing eight distinct plants while the plant card section displays five, because three names did not match the catalog with enough confidence.
For Build a Border, there was an additional concern: the 2D planting scheme. In an earlier version of the pipeline, the scheme was generated from a deterministic grid algorithm that had no relationship to the generated image. A user comparing the photorealistic visualization with the top-down plan could see the same plants arranged differently in each, which reduced the practical value of the scheme.
Root Cause: Creative AI Output and Deterministic Catalog Logic
The root cause was structural. Creative image generation and deterministic catalog matching operate on different logic and toward different goals. An image model optimizes for visual plausibility and aesthetic coherence. A product catalog requires exact records: a plant either exists in the database with a purchasable affiliate product or it does not.
Running these two steps without coordination means they can diverge. A plant that looks good in an image may have no catalog record. A plant that has a perfect catalog record may not be what the image model chooses to render most prominently. Connecting a creative AI output to a structured product system requires more than passing a list of names from one to the other.
Prompting alone is not sufficient to solve this. Instruction-following in generative models is probabilistic, not guaranteed. The more reliable approach is to enforce structural constraints at the pipeline level, so that the catalog match and the image generation are not independent steps.
How We Solved It
The solution implemented in Build a Border is a catalog-first pipeline where the catalog match happens before image generation, and the image becomes the authoritative source for the 2D scheme layout.
Here is the core sequence in simplified form:
`preferences = collectBorderPreferences()`\
\
`// Step 1: suggest and match in a loop until we reach the target species count`\
`matchedPlants = []`\
`while matchedPlants.count < targetCount and attemptsRemaining:`\
`suggestions = gemini.suggestPlantNames(preferences, exclude=alreadySuggested)`\
`newMatches = catalog.matchByTrigram(suggestions, requireInStock=true)`\
`matchedPlants.addNew(newMatches)`\
\
`if matchedPlants.isEmpty():`\
`return emptyResult()`\
\
`// Step 2: design notes from confirmed plants only`\
`designNotes = gemini.generateNotes(matchedPlants, preferences)`\
\
`// Step 3: image generated from confirmed plants as authoritative input`\
`imageBuffer = gemini.generateBorderImage(matchedPlants, dimensions, style)`\
\
`// Step 4: vision model reads the image to derive spatial placement`\
`derivedGrid = gemini.deriveGridFromImage(imageBuffer, matchedPlants)`\
`plantingGrid = derivedGrid or fallback to deterministicGrid`\
\
`// Step 5: render 2D scheme from the derived grid`\
`schemeImage = renderPlantingScheme(plantingGrid)`\
\
`return { image, plants: matchedPlants, scheme: schemeImage, notes: designNotes }`The key decisions in this structure:
Suggest and match before generating the image. Because catalog matching happens first, the image prompt is constructed from confirmed, purchasable plants. The user sees plant cards that correspond to what was actually used to generate the image, not a subset filtered afterward.
Design notes reference only matched plants. A separate Gemini call generates the Dutch-language design rationale after the catalog match is complete. This ensures that the notes describe the same plants that appear in the image and the plant cards, without referencing species that did not make it through the match step.
The generated image drives the planting scheme. After generating the photorealistic visualization, the system sends it back to a Gemini vision model with a structured prompt. The vision model reads the image and returns a grid mapping, identifying which plant occupies each cell based on what is visually present. The 2D scheme is then rendered from that derived mapping rather than from an independent algorithm. If the vision-based derivation does not return a usable result, the system falls back to a deterministic grid layout computed from planting density data in the flora records.
Fallback behavior is explicit. If the catalog match returns no confirmed plants after the retry loop completes, the pipeline returns an empty result and the frontend shows an appropriate state. If image generation fails, the plant cards and design notes are still returned. If the vision-based grid derivation fails, the scheme uses the deterministic fallback. Each failure mode degrades gracefully without blocking the rest of the response.
Before and after: moving from independent AI steps to a catalog-first pipeline where the generated image, plant list, and 2D scheme are grounded in the same matched plant data.
Business Value
The catalog-first approach has practical consequences beyond engineering quality.
The plant list is grounded in real catalog data. The system is designed so that recommended plants are matched to affiliate catalog records with in-stock products, reducing the gap between the AI suggestion and what a user can actually purchase.
The visual and the plan are designed to tell the same story. Because the planting scheme is derived from the generated image, users can compare the photorealistic visualization with the top-down plan and find more consistent plant placement. This makes the scheme more useful in practice.
Structured design notes add real context. AI-generated notes that reference the matched plants help users understand the design rationale, flowering periods, height layering, and maintenance requirements of the plants they are viewing.
The retry mechanism improves result quality automatically. When a suggested scientific name does not match a catalog record, the system requests a replacement without interrupting the user experience. The result is a more complete plant selection rather than a partial one.
The feature extends naturally to new partners and catalogs. The catalog match is domain-aware: it can be scoped to a specific affiliate’s product catalog. Adding a new partner means pointing the match step at a new catalog scope, with no changes to the AI pipeline itself.
The architecture reduces manual design effort for the user. Users who previously had to research, select, and quantity-plan their own planting borders can arrive at a catalog-matched, positioned plant list in a single session.
Final Result
The Build a Border feature now delivers a more cohesive AI design experience. The architecture is designed so that the photorealistic image, the recommended plant cards, the design notes, and the 2D planting scheme all reflect the same catalog-matched plant selection. Users can move from entering their preferences to reviewing a complete design concept with a corresponding plant list in one flow.
The Inspire Me flow follows a different architecture because it starts from an existing user photo rather than a blank canvas. The grounding approach developed for Build a Border gives us a strong foundation for future improvements across photo-based garden design workflows.
Both features integrate with the platform’s subscription and credit system, are stored per conversation, and are accessible from the user’s project workspace for review and planning.
Lessons Learned
Image generation is one step in a product, not the product itself. The engineering effort required to make AI-generated garden visuals genuinely useful was larger than the image generation step. Catalog matching, structured data handling, layout derivation, fallback logic, and frontend presentation each required careful design.
Ground AI output in real application data before generating visuals, not after. Running the catalog match before image generation means the image is built from confirmed data. Running it after means the image may represent plants that do not exist in the catalog. The order of operations has significant consequences for the user experience.
Use the AI model’s own output as a reference for derived artifacts. The most accurate way to determine where plants are placed in a generated image is to ask a vision model to read the image and describe it. A separate algorithmic approach cannot match what the image model actually rendered. This pattern, where one model’s output becomes structured input to another, is broadly applicable in multi-model pipelines.
Prompt engineering matters, but system architecture matters more. Carefully constructed prompts improve AI output quality. But structural decisions, like when catalog matching runs, whether design notes are generated from confirmed or unconfirmed data, and whether the planting scheme derives from the image or runs independently, have a larger and more reliable effect on consistency.
Fallback behavior is part of the feature. AI pipelines have more failure modes than deterministic software. Designing explicit fallbacks for each step, rather than treating failures as exceptional, makes the experience reliable for users even when individual steps do not produce expected output.
Connecting creative AI output to actionable user journeys is where the business value lives. A visually appealing garden image has limited practical use on its own. The same image paired with a matched plant list, a layout plan, and direct links to purchasable products creates a user journey with a clear next step.
Conclusion
The engineering challenge in building Tuindirigent’s AI garden design features was not making Gemini generate an attractive garden image. That part, while technically interesting, was a single step. The real challenge was building a system around that step that makes the output trustworthy and actionable.
When a user finishes a Build a Border session, they have a photorealistic image of their border, a set of plant cards they can add to a shopping list, a Dutch-language design rationale explaining the planting choices, and a top-down scheme showing where each plant should go. The architecture is designed so that all four outputs reflect the same plant selection: catalog matching happens before the image is generated, the image becomes the reference for spatial layout, and every user-visible output traces back to the same set of catalog-matched plants.
That is the engineering value of treating AI image generation as one step in a larger product system, rather than the product itself.