Replicate's image model benchmarks: SDXL for quality, Kandinsky for speed

Replicate's comparison of leading open-source models reveals clear winners for inpainting and instruction-based edits. We analyze their findings on quality, latency, and cost for founders building AI features.

THE ANSWER UP FRONT

For product teams requiring the highest fidelity inpainting, Replicate's data points to SDXL Inpainting as the clear choice. If your application prioritizes speed and lower cost for iterative edits, Kandinsky 2.2 is the superior option. For simple, prompt-based changes like "make the dress blue," InstructPix2Pix remains a reliable workhorse. The bottom line from this comparison is that the best model is task-specific. There is no single winner for all image editing use cases. Teams should budget for and test multiple models against their specific product requirements.

METHODOLOGY

This v0 review analyzes the findings from Replicate's blog post, "Which image editing model should I use?", published in early July 2026. The analysis is based entirely on the performance and cost claims presented by Replicate, a commercial platform that hosts the models. Independent benchmarks are pending. Replicate's methodology involved testing four prominent open-source models: stability-ai/sdxl-inpainting, ai-forever/kandinsky-2.2, timbrooks/instruct-pix2pix, and advanceto/lama-cleaner. The models were benchmarked on three distinct tasks: inpainting (removing an object), outpainting (extending an image), and instruction-based editing (modifying an image based on a text prompt). All tests were reportedly run on NVIDIA A100 (80GB) GPUs. This review covers Replicate's reported results on output quality, inference speed, and cost-per-edit. It does not cover long-term workflow integration, performance on other hardware, or the process of fine-tuning these models.

WHAT IT DOES

Inpainting: SDXL's quality vs. LAMA's speed

Replicate tested inpainting by removing a person from a detailed street photo. The results showed a clear trade-off. SDXL Inpainting produced the highest-quality output, seamlessly reconstructing complex backgrounds and textures with minimal artifacting. However, it was also the slowest and most expensive model tested for this task. In contrast, LAMA (Large Mask Inpainting) was significantly faster and cheaper but sometimes struggled with high-frequency details, producing slightly blurred or smoothed-out results in the filled area. For applications where photorealism is non-negotiable, like product photography editing, SDXL is the reported winner. For less critical tasks, LAMA offers a viable, high-speed alternative.

Outpainting and instruction-following

For outpainting, or extending an image's canvas, Kandinsky 2.2 reportedly performed best. It generated coherent and contextually appropriate extensions to a landscape photograph, maintaining the original style. SDXL also performed well but with higher latency. The most distinct category was instruction-based editing, where InstructPix2Pix was the primary model tested. Given a prompt like "add sunglasses to the cat," the model successfully applied the change while preserving the rest of the image. Replicate's analysis notes that the model's effectiveness is highly dependent on prompt clarity and works best for contained, specific modifications rather than broad stylistic changes.

WHAT'S INTERESTING / WHAT'S NOT

The most valuable takeaway from Replicate's comparison is the clear quantification of the quality-speed-cost trade-off. Founders can map these benchmarks directly to their product's needs. For a freemium photo editor, using a fast, cheap model like LAMA for a "quick fix" tool makes sense. For a professional-grade service, the higher cost and latency of SDXL Inpainting is a necessary investment for superior output. This makes the model choice a strategic product decision, not just a technical one.

What's missing is a direct comparison to leading proprietary APIs. While this is a review of open-source models on Replicate, many founders are choosing between this stack and APIs from Adobe, OpenAI, or Google. Without that context, it's hard to gauge the true state of the open-source ecosystem. Furthermore, the benchmarks focus on single-shot edits. Many real-world applications involve a chain of operations (e.g., outpaint, then inpaint, then apply a style). The performance and coherence of these models in a complex workflow remains an open question that these isolated tests do not answer.

PRICING

Pricing on Replicate is based on the GPU time used for an inference. The reported costs are an estimate based on average run times for the benchmarked tasks.

• SDXL Inpainting (A100 80GB): ~$0.0055/sec. Average inpaint: 10-15 seconds. Cost: ~$0.055 - $0.083 per edit.
• Kandinsky 2.2 (A100 80GB): ~$0.0055/sec. Average edit: 4-6 seconds. Cost: ~$0.022 - $0.033 per edit.
• InstructPix2Pix (A100 80GB): ~$0.0055/sec. Average edit: 3-5 seconds. Cost: ~$0.017 - $0.028 per edit.
• LAMA Cleaner (A100 80GB): ~$0.0055/sec. Average inpaint: 2-3 seconds. Cost: ~$0.011 - $0.017 per edit.

Pricing snapshot from July 7, 2026. Costs are illustrative and vary with image size and complexity.

VERDICT

Replicate's benchmarks provide a clear, task-oriented guide for selecting an image editing model. For teams building products where output quality is the primary concern, SDXL Inpainting is the correct choice, and its higher cost should be factored into the business model. For features that require interactivity and speed, such as real-time previews or rapid content generation, Kandinsky 2.2 provides a balanced compromise. InstructPix2Pix is a specialized tool for prompt-based edits and should be used for that specific purpose. There is no "best" model, only the right model for the job. The prudent approach is to select a primary model based on your core use case and potentially use a cheaper, faster model for secondary, lower-stakes features.

WHAT WE'D TEST NEXT

A v1 of this review would require independent verification of Replicate's performance claims. We would also design a test suite for multi-step, chained editing workflows to assess model compatibility and cumulative artifacting. A crucial next step is a qualitative comparison against closed-source competitors like Adobe Firefly's Generative Fill and DALL-E's editing API, including a panel for human preference scoring on the outputs. Finally, we would benchmark the performance uplift and cost implications of fine-tuning these models on a specific domain, such as e-commerce product images or architectural renderings.

The investor read

The key signal here is the commoditization of powerful, specialized AI models. The market is not converging on a single "Photoshop killer" but is fragmenting into a suite of task-specific tools. This creates opportunities for startups building opinionated, workflow-centric applications for specific verticals (e.g., real estate, fashion, advertising) by abstracting away the model choice for the user. Replicate itself is a strong infrastructure play, positioning itself as the neutral benchmark provider and model supermarket. The most investable companies in this space will not be those building base models, but those creating a superior user experience and workflow layer that leverages the best open-source model for each sub-task, likely passing the cost-per-edit directly to the customer.