Revised December 12, 2023 (to include Meta – Imagine)
AI Image Generator Crash Test
AI image generators have gained significant popularity in recent years, revolutionizing the way we create visual content. These advanced algorithms leverage deep learning techniques to generate images that are remarkably realistic and visually stunning. From creating lifelike photographs to imaginative illustrations, AI image generators have become indispensable tools for artists, designers, and content creators.
With the increasing number of AI image generators available, it has become crucial to test and compare their performance to identify the best one for specific creative needs. This article aims to conduct a comprehensive crash test among the top image generators, putting them through their paces using the same prompt. By evaluating their results side by side, we can gain valuable insights into their capabilities and determine which one stands out as the most impressive.
The objective of this article is to provide an unbiased assessment of various AI image generators and shed light on their strengths and weaknesses. By subjecting them to identical prompts, we can objectively evaluate their performance across different styles, such as photographs, illustrations, and abstract art. Through this crash test, we hope to guide you in making informed decisions when choosing an AI image generator for your creative endeavors.
Methodology
For our crash test of the top AI image generators, we wanted to select contenders that are widely used and accessible to most users. As such, we chose to test :
- DALL-E 2by and DALL-E 3 by OpenAI: OpenAI’s DALL-E models are currently at the forefront in the field of AI-driven image generation. DALL-E 2, renowned for its advanced capabilities and impressive outputs, is broadly available through OpenAI’s free tier. On the other hand, the more advanced DALL-E 3 is reserved for plus subscribers, offering even more sophisticated image generation features. These systems were chosen for their prominence in the AI landscape and their ability to generate high-quality images from text prompts.
- Midjourney: Gaining significant traction in 2023, Midjourney has become a popular choice for AI image generation. Its unique integration with Discord and a free tier offering limited monthly image generations have contributed to its widespread use. Midjourney’s ability to produce artistically compelling images with a distinct style made it a crucial inclusion in our tests.
- Bing Image Creator: As a mainstream search engine’s offering, Bing Image Creator provides a user-friendly platform for AI image generation. Its accessibility to a large audience and integration with Bing’s search technology offered a different perspective on the capabilities of AI-driven image creation.
- Stable Diffusion XL Playground: Known for its open-source nature, Stable Diffusion has rapidly gained a reputation for flexibility and high-quality image generation. The tool’s free access and the ability for users to run it on their own hardware underscore its appeal to a tech-savvy audience, making it an essential part of our evaluation.
- Clipdrop by stability.ai: Clipdrop stands out for its innovative approach to image generation and editing. Developed by Stability AI, it offers unique features for real-time image capture and manipulation, blending AI with practical utility. This tool’s versatility and ease of use made it a key contender in our assessment.
- Adobe Firefly: As Adobe’s entry into the AI image generation space, Firefly brings the brand’s longstanding expertise in creative software to the AI realm. Known for its intuitive interface and integration with Adobe’s ecosystem, Firefly targets both creative professionals and general users, offering a distinct approach to AI-driven image creation.
- Leonardo.ai:This AI generator, though less known than some of its counterparts, has shown promise in delivering high-quality images with a focus on artistic expression. Leonardo.ai’s inclusion in our test was driven by its potential to offer unique insights into the evolving capabilities of AI image generators.
- Meta’s Imagine: Joining the ranks of innovative AI image generation tools is Meta’s latest offering, “Imagine.” Developed by the social media and technology giant Meta, Imagine leverages the company’s extensive research and development in AI to create a tool that is both powerful and user-friendly. Designed to cater to both creative professionals and enthusiasts, Imagine stands out for its ability to integrate seamlessly with Meta’s suite of products and services. Its unique selling point lies in the integration of social media insights, allowing for image generation that’s not only visually striking but also socially relevant and engaging. With Imagine, Meta aims to democratize AI-driven creativity, making it accessible to a wider audience and adding a new dimension to the field of AI-powered image generation.
Each of these tools was selected for its unique features, accessibility, and potential to provide a comprehensive overview of the current state of AI image generation. Our testing methodology was designed to assess the strengths and limitations of these varied platforms in a range of challenging scenarios.
Prompts and Testing Process
We are going to give five different types of prompts to ensure diverse styles and evaluate the versatility of image generators:
- Photograph Style: “A photo of a busy city intersection at night with neon signs and many cars”
- Photography of a famous person: “A photo of Che Guevara visiting Acropolis, Athens Greece”
- Illustration Style: “An illustration of a robot walking a dog in a futuristic city”
- Abstract Style: “Produce an abstract artwork inspired by vibrant colors and geometric shapes.”
- Realistic Object: A photorealistic image of a bowl of ramen noodles
These prompts were chosen to evaluate performance across different image styles.
We used the default model and settings for each AI system, as we wanted to evaluate their general “out of the box” capabilities (for Leonardo it is DreamShaper_v7).
The prompts were entered into each image generator through their standard user interface just as any typical user would submit them, without making any other special selection.
Each AI generator was given the same set of 5 prompts one at a time. We allowed them to generate the default number of images per prompt as a reasonable sampling to choose from. If the model generated more than one image, we selected (and presented) the best (you have to trust our objectivity and … taste on this).
We omitted any error messages or failed generations, only collecting successfully completed images.
With a consistent methodology using the exact same prompts given to each AI image generator under their default conditions, we could closely compare the performance and output of the top image generators. This head-to-head crash test allows us to crown an overall winner.
Prompt 1: Photograph Style
Analysis of Image Generation Results
Overall, this prompt category (”A photo of a busy city intersection at night with neon signs and many cars”) brought out the best across the board, with several AIs proving capable of near-photorealistic generation.
Adobe Firefly and Bing produced average images that failed to stand out.
DALL-E 2 gave one of its strongest showings from all the prompts we tried using this model, generating images with good detail and lighting.
DALL-E 3 impressed with its image quality, although its perfection verged on surpassing photorealism.
Clipdrop, Midjourney, and Leonardo emerged as top contenders, producing equally high-quality results, each with different strong points.
Leonardo delivered a very balanced image, with excellent neon sign lettering, a natural photographic style, and good clarity.
Midjourney stumbled slightly on the lettering, but its images had a captivating “wow factor” with a razor-sharp focus on the wet, nighttime street.
Stable Diffusion and Meta’s Imagine generated solid yet unremarkable images, lacking artistic flair and intricate details.
Clipdrop generated uniquely stylized images with an eye-catching retro 1970s aesthetic. Minor faults like lettering issues and unrealistic cars were forgivable given the artistic twist that grabbed attention.
Prompt 2: Photography of a famous person
Analysis of Image Generation Results
This famous person photography prompt revealed limitations and weaknesses in many of the AI image generators.
Adobe Firefly failed epically, producing completely inaccurate images.
DALL-E 2 also struggled, with its images not matching the prompt well.
DALL-E 3 produced an intriguing result by blending styles; it represented Che Guevara in a painting superimposed on a photorealistic image of the Acropolis, though failing to achieve complete photorealism.
Meta’s Imagine result was quite impressive within the same style mixture, though overall better than DALL-E 3. However, it still fell short of achieving complete photorealism.
Bing ran into content restrictions, blocking the prompt altogether (!!!).
Leonardo generated a somewhat interesting stylized young Che Guevara but lacked photorealism (unless Che himself had chosen heavy Instagram filters!).
If not for side issues like unrealistic hands and background figures, Stable Diffusion could have produced near-perfect images, excellently rendering both the Acropolis and Che Guevara.
Clipdrop and Midjourney achieved the best results, with Clipdrop as the winner for its superior representation of both Che Guevara and the Acropolis.
Midjourney did not depict the Acropolis at all, though it did convey a realistic overall feeling of Guevara visiting in the image. We have to admit Midjourney’s result was impressively realistic in general.
This prompt challenged the AIs’ capabilities, exposing weaknesses in generating believable images of (famous) people in specified locations. Only Clipdrop and Midjourney succeeded by staying faithful to the prompt and maintaining photorealism.
Prompt 3: Illustration Style
Analysis of Image Generation Results
The illustration prompt produced varying degrees of success among the AI image generators.
DALL-E 2 generated a simplistic illustration that could have been interesting if not for multiple artistic failures.
Bing produced a decent result with many flaws upon closer inspection.
Adobe Firefly failed to properly depict the dog.
Stable Diffusion came very close to a winning result with an artistically intriguing image, but fell short on the dog representation.
Clipdrop was artistically unique, with a disproportionately large dog compared to the small figure in the back. Also, the robot did not seem to actually be walking the dog.
Leonardo, Midjourney, and DALL-E 3 emerged as the top performers in this test.
Midjourney excellently balanced all elements of the prompt, resulting in a cohesive and well-executed illustration.
DALL-E 3 stood out for its accuracy to the prompt and the luminous quality of its artwork.
Meta’s Imagine provided a result that was very accurate to the prompt but more simplistic, lacking the illustrative details generated by Midjourney, DALL-E 3, or Leonardo.
In this test, Midjourney and DALL-E 3 share the top spot, each delivering excellent results but in distinctly different artistic styles, showcasing their unique strengths in visual interpretation.
Prompt 4: Abstract Style
Analysis of Image Generation Results
This abstract art prompt proved to be an easier challenge that most generators handled well.
Bing and Stable Diffusion produced images with vibrant colors but lacked artistic interest or variety in geometric shapes, resulting in flat outcomes.
DALL-E 2 generated a unique design absent bright colors.
DALL-E 3 excelled, generating an image that precisely matched the prompt’s request for a blend of geometric shapes in bright colors.
Meta’s Imagine generated a very interesting color combination but failed in terms of geometric shapes, resulting in a messy outcome.
Clipdrop blended shapes and colors in an appealing way.
Midjourney, Adobe Firefly, and Leonardo had the best results, with Midjourney and Leonardo creating very similar abstract pieces.
Surprisingly, Adobe Firefly, which failed on most other prompts, shone here by producing the most interesting and unique result. Its abstract image stood out with bold geometric forms and vivid yet balanced colors.
This exercise highlighted the varying artistic interpretations among the AI generators. While Adobe Firefly made a notable comeback, Midjourney and Leonardo continued their streak of quality outputs, and DALL-E 3 emerged as a clear standout. In contrast, others showed a need for greater artistic flair.
Overall, generating captivating abstract art from this prompt seemed well within most models’ technical capabilities.
Prompt 5: Realistic Object
Analysis of Image Generation Results
In this photorealistic food prompt, we saw mostly good results, one failure from DALL-E 2, and a clear winner in Midjourney.
Bing fell short on achieving true photorealism.
Adobe Firefly, which struggled on some other prompts, generated a quality image here – very photorealistic but with an unnaturally piled mound of noodles.
Clipdrop came close to perfection but had oddly uniform eggs.
Similarly, Stable Diffusion lacked realism in its depiction of the ingredients.
Meta’s Imagine, DALL-E 3 and Leonardo impressed with their renditions of a photorealistic bowl of ramen, offering a high level of detail and visual appeal.
Midjourney flawlessly rendered the noodles, broth, eggs, and garnishes in an indistinguishable-from-reality way. While most generators capably produced appetizing images, only Midjourney achieved full authentic photorealism for this prompt. Its ability to generate natural food scenes gave it the win for this final test.
AND THE BEST AI IMAGE GENERATOR IS
The clear winner across all the prompts tested is Midjourney. Their algorithm consistently produces good to perfect images, regardless of style or prompt. No other image generator came close to Midjourney’s level of reliable, quality results.
DALL-E 3, Clipdrop, Stable Diffusion, Meta’s Imagine and Leonardo also produced mostly good outputs, but they only occasionally achieved near-perfection, while still failing on some details or parameters. These models are decent AI art generators, but ultimately fall short of Midjourney’s prowess.
Midjourney reigns supreme as the undisputed king among the image creators. While the other AIs have strengths in certain areas, none could match Midjourney’s versatility, attention to detail, and ability to excel in photographic, illustrative, abstract, and photorealistic styles. Our extensive head-to-head comparisons prove Midjourney is in a class of its own when it comes to AI-generated art.
