AI Image generation
Automate AI Image Generation with n8n and ComfyUI
This blog post explores integrating AI image generation into your n8n workflows using ComfyUI. Whether you’re looking to automate visual creation or streamline your content generation process, this guide comes from my differents trials and aims to provide a direct solution for integrating ComfyUI into an n8n workflow. In this n8n workflow, we set up an automated system that sends HTTP requests to ComfyUI, monitors the generation status, and retrieves the generated images for further use.
Introduction
For low-code users, this article will explain how to integrate AI image generation into an n8n workflow. This solution leverages the power of open-source and self-hostable setups. In my case, ComfyUI runs on my primary desktop equipped with a recent GPU, while my n8n instance, which I use as a backend for several applications, is self-hosted on my home server.
The ComfyUI documentation is not giving much details about how to set-up and reach the API through HTTP Request but they provide on their Github repository some socket scripts [GitHub – ComfyUI Repository] that give the basic information about how to format the HTTP calls
This blog post will first dive into the detailed setup and explanation of the n8n workflow, followed by a sort of ComfyUI API documentation.
Pre-requisites
- This post assumes that you have both ComfyUI and n8n already running properly on your systems, whether self-hosted or standalone, inside or outside of a Docker environment. This article will not cover installation details, as plenty of resources are available online for that.
- Activate ComfyUI remote access by adding the following launch parameters either at the end of the command use to launch ComfyUI or on windows by modifying with a text editor the file used to launch the application (run_nvidia_gpu.bat or run_cpu.bat at the root of the ComfyUI folder)
--listen
. After restarting the application. ComfyUI will then be running on your computer [COMFYUI_IP_ADRESS] on the port by default 8188 [COMFYUI_PORT] - You have on hand the ComfyUI JSON workflow file of your image generation flow. This JSON file can be obtain from the ComfyUI interface, by clicking on the “Save (API Format)” button on the application menu
n8n workflow
This node allows you to manually start the workflow by clicking on the ‘Test workflow’ in the n8n interface. This node can be replaced either by a webhook call or included in another workflow of yours.
Node Type: Manual Trigger
A custom JavaScript code node is used to set up the parameters and configuration for the image generation process. This includes defining the prompts and settings for the AI model. The code generates a prompt structure necessary for the ComfyUI API to initiate image generation. AS mentioned in the Pre-requisites, you can extract this ComfyUI JSON workflow from the ComfyUI interface, by clicking on the “Save (API Format)” button on the application menu and opening the downloaded file content with a text editor.
We will slightly modified this JSON and define two variables at the top of the JS Script that will need to be recalled inside of the ComfyUI JSON
-
- positiveprompt: This define the prompt to be passed to the AI image generation process. We will use this variable inside of the ComfyUI JSON
- seednumber: We will generate a random seed number in order to randomize the AI image generation. If you use always use the same seed number and repeat the workflow you will always get the same image generated
Node Type: Code
Language: JavaScript
JavaScript code:
// VARIABLE DEFINITION // 1. Positive prompt used for the image generation. If used you also need to manually edit the COMFYUI JSON workflow and insert the variable. const positivePrompt = "A tech-savvy engineer working on a computer with a holographic display, showing AI-generated images and some automation workflow nodes, capturing the essence of integration and automation in a high-tech setting. Accent color should be a warm yellow orange"; // 2. Random seed generation. If you want to randomize the image generation process otherwise you will get the same image for a prompt generation const seednumber = Math.floor(Math.random() * 1000000000000000); const prompt = //BEGINING - PASTE HERE THE COMFYUI JSON WORKFLOW (and insert the above variable call) { "3": { "inputs": { "seed": seednumber, "steps": 50, "cfg": 8, "sampler_name": "euler", "scheduler": "normal", "denoise": 1, "model": [ "4", 0 ], "positive": [ "6", 0 ], "negative": [ "7", 0 ], "latent_image": [ "5", 0 ] }, "class_type": "KSampler", "_meta": { "title": "KSampler" } }, "4": { "inputs": { "ckpt_name": "sd_xl_base_1.0.safetensors" }, "class_type": "CheckpointLoaderSimple", "_meta": { "title": "Load Checkpoint" } }, "5": { "inputs": { "width": 512, "height": 512, "batch_size": 1 }, "class_type": "EmptyLatentImage", "_meta": { "title": "Empty Latent Image" } }, "6": { "inputs": { "text": positiveprompt, "clip": [ "4", 1 ] }, "class_type": "CLIPTextEncode", "_meta": { "title": "CLIP Text Encode (Prompt)" } }, "7": { "inputs": { "text": "", "clip": [ "4", 1 ] }, "class_type": "CLIPTextEncode", "_meta": { "title": "CLIP Text Encode (Prompt)" } }, "8": { "inputs": { "samples": [ "3", 0 ], "vae": [ "4", 2 ] }, "class_type": "VAEDecode", "_meta": { "title": "VAE Decode" } }, "9": { "inputs": { "filename_prefix": "ComfyUI", "images": [ "8", 0 ] }, "class_type": "SaveImage", "_meta": { "title": "Save Image" } } } //END - PASTE HERE THE COMFYUI JSON WORKFLOW ; const jsonData = {"prompt": prompt}; return [{ json: jsonData }];
This node sends a POST request to the ComfyUI API, initiating the image generation process with the specified parameters.
Node Type: HTTP Request
Parameters:
- Method: POST
- URL:
http://COMFYUI_IP_ADRESS:COMFYUI_PORT/prompt
- Body:
- Content-Type: JSON
- Specify Body: Using Fields Below
- Body parameters:
- Name: prompt
- Value:
{{ $('2. Set-up the ComfyUI workflow').item.json.prompt }
The JSON payload generated in the previous step.
This node sends a GET request to the ComfyUI API to check the status of the image generation process using the prompt_id
from the previous node response. We will create a loop with the 2 next nodes WAIT and IF, in order to repeat this status check until the generation process in ComfyUI is finished.
Node Type: HTTP Request
Parameters:
- Method: GET
- URL: Expression
http://COMFYUI_IP_ADRESS:COMFYUI_PORT/history/{{ $('3. ComfyUI HTTP Request').item.json.prompt_id }}
This node evaluates the status of the image generation process status.status_str
coming from the response of the previous node If the status is ‘success’, it proceeds to the next step; otherwise, it waits and rechecks the status later awaiting for the ComfyUI image generation process to be finished..
Node Type: If
Condition:
- Left Value: Expression
{{ $('ComfyUI - Check Image generation status').item.json[$('3. ComfyUI HTTP Request').item.json.prompt_id].status.status_str }}
- Operator: is equal to
- Right Value: success
This node pauses the workflow for a few seconds if the results of the previous status check is not yet completed before rechecking the status of the image generation process. I have set-up this temporization to 10 seconds, by experience as on my system an image generation in ComfyUI takes between 30sec to 50sec depending on the workflow.
Node Type: Wait
Parameters:
- Resume: After time interval
- Wait amount: 10
- Wait unit: Seconds
Once the image generation is successful, this node retrieves the generated image from the API for viewing or further processing.
Node Type: HTTP Request
Parameters:
- Method: GET
- URL:
http://COMFYUI_IP_ADDRESS:COMFYUI_PORT/view
- Query Parameters: Using fields Below, parsed from the response of the previous step
filename
: The filename of the generated image:{{$json[$node["3. ComfyUI HTTP Request"].json.prompt_id].outputs[Object.keys($json[$node["3. ComfyUI HTTP Request"].json.prompt_id].outputs)[0]].images[0].filename}}
subfolder
: The subfolder where the image is stored.{{$json[$node["3. ComfyUI HTTP Request"].json.prompt_id].outputs[Object.keys($json[$node["3. ComfyUI HTTP Request"].json.prompt_id].outputs)[0]].images[0].subfolder}}
type
: The type of the image:{{$json[$node["3. ComfyUI HTTP Request"].json.prompt_id].outputs[Object.keys($json[$node["3. ComfyUI HTTP Request"].json.prompt_id].outputs)[0]].images[0].type}}
The recovery of the filename
, subfolder
,type
parameters from the previous node is a bit tricky as the structure of the returned JSON is dynamic depending on the prompt_id
Comfy UI – API HTTP Request documentation
Base URL: http://COMFYUI_IP_ADRESS:COMFYUI_PORT/ with COMFYUI_IP_ADDRESS the IP address of your ComfyUI instance running on COMFYUI_PORT (By defaut: 8188)
Parameters
Request | Parameter name | Type | Value & Comments |
---|---|---|---|
BODY | prompt | JSON | A JSON ComfyUI workflow exported from the ComfyUI User interface by clicking on the "Save (API Format)" button (Or generated from another of your workflow) |
Response
Code | Description |
---|---|
200 (Success) |
Example:
{ "prompt_id": "782a0bc6-cc9f-4c82-9043-914294a26f8d", "number": 0, "node_errors": {} } |
Parameters
Request | Parameter name | Type | Value & Comments |
---|---|---|---|
URL | prompt_id | String | The unique workflow prompt_id created by ComfyUI and returned during the prompt creation request and returned in the request response |
Response
Code | Description |
---|---|
200 (Success) |
Example with prompt_id = “91fae807-ff1a-4bf0-a493-0c56b0f8fba2”
{ "91fae807-ff1a-4bf0-a493-0c56b0f8fba2": { "prompt": [WORKFLOW STEP DESCRIPTION - DELETED FOR MORE CLARITY ] , "outputs": { "9": { "images": [ { "filename": "ComfyUI_00492_.png", "subfolder": "", "type": "output" } ] } }, "status": { "status_str": "success", "completed": true, "messages": [ [ "execution_start", { "prompt_id": "91fae807-ff1a-4bf0-a493-0c56b0f8fba2" } ], [ "execution_cached", { "nodes": [ "7", "5", "4", "6" ], "prompt_id": "91fae807-ff1a-4bf0-a493-0c56b0f8fba2" } ] ] } } } |
Parameters
Request | Parameter name | Type | Value & Comments |
---|---|---|---|
QUERY | filename | String | The generated image filename created by ComfyUI and returned by the status check request and returned in the request response |
QUERY | subfolder | String | The generated image subfolder created by ComfyUI and returned by the status check request and returned in the request response |
QUERY | type | String | The generated image type created by ComfyUI and returned by the status check request and returned in the request response |
Response
Code | Description |
---|---|
200 (Success) |
An image as a binary file |
Conclusion
This n8n workflow efficiently integrates AI image generation using ComfyUI, allowing for automation and streamlined content creation. By setting up an automated system to send HTTP requests to ComfyUI, monitor the generation status, and retrieve the generated images, you can leverage AI image generation in your projects seamlessly. The approach described in this post serves as a foundation for further customization and integration based on your specific requirements. You can also very easily reproduce this workflow in other automation tools like Make.com
Ressources
n8n: Automate Workflows Easily — n8n is an extendable workflow automation tool, that allows to build powerful workflows, really fast with a low-code approach. Insert code only when you need it. n8n is self-hostable, an extended version is also available on their online platform depending on your need.
- Github project link: https://github.com/n8n-io/n8n
- n8n official website: n8n Website
ComfyUI: A very powerful and modular stable diffusion GUI and backend to personalize, test image and video AI generation workflow. Alternative to AUTOMATIC1111, self-hostable and with a large and active community developing custom nodes for all kind of extra feature support: controlnet, CLIP Encoder…
- Github project link: https://github.com/comfyanonymous/ComfyUI
Try it out and share your experience in comments. Thanks.
thank you
Nice post. Two things to node:
One: the ComfyUI ipadress has to be your computer ip. You can not use 127.0.0.1 or 0.0.0.0.
Two: currently the /view is not working, at least on my system, so retrieving the image fails.
Perhaps because after the task the json is unloaded?
N8n and Comfy local