Add a Diffusers usage section (Modular Diffusers custom blocks)

#2
by multimodalart HF Staff - opened
Files changed (1) hide show
  1. README.md +62 -0
README.md CHANGED
@@ -252,6 +252,68 @@ language you request. `target_aspect_ratio` is a positive-integer `W:H` string.
252
  If you select an explicit ratio, the model preserves it exactly. If you select
253
  `adaptive`, the model determines a numeric ratio for the intended edited image.
254
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
255
  ## License
256
 
257
  - MiniMax-H3-Image-PE is part of MiniMax-H3-Image and is released under the [MiniMax-H3-Image Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3-Image/blob/main/LICENSE).
 
252
  If you select an explicit ratio, the model preserves it exactly. If you select
253
  `adaptive`, the model determines a numeric ratio for the intended edited image.
254
 
255
+ ## Use with Diffusers
256
+
257
+ MiniMax-H3-Image-PE can also run locally in Transformers through the
258
+ [`diffusers/minimax-h3-image-pe-blocks`](https://huggingface.co/diffusers/minimax-h3-image-pe-blocks)
259
+ Modular Diffusers blocks. The blocks use the released prompts and request
260
+ parameters for T2I PE and Image editing PE, and XGrammar constrains the answer
261
+ to the JSON schema.
262
+
263
+ ### Load MiniMax-H3-Image-PE
264
+
265
+ ```bash
266
+ pip install -U diffusers transformers xgrammar bitsandbytes kernels flash-linear-attention
267
+ ```
268
+
269
+ ```python
270
+ import torch
271
+ from diffusers import ModularPipeline
272
+
273
+ pe = ModularPipeline.from_pretrained("diffusers/minimax-h3-image-pe-blocks", trust_remote_code=True)
274
+ pe.load_components(dtype=torch.bfloat16, device_map="cuda")
275
+ ```
276
+
277
+ The FP4 weights stay in 4-bit and use about 19 GiB of GPU memory. GPUs with
278
+ native FP4 support run the NVFP4 weights directly; other GPUs use bitsandbytes
279
+ FP4.
280
+
281
+ ### Expand a prompt
282
+
283
+ Pass a request to run T2I PE:
284
+
285
+ ```python
286
+ state = pe(prompt="An orange tabby cat sleeping on a sunlit windowsill.")
287
+ print(state.get("prompt"), state.get("aspect_ratio"))
288
+ ```
289
+
290
+ Pass reference images to run Image editing PE. Refer to the images in their
291
+ supplied order:
292
+
293
+ ```python
294
+ state = pe(
295
+ prompt="Replace the bag in the first image with the bag in the second image.",
296
+ image=["source.png", "bag.png"],
297
+ aspect_ratio="adaptive",
298
+ )
299
+ print(state.get("prompt"), state.get("aspect_ratio"))
300
+ ```
301
+
302
+ ### Inputs and outputs
303
+
304
+ | Name | Description |
305
+ | --- | --- |
306
+ | `prompt` | Input: the request or editing instruction. Output: the expanded caption. |
307
+ | `image` | One or more reference images (PIL images, paths or URLs). Selects Image editing PE. |
308
+ | `aspect_ratio` | Input, Image editing PE only: `adaptive` or an explicit `W:H` ratio. Output: the target ratio. |
309
+ | `enable_thinking` | Reasoning before the answer, `True` by default as in the released request. |
310
+ | `max_new_tokens` | Token budget, including thinking and the final answer. |
311
+ | `seed` | Seed for a reproducible expansion. |
312
+ | `reasoning` | Output: the model's reasoning. |
313
+
314
+ Activations run in bf16, so outputs are not identical to the SGLang deployment
315
+ above.
316
+
317
  ## License
318
 
319
  - MiniMax-H3-Image-PE is part of MiniMax-H3-Image and is released under the [MiniMax-H3-Image Community License Agreement](https://huggingface.co/MiniMaxAI/MiniMax-H3-Image/blob/main/LICENSE).