PAN2 ComfyUI models and nodes
retained text
Edit: I also edded experimental lora training to my trainer on this branch: https://github.com/speedyrulz/RulZ-AI-Tools-Suite/tree/claude/rulztrainer-pan2 I'm testing that now. --- I wanted to try PAN2 and got impatient, so I converted the models and vibecoded a ComfyUI update with new nodes to add compatibility for PAN2. Here is my fork of ComfyUI that supports PAN2: [https://github.com/speedyrulz/ComfyUI](https://github.com/speedyrulz/ComfyUI) Models and workflows: [https://huggingface.co/speedyrulz/PAN2-ComfyUI/tree/main](https://huggingface.co/speedyrulz/PAN2-ComfyUI/tree/main) The example video here was generated is 5 seconds at 1MP and took about 10 minutes then it was frame interpolated to 48fps. \## PAN 2 in ComfyUI: support, quantized models, and speedups I added PAN 2 (IFM's 29B video model) support to a ComfyUI fork, quantized it down to sizes that fit consumer cards, and added two nodes that make it a lot faster. Everything was tested on an RTX 5060 Ti 16 GB. \### What was added to ComfyUI \- \*\*Model code\*\* for the PAN 2 transformer (44 joint blocks + 4 text refiner blocks), with auto-detection in Load Diffusion Model \- \*\*Text encoder:\*\* a new \`pan2\` type in Load CLIP for the Qwen3.5-9B encoder \- \*\*VAE:\*\* Load VAE now reads the Wan 2.2 VAE in the layout PAN 2 ships \- \*\*New nodes:\*\* \- PAN 2 Text Encode \- PAN 2 Negative Prompt (builds the model's structured default negative) \- PAN 2 Image To Video (also does text-to-video with no start image) \- PAN 2 Spectrum \- PAN 2 Mem Eff Sage Attention \- \*\*Fix:\*\* splitting the positive/negative passes across two GPUs used to crash \### Quantized models The original release is about 81 GiB. Converted to single files: | File | Size | Format | |---|---|---| | PAN2\_int8\_convrot | 39.8 GiB | int8 ConvRot, fp32 norm layers kept | | PAN2\_int8\_convrot\_full | 27.4 GiB | int8 ConvRot, norm layers quantized too | | PAN2\_nvfp4 | 15.5 GiB | NVFP4 (4-bit) | | Qwen3.5 text encoder, int8 ConvRot | 8.4 GiB | | | Qwen3.5 text encoder, NVFP4 | 4.6 GiB | | | VAE fp16 | 1.3 GiB | | \- Only the large weight matrices are quantized; everything else is copied unchanged. \- int8 ConvRot: about 0.9% weight error. \- NVFP4: about 9% weight error. Runs natively on RTX 50 series cards. \- Conversion scripts are included, so you can build the files yourself from the official release (CPU only, no GPU needed). \### Speed (RTX 5060 Ti 16 GB, NVFP4 model) | Change | Test | Before | After | |---|---|---|---| | Spectrum node | 640x352, 49 frames, 50 steps | 149 s | 58 s | | Mem Eff Sage Attention node | 1248x704, 121 frames | 42.2 s/step | 25.7 s/step | | Mem Eff Sage Attention node | 640x352, 49 frames | 3.03 s/step | 2.75 s/step | | Feed-forward slicing (int8 full model) | 1248x704, 121 frames | 47.7 s/step | 43.8 s/step | \- \*\*Spectrum\*\* forecasts some steps instead of running the model: with default settings only 15 of 50 steps run it. The output differs from a normal run by about as much as ordinary numerical drift. \- \*\*Mem Eff Sage Attention\*\* needs the \`sageattention\` package. The gain is biggest at full resolution. \- \*\*Two GPUs:\*\* splitting CFG across a 5060 Ti and a 4060 Ti works, but was slower than the 5060 Ti alone. \### Tips \- Recommended settings: 50 steps, CFG 2.5, euler / simple, 24 fps. \- PAN 2 expects a structured JSON prompt. In one test, a plain-text negative prompt made the video look clearly worse than the default structured negative.
document fetched 2026-11-12 00:32 · canonical https://reddit.com/r/StableDiffusion/comments/1xtl14x/pan2_comfyui_models_and_nodes