ddcolor modelscope in HAVCServerDiT

[Roy-Bertoli]

Hi Dan, would it be possible to insert ddcolor modelscope which has the most saturated colors into HAVCServerDiT?

[Roy-Bertoli]

or you could add a button to make the colors more saturated with 50's technicolor movies.

[dan64]

in the folder GUI\scripts, copy the file encode_cmnet2.vpy in encode_cmnet2_vivid.vpy, then in the new created script, modify the row

clip = cmnet2.vs_cmnet2(clip, method=4, sc_framedir=RefDir, render_speed=RenderSpeed, max_memory_frames=int(MemoryFrames), ref_mode=0)

in

clip = cmnet2.vs_cmnet2(clip, method=4, render_vivid=True , sc_framedir=RefDir, render_speed=RenderSpeed, max_memory_frames=int(MemoryFrames), ref_mode=0)

Then in the GUI "Tab3 - Encode/Merge" select, in the "Encode VPY" text box, the script encode_cmnet2_vivid.vpy instead of encode_cmnet2.vpy

[Roy-Bertoli]

Thank 👍

[dan64]

If you are using the GGUF models, in the README I wrote the following warning: The GGUF backend is experimental. In some cases the frames colors may be faded or little colored.
If you have this problem, a workaround is to manually check the colorized frames in the folder "ref_qwen" and recolor the frames with faded colors using "Fix Image".

[Roy-Bertoli]

yes I use gguf, when I try to install nunchaku-qwen, the weight download etc... it always times out, I let it run for 16 hours but nothing, or tried several times but nothing, I can't install it.

[Roy-Bertoli]

I always check the images in "ref_qwen", and delete the few (2 or 3) in black and white, and those with different colors. The most annoying problem is that it almost never colors, or colors badly, the initial and final titles. With ddcolor modelscope it colored them well.
The colors are good, the stability of the video is incredible, but I really like the colors of 1950s Technicolor movies, I tried entering vivid and saturated colors in the prompt, but it seems to do little or nothing.

[dan64]

To be able to use nunchaku-qwen you need a PC with 64GB+ RAM, without any further detail I can't help.

[Roy-Bertoli]

Ha... ok! I only have 32GB, but I expanded the Windows page file to 128GB. I don't know if that's enough. Even gguf, according to the readme, shouldn't have been enough on 8GB cards; you needed a 12GB card. But it works very well on an 8GB GPU, and I thought the Nunchuk-Qwen would also work if I expanded the page file, even if it was slower.
My PC is like this:
Win 11
R5 7600
GPU RTX 3060 Ti 8GB
GPU 4060 8GB
4TB NVMe disk C:
2TB NVMe disk F:
32GB 6000Hz CL30

[dan64]

The Comfyui GGUF extension has been developed to work with low memory PC. I was able to run it on PC with 16GB RAM and RTX2080 with 6 GB VRAM. But Nunchaku-Qwen was developed for getting max performance on inference (in effect an inference speed of only 4 sec is very impressive). So you need to use the GGUF version, but I don't understand why in your case the GGUF version never colors, or colors badly, the initial and final titles. Could you provide a very small sample (even better with the reference frames colorized using the GGUF version).

[Roy-Bertoli]

I don't think the problem with the colors of the titles is gguf, I think the reference images are not created immediately and the titles are skipped, the first minute of the film should have reference images and the last minute of the film the same, I don't know if it's possible.

[dan64]

I just updated the GUI with the new option "Batch Processing" (see README). In your case, before request the colorization using the standard prompt "color this image, natural colors.", you need to check the box: "Convert in B&W before colorization", otherwise the image will not be recolored because Qwen-IE see that the input image is already colorized. You should get an image like this:

[Roy-Bertoli]

Thanks, Dan, for introducing the batch.
As for the images above, they were colored throughout the entire film with Comnet2, but since they didn't have reference images (because they were skipped by the program that reveals the images), they came out like this. What I meant was whether it was possible to detect more ref images than normal in the first few minutes of the film. I mean only the first 1 or 2 minutes where the titles are, and then proceed with the normal extraction of the ref images. I was wondering if this variability could be introduced into the program that extracts the reference images. I don't know if I can explain myself, unfortunately, my native language is not English, and I use Google to translate.

[dan64]

In effect I noted that there are films where the frames at the beginning and at the end are not extracted (due to small contrast and blending). You could increase the sensitivity by lowering the threshold to 0.025, but there is the risk that are introduced a lot of reference images that are not necessary. My workaround solution is to put the film in Hybrid and using the preview to manually save the missing starting images in the folder ref_tht10 (using the naming convention ref_nnnnnn.jpg), usually are very few not more that 5 or 10,

[Roy-Bertoli]

Hybrid I tried a couple of times to get it to work and I didn't succeed, since now you have a program it would be nice to improve your program, if you could make the extraction of the frames variable with a timer, the first minutes with the 0.025 rate and after with the default 0.035 rate, I don't know if it's possible but it would be useful in the program, in the meantime I will divide the film into 3 pieces, the first of 2 minutes, the second almost the whole film except the last minute and the third 1 minute clip of film, that should work I think.

[dan64]

I added the new script extract_refs_edge_ext.vpy. Update the project as described in README. After having updated the project in the field "Extract VPY" of tab-1--extraction, will be possible select the new script extract_refs_edge_ext.vpy.
With this new script the frames in the first 1% and in the last 1% of the clip, will extracted using high sensitivity threshold (0.02), try this script. If are still missing frames you can modify the script and change the threshold from 0.02 to 0.01.