Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+videos/sample_1_inpaint_10000.mp4 filter=lfs diff=lfs merge=lfs -text
+videos/sample_1_inpaint_2500.mp4 filter=lfs diff=lfs merge=lfs -text
+videos/sample_1_masked_r2v_3000.mp4 filter=lfs diff=lfs merge=lfs -text
+videos/sample_2_masked_r2v_3000.mp4 filter=lfs diff=lfs merge=lfs -text
+videos/sample_3_masked_r2v_3000.mp4 filter=lfs diff=lfs merge=lfs -text
+videos/sample_4_masked_r2v_3000.mp4 filter=lfs diff=lfs merge=lfs -text

README.md

@@ -18,8 +18,9 @@ These LoRAs may cover different use cases over time, so this repository is not l
 
 | File | Description |
 |---|---|
-| `ltx23_inpaint_rank128_v1_02500steps.safetensors` | Sometimes this checkpoint follow the prompt better, probably because it experienced less overfitting. |
-| `ltx23_inpaint_rank128_v1_10000steps.safetensors` | Sometimes this checkpoint doesn't follow instructions quite right, because it focuses more on the size of the mask.; but other than that, it uses the masked region better. This is probably because it experienced more overfitting after a longer training period on a more limited dataset.. |
+| `ltx23_inpaint_rank128_v1_02500steps.safetensors` | Sometimes this checkpoint follows the prompt better, probably because it experienced less overfitting. |
+| `ltx23_inpaint_rank128_v1_10000steps.safetensors` | Sometimes this checkpoint doesn't follow instructions quite right, because it focuses more on the size of the mask; but other than that, it uses the masked region better. This is probably because it experienced more overfitting after a longer training period on a more limited dataset. |
+| `ltx23_inpaint_masked_r2v_rank32_v1_3000steps.safetensors` | Inpainting LoRA with reference support. This model allows inpainting while also using a visual reference, which can help guide the desired replacement more precisely. Prompt quality is extremely important for good results, and mask size matters even more. |
 
 Use whatever suits you best.
 
@@ -40,6 +41,13 @@ must be treated as **a single video**.
 
 After that, you need to use the **`LTXVAddGuideMulti`** node to pass the guide video into the model.
 
+### Required colors
+
+For inference to match the training setup, the colors are important:
+
+- the **mask must be magenta**: **`(255, 0, 255)`**
+- the **green area of the reference must be chroma key green**: **`(0, 255, 0)`**
+
 ## About the mask format used during training
 
 My dataset included samples where the mask was more **blockified**. In other words, the default pattern used **8x8 blocks**.
@@ -50,6 +58,13 @@ To better reproduce the training conditions during inference, you can use:
 
 This may help make the mask distribution closer to what the model saw during training.
 
+For the new reference-based inpainting LoRA, this is especially important:
+
+- sometimes you need to use **blockify** so the mask becomes more agnostic to the previous object's shape
+- sometimes you need to expand the mask to give the new object more room to work properly
+- a good default recommendation is **Blockify Mask** with **size 8**
+- you can expand up to **512**, which effectively makes the mask become a full rectangle
+
 ## Notes
 
 - **Base model:** `Lightricks/LTX-2.3`
@@ -57,6 +72,8 @@ This may help make the mask distribution closer to what the model saw during tra
   - prompt adherence
   - use of the masked area
   - overfitting tendency
+- For the reference-based LoRA, **prompting is extremely important** for result quality
+- For the reference-based LoRA, **mask size and mask preparation are critical**
 
 ## Practical recommendations
 
@@ -64,6 +81,7 @@ For the inpainting LoRAs in this repo:
 
 - If you want **better prompt adherence**, try the **2500 steps** checkpoint first
 - If you want **better use of the masked area**, try the **10000 steps** checkpoint first
+- If you want **an inpainting LoRA that works very well both with a visual reference and in a text-only setup**, try **`ltx23_inpaint_masked_r2v_rank32_v1_3000steps.safetensors`**
 
 The best approach is to compare both in your workflow, since preference may vary depending on the scene, mask, and prompt.
 
@@ -75,38 +93,89 @@ The best approach is to compare both in your workflow, since preference may vary
 
 **Model:** `ltx23_inpaint_rank128_v1_02500steps.safetensors`  
 
-**Video:**  
+**Video:** `videos/sample_1_inpaint_2500.mp4`  
+
+<video src="videos/sample_1_inpaint_2500.mp4" controls></video>
 
 **Prompt:**  
 
 ---
 
-### Example 2
+## Examples — 10000 Steps
 
-**Model:** `ltx23_inpaint_rank128_v1_02500steps.safetensors`  
+### Example 1
 
-**Video:**  
+**Model:** `ltx23_inpaint_rank128_v1_10000steps.safetensors`  
+
+**Video:** `videos/sample_1_inpaint_10000.mp4`  
+
+<video src="videos/sample_1_inpaint_10000.mp4" controls></video>
 
 **Prompt:**  
 
 ---
 
-## Examples — 10000 Steps
+## Examples — Reference Inpainting (R2V) — 3000 Steps
+
+This LoRA can be used in two ways:
+
+- **Reference-guided inpainting**, where the reference image actively guides the replacement
+- **Text-only style inpainting**, by sending a **blank image** as the reference input
+
+A practical issue to keep in mind is **identity leakage**. In some scenes, if the prompt is not specific enough, the model may copy identity traits or visual details from another character already present in the source scene instead of following the intended reference closely. This is especially important for full-body references, so prompt specificity matters a lot.
 
 ### Example 1
 
-**Model:** `ltx23_inpaint_rank128_v1_10000steps.safetensors`  
+**Model:** `ltx23_inpaint_masked_r2v_rank32_v1_3000steps.safetensors`  
+
+**Video:** `videos/sample_1_masked_r2v_3000.mp4`  
 
-**Video:**  
+<video src="videos/sample_1_masked_r2v_3000.mp4" controls></video>
 
 **Prompt:**  
+`A man resembling Donald Trump playing an electric guitar on stage, making energetic performance movements, with a confident pose, expressive body language, and dynamic rockstar attitude. He is holding the guitar dramatically while performing, with stage lighting, motion, and a lively concert atmosphere.`
+
+**Note:** This example used a **full-body reference**. The prompt had to be very specific, otherwise the model would leak identity details from another character already present in the scene instead of following the intended reference. In practice, this means prompt specificity is important to reduce **identity leakage**.
 
 ---
 
 ### Example 2
 
-**Model:** `ltx23_inpaint_rank128_v1_10000steps.safetensors`  
+**Model:** `ltx23_inpaint_masked_r2v_rank32_v1_3000steps.safetensors`  
+
+**Video:** `videos/sample_2_masked_r2v_3000.mp4`  
+
+<video src="videos/sample_2_masked_r2v_3000.mp4" controls></video>
+
+**Prompt:**  
+
+`A nighttime mountain road drift scene with a Tesla Cybertruck performing a dramatic high-speed drift around a sharp curve, sliding sideways across the asphalt with aggressive motion and strong driving energy. The Cybertruck has its iconic angular triangular wedge-shaped body, metallic panels, sharp geometric silhouette, bright headlights, and futuristic design clearly visible. Tire smoke and drift streaks trail behind the vehicle, matching the speed and intensity of the original action scene. Streetlights illuminate the road, the forest background remains dark and dense, and the camera perspective stays low and cinematic, emphasizing motion, speed, and control. The Cybertruck is fully integrated into the scene with realistic scale, lighting, shadows, reflections, and ground contact.`
+
+---
+
+### Example 3
+
+**Model:** `ltx23_inpaint_masked_r2v_rank32_v1_3000steps.safetensors`  
+
+**Video:** `videos/sample_3_masked_r2v_3000.mp4`  
+
+<video src="videos/sample_3_masked_r2v_3000.mp4" controls></video>
+
+**Prompt:**  
+
+`A nighttime mountain road drift scene with a classic Volkswagen Beetle performing a dramatic high-speed drift around a sharp curve, sliding sideways across the asphalt with strong motion and driving energy. The Beetle has its iconic rounded body, compact vintage shape, circular headlights, curved roofline, and unmistakable classic design clearly visible. Tire smoke and drift streaks trail behind the car, matching the speed and intensity of the original action scene. Streetlights illuminate the road, the forest background remains dark and dense, and the camera perspective stays low and cinematic, emphasizing motion, speed, and control. The Beetle is fully integrated into the scene with realistic scale, lighting, shadows, reflections, and ground contact.`
+
+**Note:** This example shows that the **masked R2V model can also be used like a text-only inpainting model without a real reference**. To do that, simply send a **blank image** in place of the reference image.
+
+---
+
+### Example 4
+
+**Model:** `ltx23_inpaint_masked_r2v_rank32_v1_3000steps.safetensors`  
+
+**Video:** `videos/sample_4_masked_r2v_3000.mp4`  
 
-**Video:**  
+<video src="videos/sample_4_masked_r2v_3000.mp4" controls></video>
 
 **Prompt:**  
+`A man riding a tiger on a rural road in daylight, with the tiger rearing upward in a dramatic wheelie-like pose, matching the same dynamic action and position as the original motorcycle. The tiger is large, powerful, and realistic, with orange fur, black stripes, strong muscles, and natural anatomy clearly visible. The man is balanced on top of the tiger as if controlling it during the stunt, with believable body posture and strong motion energy. Keep the same road, camera angle, lighting, shadows, background, and overall composition unchanged. The scene should feel like a real action moment captured outdoors.`

ltx23_inpaint_masked_r2v_rank32_v1_3000steps.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0da59538eaa5737ad030e1459c9b22505696add818a413322cd04911798d220a
+size 327287384

ltx23_inpaint_masked_t2v_rank128_v1_02500steps.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7e8d2082f79be715774026a4fbbddaa2d64154d4dc9b956ade5208cc4dd8adf8
+size 1308756416

ltx23_inpaint_masked_t2v_rank128_v1_10000steps.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1f2089f5cd3cac56f93d3641c90695a7296514aa1292ec8c6f3a6ad369eda728
+size 1308756416

videos/sample_1_inpaint_10000.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:167c53ffcc2243d44b7afa2f0c7234918d945f38dd36be6b6924bc9126ed3bb9
+size 2188724

videos/sample_1_inpaint_2500.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a65e4805d85a2a642ffcd93f6ec88ce748d786cdb596c0856bd1af8902ffd1cc
+size 2194903

videos/sample_1_masked_r2v_3000.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:822cdf8d15875867b3d0638f368d998459ca7d678660166faa04b6cc2767db32
+size 3485848

videos/sample_2_masked_r2v_3000.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ff6fb69316e686d8e85c62a2f294b9596828adeb6fd7620003a7c0138e19c709
+size 2929012

videos/sample_3_masked_r2v_3000.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af20d824f50b257a5a19092fd709d6f35a1175e6b6fd0ef1effa5d3be50fbad9
+size 3005551

videos/sample_4_masked_r2v_3000.mp4

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0923c1ea4075016e4e4219b60adc6663865118de408d43885a1657974b6e389b
+size 3561935