Upload 5 files

README.md

@@ -1,14 +1,53 @@
----
-title: GestureLSM
-emoji: 🐨
-colorFrom: yellow
-colorTo: pink
-sdk: gradio
-sdk_version: 6.0.1
-app_file: app.py
-pinned: false
-license: mit
-short_description: Project
----
+# Deploying GestureLSM to Hugging Face Spaces
+
+This directory contains a minimal scaffold for running the `demo.py` Gradio UI inside a Hugging Face Space. Copy the files into a new Space repository (or push this folder as-is) and provide the model checkpoints via the Hugging Face Hub so the app can download them at startup.
+
+## 1. Create the Space
+1. In your Hugging Face account, click **New Space**.
+2. Choose a Space name, set **SDK** to **Gradio**, and select **CPU Basic** hardware.
+3. Leave the default visibility or mark it **Private** while testing.
+
+## 2. Populate the Space repository
+Upload the following from this folder to the Space:
+
+- `app.py` – boots the Gradio interface, downloads weights if available, and ensures output folders exist.
+- `requirements.txt` – Python dependencies.
+- `packages.txt` – system packages (ffmpeg + openfst).
+- `prepare_assets.py` (optional helper described below).
+- Any configs, sample audio, and auxiliary data your demo needs (e.g. `configs/`, `demo/examples/`, `mean_std/`).
+
+> **Tip**: keep the repository lightweight. Large checkpoints should live in a separate dataset repo and be fetched at runtime.
+
+## 3. Host the checkpoints
+1. Create a private **dataset** repo on Hugging Face (e.g. `username/gesturelsm-assets`).
+2. Upload the required files:
+   - `ckpt/net_300000_upper.pth`
+   - `ckpt/net_300000_lower.pth`
+   - `ckpt/net_300000_hands.pth`
+   - `ckpt/net_300000_lower_trans.pth`
+   - `ckpt/net_300000_upper.pth`
+   - `ckpt/new_540_shortcut.bin`
+   - `mean_std/*.npy`
+3. In your Space’s **Settings → Variables and secrets**, add a variable named `HF_GESTURELSM_WEIGHTS_REPO` with the value of the dataset repo (for example `username/gesturelsm-assets`).
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+When the Space boots, `app.py` will call `snapshot_download` to pull everything into `ckpt/`, preserving the original directory layout expected by `demo.py`.
+
+## 4. Optional asset preparation script
+If you need to perform additional setup (e.g. copying assets after download), you can push the provided `prepare_assets.py` and call it from `app.py` or `__init__.py` before launching the interface. Modify it to match your workflow.
+
+## 5. Verify locally
+Before pushing, test with the same layout on your machine:
+
+```bash
+conda activate gesturelsm
+pip install -r hf_space/requirements.txt
+python hf_space/app.py
+```
+
+Ensure the UI launches and generates outputs using locally stored checkpoints.
+
+## 6. Push & run
+Commit and push the Space repository. After the build completes, the public URL will auto-refresh and display the Gradio interface. Upload audio, wait for inference to finish, then download the generated video/NPZ results just like the local demo.
+
+---
+Add or adjust dependencies as new features require. Heavy rendering tasks can be slow on free CPU hardware; consider upgrading the Space or trimming the pipeline (e.g. precomputing alignments) if latency becomes an issue.

app.py

@@ -0,0 +1,34 @@
+import os
+from pathlib import Path
+
+from huggingface_hub import snapshot_download
+
+# ---------------------------------------------------------------------------
+# Optional: pull checkpoints and auxiliary assets at startup.  Set the
+# HF_GESTURELSM_WEIGHTS_REPO environment variable in the Space settings to the
+# dataset or model repo that hosts the pre-trained weights (e.g. "username/gesturelsm-assets").
+# Files will be placed under ckpt/ so the existing config paths keep working.
+# ---------------------------------------------------------------------------
+BASE_DIR = Path(__file__).parent.resolve()
+CKPT_DIR = BASE_DIR / "ckpt"
+CKPT_DIR.mkdir(parents=True, exist_ok=True)
+
+weights_repo = os.environ.get("HF_GESTURELSM_WEIGHTS_REPO", "").strip()
+if weights_repo:
+    snapshot_download(
+        repo_id=weights_repo,
+        repo_type="dataset",
+        local_dir=CKPT_DIR,
+        local_dir_use_symlinks=False,
+        allow_patterns=["*.pth", "*.bin", "*.npz", "*.npy"],
+    )
+
+# Ensure expected runtime directories exist so the demo can write outputs.
+for relative in ["outputs/audio2pose", "datasets/BEAT_SMPL"]:
+    (BASE_DIR / relative).mkdir(parents=True, exist_ok=True)
+
+# Reuse the existing Gradio interface defined in demo.py.
+from demo import demo as gesture_demo  # noqa: E402
+
+if __name__ == "__main__":
+    gesture_demo.queue(concurrency_count=1).launch(server_name="0.0.0.0", share=False)

packages.txt

@@ -0,0 +1,2 @@
+ffmpeg
+openfst-bin

prepare_assets.py

@@ -0,0 +1,22 @@
+"""Utility helpers for the Hugging Face Space deployment.
+
+Call these functions from ``app.py`` if you want extra setup beyond the basic
+snapshot download (for example copying files into specific folders).  This file
+is optional; include or edit it as needed for your workflow.
+"""
+
+from __future__ import annotations
+
+import shutil
+from pathlib import Path
+
+
+def copy_if_missing(src: Path, dst: Path) -> None:
+    """Copy ``src`` into ``dst`` if the destination path does not yet exist."""
+    if dst.exists():
+        return
+    dst.parent.mkdir(parents=True, exist_ok=True)
+    if src.is_dir():
+        shutil.copytree(src, dst)
+    else:
+        shutil.copy2(src, dst)

requirements.txt

@@ -0,0 +1,15 @@
+gradio>=4.42.0
+huggingface-hub>=0.24.0
+torch==2.4.0
+torchvision==0.19.0
+torchaudio==2.4.0
+librosa>=0.10,<0.11
+einops
+loguru
+omegaconf>=2.3
+tqdm
+pyvirtualdisplay
+soundfile
+montreal-forced-aligner==2.1.0
+numpy==1.26.4
+scipy>=1.11