Skintokens auto rigging skill for AI agents

skintokens-auto-rigging is a skill from the Summer skill library: a guide your AI agent reads when the task matches, so it builds the good version on the first try. Offline auto-rigging with skin-tokens.cpp (GGML SkinTokens/TokenRig port) — skeleton and skin weights from a static GLB mesh on CPU/Vulkan, into Summer Engine.


Status	Preview: not yet exercised in-engine by the Summer team
Domains	Asset pipeline, Pipeline, 3D models, Animation
Version	`0.1.1`
Engine version	`>=4.6`
Recommended set	No, installed by `--all` or by name

When your agent uses it

Auto-rig this static GLB mesh so I can animate it
Generate skin weights for a mesh that already has a skeleton
Batch-rig many creature or character meshes without hand-painting weights
I have an unrigged AI-generated character and need a first-pass rig

Install

summer setup installs every skill for your agent. To add only this one:

npx -y summer-engine@latest skills install skintokens-auto-rigging --agent claude-code

Swap claude-code for your agent, such as codex or cursor. See Skills for every option.

The skill

About the tool

skin-tokens.cpp is published by the localai-org GitHub organization. It is an Apache-2.0 C++23/GGML port of VAST-AI’s MIT-licensed SkinTokens/TokenRig research model (arXiv:2602.04805). It ships a flat C11 API, an ASan/UBSan libFuzzer build, CPU/Vulkan numerical-parity tests against the upstream reference, and a Go/WebGL demo. The weights are a non-retrained GGUF conversion (~1.25 GB F16) on Hugging Face. Limitations to plan around:

The fully automatic rig command (unconstrained skeleton generation) is explicitly experimental — the policy can produce unsuitable bone topologies.
The author’s own demo shows open issues combining the output with kimodo.cpp skeletal animations.
Generation recomputes the Qwen3 graph without a KV cache and may take several minutes per mesh; dense decoder queries are capped at 16K vertices per graph (chunked beyond that).
Results are strongest on shapes resembling the training distribution (humanoids; trained on ArticulationXL 2.0, VRoid Hub, ModelsResource).

Recommendation: worth adopting as an offline asset-prep tool in the art pipeline, primarily the skin command (weights for an existing skeleton). Treat rig-generated skeletons as drafts that need artist review before animation use.

Outcome

Static .glb meshes (e.g. AI-generated or scanned characters/creatures) become rigged, skinned .glb assets that Summer Engine imports natively as Skeleton3D + skinned MeshInstance3D, ready for AnimationPlayer/retargeted animation — without manual weight painting.

When to Use

You receive an unrigged character/creature mesh and need a fast first-pass rig or skin weights.
You have an existing armature (e.g. a mocap/Kimodo-style skeleton GLB) and only need learned skin weights for a mesh (skin command — the reliable path).
Batch-rigging many props/characters where hand-painting weights is not cost-effective.

When NOT to Use

Hero characters needing production-quality, art-directed deformation — plan for manual weight cleanup after auto-rigging.
Non-triangle meshes (tool accepts arbitrary triangle meshes only; triangulate first).
Shapes far from the training distribution (expect weaker results; run --geometric diagnostic to compare).
Runtime/in-engine rigging — this is an offline CLI/library step, not an editor plugin.

Core Principle

TokenRig models the whole rig — skeleton hierarchy followed by discrete skin-weight tokens (SkinTokens, an FSQ-CVAE vocabulary) — as one autoregressive sequence generated from a Michelangelo point-cloud encoding of the mesh (Qwen3-0.6B backbone, GRPO-refined). skin-tokens.cpp re-runs this stack through GGML on CPU or Vulkan and exports a standard skinned glTF/GLB with a one-frame rest pose, so it opens as a conventional rigged asset in any engine.

Pipeline shape (Summer Engine)

static character.glb
      │
      ▼
[skin-tokens-cli]  rig | skin   (offline, CPU or Vulkan, minutes per mesh)
      │
      ▼
character-rigged.glb   (skeleton + skin weights + rest pose, skinned glTF)
      │
      ▼
Summer Engine import  →  character-rigged.glb (scene)
      └── Skeleton3D
            └── MeshInstance3D (skinned, Skin resource)
      └── AnimationPlayer / retargeted animations

Keep raw static meshes and rigged outputs as separate files under the asset folder convention, e.g. assets/chars/<name>/source.glb and assets/chars/<name>/rigged.glb, so re-rigging never destroys source art.

Setup (once)

Build the CLI (Linux; C++23 toolchain, CMake ≥ 3.25, Ninja, nlohmann-json, Vulkan unless disabled):

   git clone https://github.com/localai-org/skin-tokens.cpp
   cd skin-tokens.cpp
   git submodule update --init --recursive
   cmake -S . -B build/release -G Ninja -DCMAKE_BUILD_TYPE=Release   # add -DSKINTOKENS_ENABLE_VULKAN=OFF if no Vulkan
   cmake --build build/release -j
   cmake --install build/release --prefix ./dist
   ctest --test-dir build/release --output-on-failure
   ./build/release/bin/skintokens-cli inspect models/SkinTokens-GGUF/F16   # sanity check after weights download

Download the F16 GGUF bundle (keep the 3 files together; runtime checks their embedded identities):

   hf download LocalAI-io/SkinTokens-GGUF --include "F16/*" --local-dir models/SkinTokens-GGUF

(~1.25 GB: mesh-encoder 57.8 MB + tokenrig 948.6 MB + skin-vae 244 MB. Use F32 only for numerical-parity work.)

Implementation Steps

1. Auto-rig a static mesh (draft skeleton + weights)

./build/release/bin/skintokens-cli rig models/SkinTokens-GGUF/F16 \
  character.glb character-rigged.glb --device vulkan --postprocess

--postprocess applies upstream’s surface-locality heuristic (smoother, local influences); omit it only to preserve raw learned weights.
Output stores a one-frame rest pose; inspect the skeleton topology before animating (experimental mode — see Failure Modes).
Use --geometric as a non-learned baseline diagnostic when results look wrong.

2. Re-weight against an existing skeleton (preferred when an armature exists)

# One rigged GLB supplies both geometry and skeleton (old weights ignored):
./build/release/bin/skintokens-cli skin models/SkinTokens-GGUF/F16 \
  character-rigged.glb character-rigged.glb character-reweighted.glb \
  --device vulkan --fit none

# Or separate mesh + animated skeleton GLBs (e.g. Kimodo animation output):
./build/release/bin/skintokens-cli skin models/SkinTokens-GGUF/F16 \
  character.glb motion.glb animated.glb --device vulkan --fit global

Fit modes:

`--fit`	When	Behavior
`global` (default)	Mesh and skeleton in different coordinates	One uniform scale + translation matching vertical extent and centre; preserves relative joint positions and bone-length ratios
`none`	Both files already share coordinates	No alignment transform
`articulated`	Recognized humanoid arm chains; experimental	Picks whichever of rest pose / first animation frame actually runs inside the arms (bind pose == frame zero → playback starts without warping); falls back to analytic two-bone IK toward conservative mesh targets, preserving arm segment lengths

3. Import into Summer Engine

Drop the rigged .glb into the project; the glTF importer creates the scene automatically:

Skeleton3D with bones and a skinned MeshInstance3D (Skin resource), rest pose included.
Check in the Import dock: mesh, skeleton, and (if supplied) animations present.
If the source GLB contains animations and the hierarchy matches, the importer puts them into an AnimationPlayer; otherwise use retargeting (SkeletonProfileHumanoid, see the retarget skill) to map animations onto the generated skeleton.
glTF round-trip note from upstream: on Blender export of results, a glTF_not_exported node may need removing (only relevant if the pipeline goes through Blender).

4. Optional editor tooling (typed GDScript)

An EditorScript wrapper to rig assets from inside the editor:

@tool
extends EditorScript

const CLI := "/opt/skin-tokens.cpp/build/release/bin/skintokens-cli"
const MODEL_DIR := "/opt/skin-tokens.cpp/models/SkinTokens-GGUF/F16"

func _run() -> void:
	var src := "res://assets/chars/goblin/source.glb"
	var dst := "res://assets/chars/goblin/rigged.glb"
	var args: PackedStringArray = [
		"rig", MODEL_DIR,
		ProjectSettings.globalize_path(src),
		ProjectSettings.globalize_path(dst),
		"--device", "vulkan", "--postprocess",
	]
	var output: Array = []
	var err := OS.execute(CLI, args, output, true, false)
	if err != 0:
		push_error("skin-tokens-cli failed (%d): %s" % [err, "\n".join(output)])
		return
	print("Rigged: ", dst)
	# Reimport so the new GLB appears in the FileSystem dock.
	EditorInterface.get_resource_filesystem().scan()

For embedding in a tool/asset server, prefer the flat C11 API (skintokens.h): st_model_load → st_rig_file / st_skin_files / st_inspect_glb_file, with caller-owned error buffers; C++ exceptions never cross the ABI.

Tunables

Parameter	Meaning	Guidance
`--device`	`cpu` or `vulkan`	Vulkan if available; CPU works but slower. F16 for normal inference
`--postprocess`	Surface-locality weight smoothing	On for game characters; off only to keep raw weights
`--fit`	Skeleton↔mesh alignment (`global`, `none`, `articulated`)	`global` default for separate files; `none` when coordinates match; `articulated` experimental humanoid-arm fix
`--geometric`	Non-learned diagnostic rigging	Compare against learned output when results look wrong
F16 vs F32 bundle	Precision	F16 (~1.25 GB) normal; F32 parity work only
Vertex budget	Dense decoder queries capped at 16K verts/graph, chunked beyond	Very dense meshes are handled in chunks; expect longer runs

Failure Modes & Gotchas

Unsuitable skeleton topology from rig (experimental policy) — always have an artist inspect bones before animating; re-run or fall back to a template skeleton + skin.
Animation combination issues: the author reports unresolved problems combining skin-tokens.cpp rigs with kimodo.cpp skeletal animations (visible at the end of the demo video). Verify any animation transfer manually.
Warped first frame when the supplied skeleton’s pose doesn’t match the mesh: use --fit global (or articulated for humanoid arms) so bind pose aligns with the mesh; --fit none only when coordinates already match.
Out-of-distribution shapes degrade quality silently — no error is raised. Spot-check deformation visually.
Runtime: several minutes per mesh (no KV cache); do not put this in a per-frame or hot-reload path — batch offline.
Bundle integrity: the three GGUF files of a bundle must stay together; load fails identity checks otherwise.
Vulkan precision: slightly looser numerics than CPU (7.1e-4 hidden state / 1.4e-3 logits relative L2 vs 8.1e-6 CPU F32); visually equivalent but use F32+CPU for parity debugging.
Upstream GPU path (Python VAST-AI repo) needs an NVIDIA GPU ≥ 14 GB — use the C++ port instead for CPU/Vulkan machines.

Verification

Not yet verified in Summer Engine. To validate:

skintokens-cli inspect models/SkinTokens-GGUF/F16 succeeds after build + weights download.
Rig a known test mesh (rig … --postprocess) and open the output in the editor: Skeleton3D with sensible bone hierarchy + skinned MeshInstance3D, no import errors.
Pose one bone in the editor; confirm the surface deforms smoothly without bleeding across disconnected parts.
skin path: feed a mesh + animated skeleton GLB with --fit global; play the animation in the editor and confirm frame zero does not warp the mesh.
Re-run ctest / parity binaries if the CLI is rebuilt or GGML is updated.

Confidence

extracted — All tool behavior, commands, flags, sizes, numerics, and caveats come from the retrieved GitHub README, Hugging Face model card, upstream VAST-AI repo, and the author’s X announcement (including the self-reported kimodo.cpp animation issue). The import mapping and GDScript wrapper are adaptations, not from the sources.

character-model: Generate a rigged humanoid (player, NPC, enemy, boss) from a T-pose reference, previewed before rigging.
retarget: Apply animation clips from one rigged character to another: one clip library, many models.

Source: library/skills/skintokens-auto-rigging/SKILL.md, rendered from summer-engine@4.3.0 on npm.

​When your agent uses it

​Install

​The skill

​About the tool

​Outcome

​When to Use

​When NOT to Use

​Core Principle

​Pipeline shape (Summer Engine)

​Setup (once)

​Implementation Steps

​1. Auto-rig a static mesh (draft skeleton + weights)

​2. Re-weight against an existing skeleton (preferred when an armature exists)

​3. Import into Summer Engine

​4. Optional editor tooling (typed GDScript)

​Tunables

​Failure Modes & Gotchas

​Verification

​Confidence

​Related skills