HeteroAvatar:
Generation of Gaussian Head Avatars With Correct Geometry Using Hetero Rendering

Ahmet Cagatay Seker1,2,†,     Jun Seok Kang1,2,†,     Sang Chul Ahn1,2,✉    
† Denotes equal contribution.     ✉ Corresponding Author.
1 University of Science and Technology,
UST Logo
2 Korea Institute of Science and Technology,
KIST Logo

TL;DR: HeteroAvatar combines 3D Gaussian and mesh rendering to reconstruct photorealistic, animatable, and geometrically accurate Gaussian head avatars from monocular videos of dynamic heads.


Sample Image.

Given a monocular video of a dynamic head, the head avatars generated through (a) 3D Gaussian splatting only and (b) HeteroAvatar, which utilizes 3D Gaussian splatting with mesh rendering concurrently, produce notably distinct outcomes. From left to right: a rendered image using 3D Gaussian splatting, followed by frontal and side views of the 3D Gaussian position visualizations. The method of 3D Gaussian splatting alone results in a head avatar composed of numerous Gaussians that fail to conform precisely to the head's geometry. In contrast, HeteroAvatar generates a head geometry with fewer 3D Gaussians that more effectively align with the actual geometric structure.

Abstract

Head avatar reconstruction is a crucial research topic with wide-ranging applications. For this purpose, utilizing the promising technique of 3D Gaussian splatting significantly improves rendering quality. However, this method encounters challenges, such as imprecise geometry and noisy surfaces. To address these issues, we propose HeteroAvatar, a novel method for generating a Gaussian head avatar with correct geometry from a monocular video of a dynamic head. To achieve this, we introduce the hetero rendering method, which concurrently employs both 3D Gaussian rendering and mesh rendering techniques during training. This approach enables HeteroAvatar to accurately capture the head geometry with smooth surfaces while maintaining photorealistic rendering quality. Additionally, our approach is capable of generating a mesh-based head avatar, which ensures compatibility with a wide range of traditional graphics applications. Experimental results demonstrate that HeteroAvatar outperforms state-of-the-art methods in terms of reconstructed geometry quality and photorealistic rendering with fewer 3D Gaussians.

Method

Given a monocular video, a head tracker algorithm is employed to extract face-related parameters (\(\boldsymbol{\beta}\), \(\boldsymbol{\psi}\), \(\boldsymbol{\theta}\)) along with a FLAME mesh model. We then utilize a geometry-guided 3D Gaussian initialization, where a 3D Gaussian is associated with each vertex of the mesh. This process yields both an initial Gaussian model \(\boldsymbol{\mathcal{G}_0}\) and a corresponding mesh model \(\boldsymbol{\mathcal{M}_0}\). Learnable features \(\boldsymbol{L_i}\) are subsequently assigned to each vertex. The training pipeline utilizes canonical and deformation modules to refine the shape and learn Gaussian parameters, followed by the use of both mesh rendering and Gaussian rendering for image generation. Mesh rendering is essential for accurately capturing the geometry, as it ensures the alignment of 3D Gaussian centers with the actual head surface. Meanwhile, 3D Gaussian rendering contributes significantly to improving the visual quality. In the lower part of the figure, the red and blue blobs represent 3D Gaussians attached to base and subdivided vertices, respectively. The green blobs represent free Gaussians for better image quality.

Generated Gaussian Head Models

The Gaussian positions of head avatars reconstructed across multiple subjects demonstrate that HeteroAvatar consistently aligns the Gaussians with the underlying head geometry, producing geometrically accurate and smooth surfaces across diverse identities.

Qualitative Comparison with State-of-the-Art

Qualitative comparisons across multiple subjects demonstrate that HeteroAvatar achieves visual quality comparable to or better than existing state-of-the-art methods, while preserving accurate head geometry and fine appearance details.

Additional Qualitative Results

Additional video comparisons across diverse subjects demonstrate that HeteroAvatar faithfully reproduces ground-truth appearance, facial expressions, head motion, and fine facial details with consistently high photorealistic quality.

Novel-View Rendering

Novel-view rendering results across multiple subjects demonstrate that HeteroAvatar synthesizes photorealistic and consistent head appearances from previously unseen viewpoints while preserving facial details and accurate head geometry.

Cross-Identity Reenactment

Cross-identity reenactment experiments across multiple subjects demonstrate that HeteroAvatar enables reconstructed head avatars to be animated using the head poses and facial expressions of different individuals while preserving the target identity and maintaining realistic visual quality.

Generated Mesh-based Head Models

HeteroAvatar additionally generates mesh-based head avatars through the mesh-rendering process, accurately capturing the underlying head geometry and producing smooth surface representations.

Interpolate start reference image.

Compatibility with Graphics Applications

The generated mesh-based head avatars are compatible with existing commercial graphics applications, enabling their direct use in various applications using only the mesh model and its corresponding texture map.

BibTeX

If you find this work useful in your research, please consider citing:

@article{seker2025heteroavatar,
  title     = {HeteroAvatar: Generation of Gaussian Head Avatars With Correct Geometry Using Hetero Rendering},
  author    = {Seker, Ahmet Cagatay and Kang, Jun Seok and Ahn, Sang Chul},
  journal   = {IEEE Access},
  volume    = {13},
  pages     = {33768--33782},
  year      = {2025},
  publisher = {IEEE}
}