CVTHead: One-shot Controllable Head Avatar with Vertex-feature Transformer

Reconstructing personalized animatable head avatars has significant implications in the fields of AR/VR. Exist- ing methods for achieving explicit face control of 3D Mor- phable Models (3DMM) typically rely on multi-view im- ages or videos of a single subject, making the reconstruc- tion process complex. Additionally, the traditional ren- dering pipeline is time-consuming, limiting real-time an- imation possibilities. In this paper, we introduce CVT- Head, a novel approach that generates controllable neu- ral head avatars from a single reference image using point- based neural rendering. CVTHead considers the sparse vertices of mesh as the point set and employs the pro- posed Vertex-feature Transformer to learn local feature de- scriptors for each vertex. This enables the modeling of long-range dependencies among all the vertices. Exper- imental results on the VoxCeleb dataset demonstrate that CVTHead achieves comparable performance to state-of- the-art graphics-based methods. Moreover, it enables ef- ficient rendering of novel human heads with various expres- sions, head poses, and camera views. These attributes can be explicitly controlled using the coefficients of 3DMMs, facilitating versatile and realistic animation in real-time scenarios. Codes and pre-trained model can be found at https://github.com/HowieMa/CVTHead.