{"repo":"emilianavt/OpenSeeFace","free":true,"listed":false,"github":"https://github.com/emilianavt/OpenSeeFace","clone":"git clone https://github.com/emilianavt/OpenSeeFace.git","description":"Robust realtime face and facial landmark tracking on CPU with Unity integration","language":"Python","stars":2030,"topics":["face-tracking","face-landmarks","depth-estimation","unity","unity3d","python","csharp","udp","onnx","onnxruntime"],"license":"BSD-2-Clause","category":"game-templates","readme_excerpt":"Overview Note : This is a tracking library, not a stand-alone avatar puppeteering program. I'm also working on VSeeFace, which allows animating VRM and VSFAvatar 3D models by using OpenSeeFace tracking. VTube Studio uses OpenSeeFace for webcam based tracking to animate Live2D models. A renderer for the Godot engine can be found here. This project implements a facial landmark detection model based on MobileNetV3. As Pytorch 1.3 CPU inference speed on Windows is very low, the model was converted to ONNX format. Using onnxruntime it can run at 30 - 60 fps tracking a single face. There are four models, with different speed to tracking quality trade-offs. If anyone is curious, the name is a silly pun on the open seas and seeing faces. There's no deeper meaning. An up to date sample video can be found here, showing the default tracking model's performance under different noise and light levels. Tracking quality Since the landmarks used by OpenSeeFace are a bit different from those used by other approaches (they are close to iBUG 68, with two less points in the mouth corners and quasi-3D face contours instead of face contours that follow the visible outline) it is hard to numerically compare its accuracy to that of other approaches found commonly in scientific literature. The tracking performance is also more optimized for making landmarks that are useful for animating an avatar than for exactly fitting the face image. For example, as long as the eye landmarks show whether the eyes ","default_branch":null,"files":null,"tree":[],"storefront":"/r/emilianavt","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/emilianavt/OpenSeeFace/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}