{"repo":"NeuralHarbour/LLM-Based-3D-Avatar-Assistant","free":true,"listed":false,"github":"https://github.com/NeuralHarbour/LLM-Based-3D-Avatar-Assistant","clone":"git clone https://github.com/NeuralHarbour/LLM-Based-3D-Avatar-Assistant.git","description":"A project which is aimed to create a virtual avatar based assistant using LLM's with dynamic voice and dynamic animations with many other cool features similar to that of GATEBOX","language":"C#","stars":41,"topics":["3d","csharp","gemini-api","gpt4o","holographic-application","llm","python","rust-lang","unity","virtualassistant"],"license":"AGPL-3.0","category":"ai-agents","readme_excerpt":"LLM Based Holographic Assistant Shaping Perspectives Shifting Realities Follow Us About The Project A virtual waifu / assistant that you can speak to through your mic and it'll speak back to you! Has many features such as: You can speak to her with a mic It can speak back to you It can recognize gestures and perform appropriate actions based on it Has dynamic voice controls Has short-term memory and long-term memory Can open apps Smarter than you Multilingual Can learn psychology Can recognize different people Can sing,dance do lot of other stuff. Can control your smart home like Alexa Dynamic animations More features I'm planning to add soon in the Roadmap. Also, here's a summary of how it works for those of you who want to know: First the speech which is obtained from the mic is first passed into the OpenAI Whisper model for detecting the language of the speech and then its passed on to the speech recognizer, the response is then sent to the client via a websocket, the transcribed text is then sent to gemini LLM, and the response from Jenna is printed to the unity console and appended to conversation log.json, and finally, the response is spoken by our finedtuned Parler TTS with the expressions for the face of the character and voice and the animation will be dynamically generated for the character by our own custom animation model which we are currently working on. For the time being we are using preset animations. Created Based on our publication https://ieeexplore.ieee.o","default_branch":null,"files":null,"tree":[],"storefront":"/r/NeuralHarbour","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/NeuralHarbour/LLM-Based-3D-Avatar-Assistant/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}