Giggle Forest (project.VoiceSpace) by Team - under5
Development Goals
As we go through the COVID-19, most of the 'meeting places' are rapidly moving to various virtual conference spaces started from Zoom. As such, there are many side effects, A typical example is Zoom Fatigue, which causes a lot of fatigue in virtual conferences than usual conversations. We think the main causes of that are
- video
- Rigid video conferencing space Accordingly, we started to development this service with the purpose of creating a service that provides a video-free, realistic conversation, and a soft and enjoyable space.
Introduction Video (with Korean language)
Demo Website Link
- url : https://giggleforest.com
- you can try it right here
Development and deployment environment
| Part | Environment | Remark(Version) |
|---|---|---|
| FrontEnd | React | 17.0.2 |
| BackEnd | NestJS | 7.6.15 |
| WebServer | Nginx | 1.14.2 |
| Publishing Server Hardware | AWS | EC2 |
Main library
| Name | Remark |
|---|---|
| WebRTC | Seamless voice and data transmission between users |
| WebGL(PixiJS) | Outputs animation instead of video, using the client's graphics processing hardware |
| WebAudioAPI | Voice analysis and control for realizing realistic conversations without video |
| Jest | Unit and Integration Test of implemented components and classes |
System diagram
Overall system configuration
- Peer-to-peer connection using WebRTC is the most important part.
- Audio streams are exchanged through this P2P connection, and avatar-related data is also continuously transmitted to each other.
- Therefore, when a user connects for the first time, the P2P connection is established using the back signaling server.
- As the connection is completed, the transmitted audio stream is continuously played, and animation is drawn every frame using WebGL through the transmitted avatar information.
Multiple peer connection architecture
- So we aim for a project that anyone can service on their own server, that a number of P2P connections were established in Full Mesh in order to minimize the role of the back server.
- If we launch this service, the architecture may be restructured using SFU.
Page Infomation
- Home Page
| Component Path | /front/src/pages/homePage/Home.tsx |
| URL | / |
| Role | Route to SettingComponent (/setting) with random or specific(input) roomId |
| Remark | 자세히 알아보세요 is connected with this github page |
- Setting Page
| Component Path | /front/src/pages/settingPage/Setting.tsx |
| URL | /setting |
| Role | Select mic, speaker device and avatar information (nickname and avatarImage) and route to SpaceComponent with selected values |
| Remark | Only Chrome browser is fully supported |
- Space Page
| Component Path | /front/src/pages/spacePage/Space.tsx |
| URL | /space |
| Role | - Draw map and avatars using pixiUtils - Establish P2P connection and transfer avatar data using Peer, PeerManager - Analyze voice for lip sync using AudioAnalyer - Control all of settings (with Navigation Component |
| Remark | - If you want to customize world map, see World.ts and world1.json - If you want to add some special stuff (like Youtube sign), see Stuff.ts - If you want to Improve lip sync logic, see AudioAnalyer - If you want to develop your own signaling server, see RTCSignalingHelper.ts - If you want to add stun/turn server, see IceServerList.ts |
Main function
- Smooth animation using WebGL in a outstanding level (reducing eye fatigue)
- Real-time voice chat with very low latency using WebRTC (reducing fatigue from delay)
- The more a distance between avatars increases, the more voice volume also decreases (like realistic meeting)
- A function that analyzes the voice and changes the shape of the avatar's mouth to match the pronunciation (like realistic meeting)
- The size of the avatar's face changes according to the volume of the voice. (realistic voice visualization)
you can try it right here
Main features being added (Advancement)
- map customization
Start project
in back folder
npm install
npm run start:dev
in front folder
npm install
npm run start
- You can change the backend address in .env.development in the front folder.
Deploy the project
Front
- npm run build (in front folder) The value of (.env.production) applies.
- Serve the build folder output from the above script as root (our team used nginx)
Back
- npm run build (in back folder) (In this case, if you want to set it to https, please change the contents of main.ts (see comments))
- Execute main.js in the dist folder as the output of the above script as NodeJS.
Document
- Information on various components, classes, and functions can be found here.
Test
In front folder
- unit test
npm run test
- unit test with coverage report
npm run testCover
- The current master's coverage report can be viewed here
License
Team Information
| Name | Role | Major Part | Minor Part | Tech Stack | |
|---|---|---|---|---|---|
| kilee | gnsdlrl@daum.net | Leader | Design, project deployment and management | Front | AWS, Github Action, CI/CD, React, Typescript |
| honlee | kij753@naver.com | member | Front | Back | AWS, React, NestJS, Typescript |
| hyeonkim | hyongtiii@gmail.com | member | Back | Front | React, NestJS, Typescript |
| mijeong | minje70@naver.com | member | Front | Back | React, NestJS, Typescript |