April 12, 2023

Full Body Tracking Using Webcams

A young boy wearing a white VR headset

How do we enhance a tech-based medical treatment experience?

Neurotechnology is our joint venture company that is trying something new and innovative in the medical technology field. We are using Virtual Reality technology to create a therapeutic environment to help treat chronic pain. This is done by Tranceducer & CognitiVR, our patented treatment software and hardware combination.

At Innovate Pain Clinic, we help people with chronic pain using virtual reality psychotherapy. It's drug free and surgery free, medical technology practice. We use immersive experience to create an analgesic effect using immersive environment in Virtual Reality. To have the said analgesic effect, we need to embody the user using full body tracking, so they would see their mirror "virtual" body in the VR experience. To do this we are using depth sensing camera to fully capture the user's body movements.

Our challenge is some of the hardware and technology we used when starting the development is starting to be obsolete and out of production. We need to replace the hardware and update the software with more ubiquitous hardware.

We created a desktop application to do a Full Body Tracking using simple camera hardware like a webcam as an input. This desktop application was developed using Python, with the help of Mediapipe Library to track the user’s full body just by capturing it with a webcam. The software calculate the body movement and positioning from landmarks that are connected to each other.

The Full Body Tracking application uses the Pose Tracking module as well as the Hand Tracking module which is available in the Mediapipe Library. This is an additional improvements to the current body tracking system we have.

With this solution, we can now utilise camera input that are readily available anywhere, reducing the pressure of availability and hardware obsolescence. By having simpler hardware yet more advanced software we can now look into creating a remote version for Innovate Pain Clinic and int he future, the home version for mass production.

Do you need a camera based solution to detect an individual or crowd characteristics?

Let's have a chat.