MediaPipe is an open-source framework from Google for building real-time, on-device perception pipelines (things that understand images, video, audio or text).
Think of MediaPipe as a toolbox of ready-made ML solutions (face mesh, hands, pose, object detection, etc.) and a low-level framework you can use to build custom ML pipelines that run fast on phones, browsers, and desktops.
Mediapipe Solutions
Mediapipe Framework
MediaPipe Solutions Vs MediaPipe Framework
The Model works by detecting landmarks . Landmarks are nothing but key points on the human body (nose, eyes, shoulders, elbows…).
Landmarks are different for each model. Meaning the landmark for face would be different that landmark for pose.
For reference :
| Model | # of Landmarks | What They Represent |
|---|---|---|
| Pose | 33 | Body joints, shoulders, elbows, knees, feet |
| Hands | 21 (per hand) | Finger joints, wrist |
| Face Mesh | 468 | Dense 3D face mesh |
| Face Detection | Only 6 keypoints | Eyes, nose, mouth corners |
| Holistic | 33 + 21×2 + 468 | Whole body (combines pose + hands + face) |
The pose landmarker model tracks 33 body landmark locations, representing the approximate location of the following body parts:
0 - nose
1 - left eye (inner)
2 - left eye
3 - left eye (outer)
4 - right eye (inner)
5 - right eye
6 - right eye (outer)
7 - left ear
8 - right ear
9 - mouth (left)
10 - mouth (right)
11 - left shoulder
12 - right shoulder
13 - left elbow
14 - right elbow
15 - left wrist
16 - right wrist
17 - left pinky
18 - right pinky
19 - left index
20 - right index
21 - left thumb
22 - right thumb
23 - left hip
24 - right hip
25 - left knee
26 - right knee
27 - left ankle
28 - right ankle
29 - left heel
30 - right heel
31 - left foot index
32 - right foot index
The Pose model predicts 33 specific points on the body.
Each point has: