mtcnn and tinyYolov2

[treedbox]

Hi Vladimir,
Pretty neat your project to update the face-api with to TensorFlow/JS 2+.

About this information:

  Removed mtcnn and tinyYolov2 models as they were non-functional in latest public version of Face-API
  If there is a demand, I can re-implement them back.

Actually mtcnn and tinyYolov2 can be used. This is the way to use all models/weights:

import {
    nets,
    createCanvasFromMedia,
    detectAllFaces,
    tinyFaceDetector,
    tinyYolov2,
    mtcnn,
    draw,
} from "face-api.js";
import type {
    FaceDetection,
    WithFaceLandmarks,
    FaceLandmarks5,
} from "face-api.js";

const canvas: HTMLCanvasElement = createCanvasFromMedia(img);

// mtcnn_model-shard1
// mtcnn_model-weights_manifest.json
nets.mtcnn.loadFromUri("./weights")
const detections: WithFaceLandmarks<
        {
            detection: FaceDetection;
        },
        FaceLandmarks5
    >[] = await mtcnn(img);
draw.drawDetections(canvas, detections);
draw.drawFaceLandmarks(canvas, detections);

// tiny_yolov2_separable_conv_model.weights
// tiny_yolov2_separable_conv_model-shard1
// tiny_yolov2_separable_conv_model-weights_manifest.json
nets.tinyYolov2.loadFromUri("./weights")
const detections:FaceDetection[] = await tinyYolov2(img);
draw.drawDetections(canvas, detections);

// tiny_face_detector_model-shard1
// tiny_face_detector_model-weights_manifest.json
nets.tinyFaceDetector.loadFromUri("./weights")
const detections:FaceDetection[] = await tinyFaceDetector(img);
draw.drawDetections(canvas, detections);

// ssd_mobilenetv1_model-shard1
// ssd_mobilenetv1_model-shard2
// ssd_mobilenetv1_model-weights_manifest.json
nets.ssdMobilenetv1.loadFromUri("./weights")
const detections = await detectAllFaces(img)
draw.drawDetections(canvas, detections);

// face_landmark_68_model-shard1
// face_landmark_68_model-weights_manifest.json
nets.faceLandmark68Net.loadFromUri("./weights")
.withFaceLandmarks()
draw.drawFaceLandmarks(canvas, detections);

// face_landmark_68_tiny_model-shard1
// face_landmark_68_tiny_model-weights_manifest.json
nets.faceLandmark68TinyNet.loadFromUri("./weights")
.withFaceLandmarks(true)
draw.drawFaceLandmarks(canvas, detections);

// face_expression_model-shard1
// face_expression_model-weights_manifest.json
nets.faceExpressionNet.loadFromUri("./weights")
.withFaceExpressions()
draw.drawFaceExpressions(canvas, detections);

// age_gender_model-shard1
// age_gender_model-weights_manifest.json
nets.ageGenderNet.loadFromUri("./weights")
.withAgeAndGender()
// draw.DrawTextField...

// face_recognition_model-shard1
// face_recognition_model-shard2
// face_recognition_model-weights_manifest.json
nets.faceRecognitionNet.loadFromUri("./weights")
.withFaceDescriptors()
draw.drawFaceExpressions(canvas, detections);

[vladmandic]

thanks for that.

and yes, I did try them quickly using older version of face-api, but since they are not on par with two remaining models, I didn't include them.
do you see the need to do so?

[treedbox]

In general, the difference is loading time and the accurate.

mtcnn is lighter than ssdMobilenetv1 but is less accurate.
tinyYolov2 seems to be faster than ssdMobilenetv1.

Some projects can benefit from this.

[vladmandic]

true, but my thoughts were:

• MTCNN is really old and it's both bigger and less accurate
• TinyYoloV2 is good, but tinyFaceDetector is very similar and smaller at the same time

in any case, models used by face-api are pretty old - i enjoyed working on this project, but for my newer work I switched to:

• BlazeFace (back variation) model for facial mesh,
• augmented with Iris model for eyes,
• SSR-Net for age/gender,
• simple Keras CNN for emotion detection,
• MobileFaceNet for face embedding

take a look at https://github.com/vladmandic/human

[vladmandic]

btw, i'm converting this to a discussion as it's not really an issue that needs solving :)