1.2.4.d3

Select a face from video

Face capture selects usable images from a video sequence. It checks face position, size and stability over time, then keeps the best candidates. Use it for enrollment, profile photos or other flows that need a small set of face images.

These examples use the capture and snapshot APIs in 1.2.4. Use wrapper classes, headers and native libraries from the same version. For Android, build the Java classes and JNI library together from 1.2.4.

How capture progresses

Capture states, track-loss recovery and completion conditions
Capture waits for a stable track, then collects candidates. READY means the capture conditions were met. FINISHED closes the round and can also be reached by timeout.
Text version of the main flow
Frame + timestamp → Track faces → Evaluate enabled filters
                                      ↓
                 IDLE → STABILIZING → COLLECTING → READY
                            ↑                         ↓
                       Retry / reset              finish()
                                                  FINISHED

Each update evaluates one input frame synchronously. The application opens the camera, schedules processing and displays prompts. Loss of the tracked face can enter TRACK_LOST; the configured grace period and a new stable track determine recovery. Call reset() to start a fresh capture round.

Start with the default policy

C API

The SDK and a HF_DETECT_MODE_LIGHT_TRACK session are already open. Create capture once, call update_capture for each valid stream, then release capture before its parent session. A maximum of eight results fits the current outputCount limit.

#include <stdio.h>
#include <inspireface.h>

static HResult create_capture(HFSession session, HFFaceCaptureSession *capture) {
    HFFaceCaptureConfig config = {0};
    HResult status = HFGetDefaultFaceCaptureConfig(&config);
    if (status != HSUCCEED) return status;
    return HFCreateFaceCaptureSession(session, &config, capture);
}

static HResult update_capture(HFFaceCaptureSession capture, HFImageStream stream,
                              HFUInt64 frame_id, HFUInt64 timestamp_ms,
                              HFFaceCaptureProgress *progress) {
    HResult status = HFUpdateFaceCaptureSession(
        capture, stream, frame_id, timestamp_ms, progress);
    if (status != HSUCCEED) return status;
    printf("state=%d reject=%llu\n", progress->state,
           (unsigned long long)progress->rejectReasons);
    if (progress->state == HF_CAPTURE_STATE_READY) {
        status = HFFinishFaceCaptureSession(capture, progress);
        if (status != HSUCCEED) return status;
    }
    HFFaceCaptureResult results[8];
    HFUInt32 count = 0;
    status = HFGetFaceCaptureResults(capture, results, 8, &count);
    if (status != HSUCCEED) return status;
    for (HFUInt32 i = 0; i < count; ++i) {
        printf("candidate=%llu score=%.3f\n",
               (unsigned long long)results[i].frameId, results[i].score);
    }
    // Retain matching candidate images in the application (see below).
    return HSUCCEED;
}
// End of the round: HFReleaseFaceCaptureSession(capture);

For Python, frame is a BGR array; native callers pass the corresponding image stream or FrameProcess. Capture evaluates a sequence, so keep submitting consecutive frames with strictly increasing frame IDs and timestamps in milliseconds. Use a monotonic clock for live capture and the video timeline for offline input.

Once capture reaches FINISHED, stop feeding frames for that round. Read the final results, or reset the capture object to begin another round.

A finished round can retain candidates even if it never reached READY. The example below saves a frame only after READY; a round that finishes earlier is reported as unsuccessful.

Set max_detect_num above 1 so the face-count filter can detect and reject frames with multiple faces.

Default filterWhat it checks
Face countOne face is available for capture.
Face sizeFace width is within the configured fraction of the image width.
Position and boundaryThe face is near the center and stays inside the frame.
StabilityPosition and size have settled over a time interval.
Track countThe tracker has seen the face for enough updates.

The current defaults request one output, at least five tracker observations, a 300 ms stable period and an 800 ms collection period. Read the default configuration through the API, then change the fields your application needs.

Keep the selected images

Capture results contain a frame ID, timestamp, score, face token and metrics. Store the selected image pixels in an application cache. Use the same cache strategy in each language: inspect current result IDs after every update, copy the current image only if its ID is selected, and remove entries no longer selected. On Apple, copy the selected source pixels into application-owned storage before the camera buffer is reused; keeping a face token does not keep the image. In C++, use Image::Clone(); in Android, copy the bitmap or camera bytes before the input buffer is reused. In ArkTS, copy selected camera bytes with new Uint8Array(bytes) and keep them by frameId. Here is the Python version:

An output_count of one keeps image pixels for the selected frame ID rather than the newest frame
Scores here only illustrate the cache policy. Frame 13 leaves frame 12 selected; frame 14 replaces it, allowing the old image to be released.
# Inside the frame loop; candidates is an initially empty dictionary.
progress = capture.update(frame, frame_id, timestamp_ms)
results = capture.results()
kept_ids = {result.frame_id for result in results}
if frame_id in kept_ids:
    candidates[frame_id] = frame.copy()
candidates = {key: value for key, value in candidates.items() if key in kept_ids}

if progress.state == isf.FaceCaptureState.READY:
    capture.finish()
    selected = capture.results()[0]
    selected_frame = candidates[selected.frame_id]
elif progress.state == isf.FaceCaptureState.FINISHED:
    raise RuntimeError("Capture finished before becoming ready; start a new round")

This keeps the current candidates, bounded by output_count (up to eight). An older selected frame stays in the cache until a better candidate replaces it or the round ends.

Save the complete code below as capture.py, then pass the video path:

capture.py — complete code
"""Select a stable face frame from a video using InspireFace 1.2.4."""
import argparse
import math
from pathlib import Path

import cv2
import inspireface as isf


def main():
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("video")
    parser.add_argument("--model", required=True)
    parser.add_argument("--output", default="captured.jpg")
    args = parser.parse_args()
    video = cv2.VideoCapture(args.video)
    if not video.isOpened():
        parser.error("Cannot open the input video")
    fps = video.get(cv2.CAP_PROP_FPS)
    if not math.isfinite(fps) or not 0 < fps <= 1000:
        video.release()
        parser.error("This example needs a video with a valid frame rate")

    launched = False
    try:
        isf.launch(resource_path=args.model)
        launched = True
        with isf.InspireFaceSession(
            isf.HF_ENABLE_NONE, isf.HF_DETECT_MODE_LIGHT_TRACK,
            max_detect_num=5, detect_pixel_level=320, auto_launch=False,
        ) as session:
            config = isf.FaceCaptureConfig.defaults()
            with session.create_face_capture(config) as capture:
                candidates = {}
                frame_id = 0
                while True:
                    ok, frame = video.read()
                    if not ok:
                        break
                    # Use video time, not the speed of this offline processing loop.
                    timestamp_ms = round(frame_id * 1000 / fps)
                    progress = capture.update(frame, frame_id, timestamp_ms)
                    results = capture.results()
                    kept_ids = {result.frame_id for result in results}
                    if frame_id in kept_ids:
                        candidates[frame_id] = frame.copy()
                    candidates = {key: value for key, value in candidates.items()
                                  if key in kept_ids}
                    print(frame_id, progress.state.name, int(progress.reject_reasons))
                    frame_id += 1
                    if progress.state == isf.FaceCaptureState.READY:
                        capture.finish()
                        selected = capture.results()[0]
                        output = Path(args.output)
                        output.parent.mkdir(parents=True, exist_ok=True)
                        if not cv2.imwrite(str(output), candidates[selected.frame_id]):
                            raise RuntimeError(f"Cannot write {output}")
                        print(f"Saved frame {selected.frame_id} to {output}")
                        return
                    if progress.state == isf.FaceCaptureState.FINISHED:
                        raise RuntimeError("Capture finished before becoming ready; start a new round")
                raise RuntimeError("Video ended before capture was ready")
    finally:
        video.release()
        if launched:
            isf.terminate()


if __name__ == "__main__":
    main()

python capture.py enrollment.mp4 --model /path/to/Pikachu --output captured.jpg

The script saves the full selected frame after READY. It stops with an error if capture finishes before becoming ready, including a collection timeout, or if the video ends first. The sample assumes a constant-frame-rate video; use actual presentation timestamps for variable-frame-rate input.

Add quality and pose checks

Enable the corresponding session options as well as the capture filters:

C API

Create the parent session with HF_ENABLE_QUALITY | HF_ENABLE_FACE_POSE. Initialize the full versioned capture configuration first, then adjust only the required fields.

HFFaceCaptureConfig config = {0};
HResult status = HFGetDefaultFaceCaptureConfig(&config);
if (status != HSUCCEED) return status;
config.filterMask |= HF_CAPTURE_FILTER_QUALITY | HF_CAPTURE_FILTER_POSE;
config.minQualityScore = 0.60f;
config.maxAbsYaw = 25.0f;
config.maxAbsPitch = 25.0f;
config.maxAbsRoll = 20.0f;
// Pass config to HFCreateFaceCaptureSession with the enabled parent session.

Sharpness and brightness are also available as optional filters. Start with the default policy, inspect rejected frames and adjust one setting at a time to see how it affects capture.

Use progress.reject_reasons to choose a concrete prompt such as “move closer” or “keep your face inside the guide.” The metric mask identifies which values were calculated: metrics.available_filters in Python, or metrics.availableMetrics in C, C++, Java and ArkTS. Read the fields included in that mask.

Reuse a detection snapshot

Snapshot lifetime and copy cost

A snapshot copies detection results, making them easier to retain and manage for later processing. That copy adds overhead and latency. For a single video stream processed in order, the C, Objective-C and Swift borrowed-result paths can avoid this extra copy: read them completely before the next detection call. Later calls can overwrite borrowed data, so do not retain it across frames or interleave its use with other processing on the same session. Keep the matching image separately; a detection snapshot does not copy its pixels.

If your frame loop already needs detection results for an overlay, avoid running tracking twice:

C API

Create a new owned snapshot for this frame, read its face boxes while it is alive, then assess it with the existing capture object. progress is an output parameter.

static HResult capture_with_snapshot(HFSession session,
                                    HFFaceCaptureSession capture,
                                    HFImageStream stream, HFUInt64 frame_id,
                                    HFUInt64 timestamp_ms,
                                    HFFaceCaptureProgress *progress) {
    HFFaceResultSnapshot snapshot = NULL;
    HResult status = HFExecuteFaceTrackSnapshot(session, stream, &snapshot);
    if (status != HSUCCEED) return status;
    HFMultipleFaceData faces = {0};
    status = HFGetFaceResultSnapshotData(snapshot, &faces);
    if (status == HSUCCEED) {
        // Read or copy faces.rects here for the overlay.
        status = HFUpdateFaceCaptureSessionWithSnapshot(
            capture, stream, snapshot, frame_id, timestamp_ms, progress);
    }
    HResult release_status = HFReleaseFaceResultSnapshot(snapshot);
    return status == HSUCCEED ? release_status : status;
}

Create a new snapshot from the same session for each frame. It preserves that frame’s tracker count and geometry, so pair it with the corresponding image when updating capture.

Native APIs and lifetimes

The C entry points are HFCreateFaceCaptureSession, HFUpdateFaceCaptureSession, HFGetFaceCaptureResults, HFFinishFaceCaptureSession, HFResetFaceCaptureSession and HFReleaseFaceCaptureSession. Use HFGetDefaultFaceCaptureConfig to initialize the versioned configuration structure.

HFUpdateFaceCaptureSessionWithSnapshot accepts an owned detection snapshot. C, Objective-C and Swift result tokens are borrowed until the next capture update, reset, finish or release; copy what must outlive those calls. Release the capture object before releasing its parent session. The Python wrapper copies result face tokens and supports context managers for both resources. C++ capture candidates contain value copies of FaceTrackWrap; Java and ArkTS capture results copy their token bytes. Keep the corresponding image pixels in the application cache described above.