Using InspireFace in Android

InspireFace's Android SDK is based on dynamic library +CAPI and provides JNI interface for users to use.

Installation and Setup

Dependent Release version

We released InspireFace's Android SDK on JitPack, which you can incorporate into your android projects in the following ways.

  • Step 1. Add the JitPack repository to your build file add it in your root build.gradle at the end of repositories:
allprojects {
    repositories {
       ...
       maven { url 'https://jitpack.io' }
    }
}
  • Step 2. Add the dependency to your app's build.gradle file:
dependencies {
    implementation 'com.github.HyperInspire:inspireface-android-sdk:1.2.0'
}

Compile from source code

TODO

Initialization

Global initialization of InspireFace is necessary. You only need to perform it once when the program starts. The initialization includes functions such as configuration reading and model loading.

Java
// Launch InspireFace, only need to call once
boolean launchStatus = InspireFace.GlobalLaunch(this, InspireFace.PIKACHU);
if (!launchStatus) {
    Log.e(TAG, "Failed to launch InspireFace");
}

// .... 

// Global release
InspireFace.GlobalRelease();

Face Algorithm Session

InspireFace's facial analysis algorithms are all concentrated in the session. You can use the session instance to perform face recognition, face embedding extraction, face detection, face tracking, landmark localization, liveness detection, head pose estimation, attribute recognition, and other functions.

Since the session contains some cache, we recommend using one session within a thread, and we don't recommend cross-using internal data from multiple sessions in tracking mode, as this can easily cause confusion. Sessions can be freely created and destroyed anywhere.

Create Session

When creating a session, there are some important parameters that need to be specified:

  • Option: Features that need to be turned on, such as face recognition, mask detection, face attributes. This step will increase the memory used by the session
  • Detect Mode:
    • Always Detection: Face detection is performed each time, usually for image input or scenes where faces do not need to be tracked
    • Light Tracking: Lightweight face tracking algorithm, support input frame sequence tracking face, tracking speed is extremely fast
    • Tracking by Detection: With detector-dependent tracking, detection is performed every frame, with low speed and high precision
  • Max Faces: Limit the maximum number of faces to detect, if the number of faces is too large, the algorithm will be slow
  • Detect Pixel Level: Face detector level, the higher the more accurate, but also affect the execution speed, usually 160, 192, 256, 320, 640

Warning

When creating a session, it will use device memory, and as more options are enabled, the memory usage increases. Appropriately disabling some unnecessary features can save memory.

Java
boolean launchStatus = InspireFace.GlobalLaunch(this, InspireFace.PIKACHU);
Log.d(TAG, "Launch status: " + launchStatus);
if (!launchStatus) {
    Log.e(TAG, "Failed to launch InspireFace");
    return;
}
CustomParameter parameter = InspireFace.CreateCustomParameter()
        .enableRecognition(true)           // Enable face recognition
        .enableFaceQuality(true)           // Enable face quality detection
        .enableFaceAttribute(true)         // Enable face attribute detection
        .enableInteractionLiveness(true)   // Enable interaction liveness detection
        .enableLiveness(true)              // Enable liveness detection
        .enableMaskDetect(true);           // Enable mask detection
// Face detection level, 160/320/640
int detectLevel = 320;
// Supports the maximum number of faces detected
int maxFaces = 1;
// Create session
Session session = InspireFace.CreateSession(
    parameter, InspireFace.DETECT_MODE_ALWAYS_DETECT, maxFaces, detectLevel, -1);
// Configure some face detection parameters
InspireFace.SetTrackPreviewSize(session, 320);
InspireFace.SetFaceDetectThreshold(session, 0.5f);
InspireFace.SetFilterMinimumFacePixelSize(session, 0);
....

// Destroy session, when you don't need it
InspireFace.DestroySession(session);

Image Stream

Image stream is a data structure that stores image data, which is used to pass image data between the SDK and the user.

  • Create Image Stream: Create an image stream from a bitmap
Java
Bitmap img = SDKUtils.getImageFromAssetsFile(this, "inspireface/kun.jpg");
ImageStream stream = InspireFace.CreateImageStreamFromBitmap(img, InspireFace.CAMERA_ROTATION_0);
  • Create Image Stream from File: Create an image stream from buffer
Java
byte[] buffer = ...; // buffer of image/video/frame...
int height = 640;
int width = 480;
int format = InspireFace.STREAM_YUV_NV21;
int rotation = InspireFace.CAMERA_ROTATION_0;
ImageStream stream = InspireFace.CreateImageStreamFromByteBuffer(buffer, width, height, format, rotation);
  • Destroy Image Stream: Destroy an image stream

When you don't need an image stream, you can destroy it:

Java
InspireFace.ReleaseImageStream(stream);

Face Detection

Face detection is the first step in the analysis of faces, which requires the input of an image or frame:

Java
MultipleFaceData multipleFaceData = InspireFace.ExecuteFaceTrack(session, stream);
for (int i = 0; i < multipleFaceData.detectedNum; i++) {
    // continue to processing
    // ...
}

Get Face Embedding

Get face Embeding is an important step in face recognition, comparison or face swap, which usually needs to be carried out after face detection or tracking:

Java
int selectIndex = 0;    // Select an index
FaceFeature feature = InspireFace.ExtractFaceFeature(session, stream, multipleFaceData.tokens[selectIndex]);

Feature Hub

FeatureHub is a globally scoped database that manages face features with full support for create, read, update, and delete operations. It supports both in-memory and persistent storage modes and only needs to be configured once globally upon initialization.

Warning

Please be mindful when selecting the storage mode. If you choose the persistent mode, make sure to securely store the database file to prevent data loss.

Initialization and configuration

This operation is performed once to initialize FeatureHub, and need to select the storage mode to work.

  • enablePersistence: Enable persistence mode, If not enabled, it will only be stored in the current memory.
  • persistenceDbPath: After this function is enabled, you need to specify the path for saving the DB file.
Java
FeatureHubConfiguration configuration = InspireFace.CreateFeatureHubConfiguration()
        .setEnablePersistence(true)
        .setPersistenceDbPath(dbPath)
        .setSearchThreshold(0.42f)
        .setSearchMode(InspireFace.SEARCH_MODE_EXHAUSTIVE)
        .setPrimaryKeyMode(InspireFace.PK_AUTO_INCREMENT);

boolean enableStatus = InspireFace.FeatureHubDataEnable(configuration);
Log.d(TAG, "Enable feature hub data status: " + enableStatus);

Insert face

Insert a face into the database

Java
boolean succ = InspireFace.FeatureHubInsertFeature(identity);
// After successful insertion, you can save the id to your system
if (succ) {
    Log.i(TAG, "Allocation ID: " + identity.id);
}

Search face

Using the face embedding feature to search for similar faces.

  • Search for the most similar faces
Java
FaceFeatureIdentity searched = InspireFace.FeatureHubFaceSearch(feature);
Log.i(TAG, "Searched id: " + searched.id + ", Confidence: " + searched.searchConfidence);
  • Search for the most similar k faces
Java
SearchTopKResults topKResults = InspireFace.FeatureHubFaceSearchTopK(feature, 10);
for (int i = 0; i < topKResults.num; i++) {
    Log.i(TAG, "TopK id: " + topKResults.ids[i] + ", Confidence: " + topKResults.confidence[i]);
}

Update face

Specify an id to update face features.

Java
int updateId = 8;
newFeature.data = new float[InspireFace.GetFeatureLength()];
FaceFeatureIdentity identity = FaceFeatureIdentity.create(updateId, newFeature);
boolean updateSucc = InspireFace.FeatureHubFaceUpdate(identity);
if (updateSucc) {
    Log.i(TAG, "Update feature success: " + updateId);
}

Remove face

Specify an id to remove a face from the database.

Java
int removeId = 4;
boolean removeSucc = InspireFace.FeatureHubFaceRemove(removeId);
if (removeSucc) {
    Log.i(TAG, "Remove feature success: " + removeId);
}

Get face embedding

Gets the embedding of a face by id.

Java
int id = 4;
FaceFeatureIdentity identity = InspireFace.FeatureHubGetFaceIdentity(id);
Last Updated::
Contributors: tunm, Jingyu