InspireCV: Lightweight CV library

InspireCV is a lightweight computer vision library that provides high-level abstract interfaces for commonly used vision algorithms. It features a flexible backend architecture, allowing users to leverage a lightweight backend by default while also offering the option to switch to a more powerful OpenCV backend for enhanced performance.

InspireCV

  • Supports both OpenCV and custom OKCV backends
  • Core functionality includes:
    • Basic image processing operations
    • Geometric primitives (Point, Rect, Size)
    • Transform matrices
    • Image I/O
  • Minimal dependencies when using OKCV backend
  • Optional OpenCV integration for debugging and visualization

Tips

InspireCV was developed to reduce SDK size and avoid dependency issues by replacing OpenCV with a lightweight, project-tailored vision library.

Build Options

Backend Selection

  • INSPIRECV_BACKEND_OPENCV: Use OpenCV as the backend (OFF by default)
  • INSPIRECV_BACKEND_OKCV_USE_OPENCV: Enable OpenCV support in OKCV backend (OFF by default)
  • INSPIRECV_BACKEND_OKCV_USE_OPENCV_IO: Use OpenCV's image I/O in OKCV (OFF by default)
  • INSPIRECV_BACKEND_OKCV_USE_OPENCV_GUI: Use OpenCV's GUI features in OKCV (OFF by default)

Other Options

  • INSPIRECV_BUILD_SHARED_LIBS: Build as shared libraries (OFF by default)
  • INSPIRECV_OKCV_BUILD_TESTS: Build test suite (ON by default)
  • INSPIRECV_OKCV_BUILD_SAMPLE: Build sample applications (ON by default)

Dependencies

Required:

  • CMake 3.10+
  • Eigen3
  • C++14 compiler

Optional:

  • OpenCV (required if using OpenCV backend or OpenCV features in OKCV)

Use Guide

Image I/0

Image has multiple ways to load from file, buffer, or other sources. Default image type is 3-channel BGR image, like OpenCV.

  • Image Constructor
C++
// Load image from file
// Load with 3 channels (BGR, like opencv)
inspirecv::Image img = inspirecv::Image::Create("test_res/data/bulk/kun_cartoon_crop.jpg", 3);

// Other load methods

// Load image from buffer
uint8_t* buffer = ...;  // buffer is a pointer to the image data
bool is_alloc_mem = false;  // if true, will allocate memory for the image data,
                            // false is recommended to point to the original data to avoid copying
inspirecv::Image img = inspirecv::Image::Create(width, height, channel, buffer, is_alloc_mem);
  • Image Save and Show

Image supports multiple image formats, including PNG, JPG, BMP, etc. You can save image to file. If you want to show image, it must depend on OpenCV.

C++
// Save image to file
img.Write("output.jpg");

// Show image, warning: it must depend on opencv
img.Show("input");
  • Get pointer of Image
C++
// Get pointer to image data
const uint8_t* ptr = img.Data();

Image Processing

Image processing is a core functionality of InspireCV. It provides a set of functions to process images.

Take this original image for example:

KunKun

Features includes:

  • ToGray
C++
inspirecv::Image gray = img.ToGray();

Gray Image

  • Apply Gaussian blur
C++
inspirecv::Image blurred = img.GaussianBlur(3, 1.0);

Blurred Image

  • Resize
C++
auto scale = 0.35;
bool use_bilinear = true;
inspirecv::Image resized = img.Resize(img.Width() * scale, img.Height() * scale, use_bilinear);

Resized Image

  • Rotate
    • Support 90, 180, 270 clockwise degree rotation
C++
inspirecv::Image rotated = img.Rotate90();

Rotated Image

  • Flip
    • Support horizontal, vertical, and both flip
C++
inspirecv::Image flipped_vertical = img.FlipVertical();

Flipped Image

C++
inspirecv::Image flipped_horizontal = img.FlipHorizontal();

Flipped Image

  • Crop
C++
inspirecv::Rect<int> rect = inspirecv::Rect<int>::Create(78, 41, 171, 171);
inspirecv::Image cropped = img.Crop(rect);

Cropped Image

  • Padding
C++
int top = 50, bottom = 50, left = 50, right = 50;
inspirecv::Image padded = img.Pad(top, bottom, left, right, inspirecv::Color::Black);

Padded Image

  • Swap red and blue channels
C++
inspirecv::Image swapped = img.SwapRB();

Swapped Image

  • Multiply
C++
double scale_factor = 0.5;
inspirecv::Image scaled = img.Mul(scale_factor);

Scaled Image

  • Add
C++
double value = -175;
inspirecv::Image added = img.Add(value);

Added Image

  • Affine transform
    • Like warpAffine in OpenCV

Origin input is rotated 90 degree image, and the transform matrix is from face location:

Rotated Image

C++
/**
 * Create a transform matrix from the following matrix
 * [[a11, a12, tx],
 *  [a21, a22, ty]]
 *
 * Face crop transform matrix
 * [[0.0, -1.37626, 261.127],
 *  [1.37626, 0.0, 85.1831]]
*/
float a11 = 0.0f;
float a12 = -1.37626f;
float a21 = 1.37626f;
float a22 = 0.0f;
float b1 = 261.127f;
float b2 = 85.1831f;

// Create a transform matrix: Face location transform matrix
inspirecv::TransformMatrix trans = inspirecv::TransformMatrix::Create(a11, a12, b1, a21, a22, b2);

// dst_width and dst_height is the size of the output image
int dst_width = 112;
int dst_height = 112;

// Apply affine transform
inspirecv::Image affine = rotated_90.WarpAffine(trans, dst_width, dst_height);

Affine Image

Image Draw

Image draw is a core functionality of InspireCV. It provides a set of functions to draw on images.

  • Draw rectangle
C++
inspirecv::Rect<int> new_rect = rect.Square(1.1f);  // Square and expand the rect
int thickness = 3;
draw_img.DrawRect(new_rect, inspirecv::Color::Green, thickness);

Draw Rectangle

  • Draw circle
C++
std::vector<inspirecv::Point<int>> points = new_rect.As<int>().ToFourVertices();
for (auto& point : points) {
    draw_img.DrawCircle(point, 1, inspirecv::Color::Red, 5);
}

Draw Circle

  • Draw Lines
C++
draw_img.DrawLine(points[0], points[1], inspirecv::Color::Cyan, 2);
draw_img.DrawLine(points[1], points[2], inspirecv::Color::Magenta, 2);
draw_img.DrawLine(points[2], points[3], inspirecv::Color::Pink, 2);
draw_img.DrawLine(points[3], points[0], inspirecv::Color::Yellow, 2);

Draw Lines

  • Fill
C++
draw_img.Fill(new_rect, inspirecv::Color::Purple);

Fill

  • Reset
C++
// Reset image to gray
std::vector<uint8_t> gray_color(img.Width() * img.Height() * 3, 128);
img.Reset(img.Width(), img.Height(), 3, gray_color.data());

Reset

Frame Process

To streamline image processing, we designed a frame processor that wraps around the input image, providing flexible support for frame sequences such as images or video streams. It integrates a processing pipeline with built-in image decoding (BGR, RGB, BGRA, RGBA, YUV, NV12, NV21), rotation, scaling, and affine transformations, while optimizing internal buffering for enhanced performance.

Warning

FrameProcess is an InspireFace module and is not yet integrated into the InspireCV library.

Create Frame Processor

C++
// BGR888 as raw data
inspirecv::Image raw = inspirecv::Image::Create("test_res/data/bulk/kun_cartoon_crop_r90.jpg", 3);
const uint8_t* buffer = raw.Data();

// You can also use other image format, like NV21, NV12, RGBA, RGB, BGR, BGRA
const uint8_t* buffer = ...;

// Create frame process
auto width = raw.Width();
auto height = raw.Height();
auto rotation_mode = inspirecv::ROTATION_90;
auto data_format = inspirecv::BGR;
inspirecv::FrameProcess frame_process = inspirecv::FrameProcess::Create(buffer, height, width, data_format, rotation_mode);

Example of raw data:

Resized Image

Pipeline

  • Set preview size
C++
// Set preview size
frame_process.SetPreviewSize(160);

// or

// Set preview scale
frame_process.SetPreviewScale(0.5f);
  • Get transform image
    • Will rotate and scale the image to the preview size
C++
inspirecv::Image transform_img = frame_process.GetTransformImage();

Transform Image

  • Get affine processing image
C++
/** 
 * Face crop transform matrix
 * [[0.0, 0.726607, -61.8946],
 *  [-0.726607, 0.0, 189.737]]
*/

// Face crop transform matrix
float a11 = 0.0f;
float a12 = 0.726607f;
float a21 = -0.726607;
float a22 = 0.0f;
float b1 = -61.8946f;
float b2 = 189.737f;
inspirecv::TransformMatrix affine_matrix = inspirecv::TransformMatrix::Create(a11, a12, b1, a21, a22, b2);
int dst_width = 112;
int dst_height = 112;
inspirecv::Image affine_img = frame_process.ExecuteImageAffineProcessing(affine_matrix, dst_width, dst_height);

Affine Processing Image

Performance Considerations

  • The library uses Eigen3 for efficient matrix operations
  • OKCV backend provides lightweight alternatives to OpenCV
  • Operations are designed to minimize memory allocations
  • Thread-safe operations for parallel processing

Thread Safety

The library is designed to be thread-safe. You can use it in multi-threaded applications.

Error Handling

The library uses error codes and exceptions to handle error conditions:

  • Image loading/saving errors
  • Invalid parameters
  • Memory allocation failures
  • Backend-specific errors

Errors can be caught using standard try-catch blocks:

C++
try {
    Image img = Image::Create("nonexistent.jpg");
} catch (const std::exception& e) {
    std::cerr << "Error: " << e.what() << std::endl;
}
Last Updated::
Contributors: Jingyu