Deploying Image Classification with TensorFlow Lite on Android

Training artificial intelligence (AI) to recognize images is analogous to teaching a child, perhaps with some coding. After the success of TensorFlow as a robust machine learning framework, TensorFlow Lite is reinforcing machine learning on mobile devices. As a well-experienced provider of TensorFlow development services, Oodles AI presents a comprehensive guide to deploy image classification with TensorFlow Lite.

What is TensorFlow Lite?

TensorFlow Lite (.TFLITE) is a lighter version of Google’s open-source machine learning framework, TensorFlow. The lightweight solution, TensorFlow Lite, is uniquely designed to run machine learning models on mobile and embedded devices.

Built to support the memory and compute constraints of quantized devices, TensorFlow Lite is emerging as a great initiative for running serverless ML applications.

However, .TFLITE can only deploy pre-existing models and a suite of tools to prepare models for mobile and embedded devices. For training models from scratch, one needs to incur a round trip to TensorFlow and convert the model to .TFLITE format. The converted model can then be pushed into a mobile interpreter with support via Java and C++ API, as demonstrated in the architecture below-

For hardware acceleration in Android devices, the interpreter can also use the Android Neural Networks API.

TensorFlow Lite’s two components, namely interpreter and converter enables developers to perform machine learning “at the edge” that improves-

a) Latency for the device

b) Privacy for on-device data

c) Connectivity, as no internet collection is required and

d) Power Consumption, with no network connections.

Also read- Deploying TensorFlow and Keras for Deep Learning Models

How Image Classification with TensorFlow Lite Works

Image classification using machine learning frameworks automates the identification of people, animals, places, and activities in an image. With domain-specific training, image classification models can predict what an image represents from fruits to food and more.

For instance, a model can identify different types of fruits, such as banana, guava, or mango. The motive is to employ machines for simple image identification tasks to accelerate routine business processes from healthcare to visual search, and more.

TensorFlow Lite is well-equipped to work with common image classification models such as Inception V3 and V4, MobileNets, NASNet mobile. The workflow for image classification with TensorFlow Lite involves four major steps beginning with-

Step 1: Picking a model

One can either train a model using TensorFlow and convert it into .TFLITE format or use a pre-trained model provided by Google. ML models, including image classification, object detection, smart reply, etc. can be re-trained to process new categories of inputs. The method of re-training is often called “transfer learning” that is considerably more convenient than training from scratch.

Step 2: Converting the model

Available as a Python API, the TensorFlow Lite converter executes the conversion of TensorFlow models into .TFLITE formats. This can be done either by running a few lines of code or by fetching the Python API, as shown below-

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)

tflite_model = converter.convert()

open(“converted_model.tflite”, “wb”).write(tflite_model)

Step 3: Running inference

The third step involves running data through the model to receive predictions, a process that is called “inference”. The process requires the model, interpreter, and data inputs. TensorFlow Lite’s interpreter can be triggered by Java, Swift, Objective-D, C++, and Python via a simple API. For hardware acceleration, TensorFlow Lite can be configured with Delegates including mobile GPU delegates to run applications, coded as below-

GpuDelegate delegate = new GpuDelegate();

Interpreter.Options options = (new

Interpreter.Options()).addDelegate(delegate);

Interpreter interpreter = new Interpreter(tensorflow_lite_model_file, options);

try {

  interpreter.run(input, output);

}

To run image classification models on Android, it is recommended to use TensorFlow Lite AAR that includes binaries for Android ABIs. Developers can even reduce application size by omitting unnecessary ABIs or build custom binaries as well.

Step 4: Optimizing the model

It is easier to optimize the size and performance of image classification models using TensorFlow Lite while maintaining accuracy. To strike an ideal balance between performance, size, and accuracy, it is vital to choose a candidate model wisely.

For improving the speed and efficiency of Android models, developers can use Neural Network API by calling UseAAPI on the interpreter instance, as shown below-

/// Enable or disable the NN API (true to enable)

void UseNNAPI(bool enable);

Now, with all these tips and tricks to initiate image classification with TensorFlow Lite, let’s explore some of its effective applications.

Also read- Innovative TensorFlow Applications for Machine Learning Models

Example Applications of TensorFlow Lite for Image Classification

1) Recognizing Medical Images

The current COVID-19 crisis has crippled healthcare infrastructures worldwide with no drug yet in sight. To flatten the curve, researchers and analysts are turning to emerging technologies for automating critical processes such as

a) Image recognition

b) Contactless Screening

c) Social Distancing enforcement

d) Testing kit delivery, and much more.

Image recognition with TensorFlow Lite can provide an efficient and easily accessible medical image identification tool for doctors. Model training with X-rays, CT scans, and MRIs can be done to automate the detection of infection and other anomalies.

An example of biomedical image analysis using TensorFlow for image segmentation and feature extraction purposes.

2) Creating Searchable Product Listing

As more and more businesses go digital, demand for automated data capture, extraction, and segmentation solutions are also rising exponentially. Under this quarantine phase, eCommerce businesses can employ artificial intelligence to work for optimizing searchable product listings.

Machine learning for auto-classifying online products or services can be a great tool for organizing millions of products on an eCommerce portal. With TensorFlow, businesses can achieve seamless product search, category classification, and more.

Also read- TensorFlow Vs H2O: The Best Enterprise-grade Machine Learning Tool

Using TensorFlow Lite to Build Image Classification Models with Oodles

We, at Oodles, are well-positioned providers of artificial intelligence services for enterprises and organizations. We have hands-on experience in deploying third-party machine learning frameworks such as TensorFlow and TensorFlow Lite for-

a) Image Recognition

b) Object Detection, and

c) Image caption generation

Our AI team has built and deployed several efficient and low latency ML models using TensorFlow. We are exploring the applications of TensorFlow Lite for image classification that accelerates business processes significantly. Connect with our AI team to learn more about our AI and ML services.