Flutter, Google’s UI toolkit for building natively compiled applications for mobile, web, and desktop from a single codebase, is becoming increasingly popular. Integrating machine learning (ML) into Flutter apps can add powerful features such as image recognition, text analysis, and predictive analytics. This blog post guides you through implementing machine learning in Flutter.
Why Integrate Machine Learning into Flutter?
Integrating machine learning into Flutter applications enhances the app’s capabilities, offering personalized and intelligent user experiences. Here’s why you should consider implementing ML in your Flutter apps:
- Enhanced User Experience: ML can personalize user interactions by providing intelligent suggestions and predictions.
- Automation: Automate tasks like image classification, text analysis, and data processing.
- Data-Driven Insights: Use ML to extract valuable insights from user data and improve decision-making.
- Competitive Advantage: Incorporating unique ML features can set your app apart from competitors.
Prerequisites
Before diving into the implementation, ensure you have the following:
- Flutter SDK installed on your machine.
- An IDE (e.g., VS Code, Android Studio) with Flutter plugins installed.
- Basic understanding of Dart programming language.
Steps to Implement Machine Learning in Flutter
There are several ways to integrate machine learning into a Flutter application, including using pre-trained models or building custom models. This guide covers integrating pre-trained models using TensorFlow Lite.
Step 1: Add TensorFlow Lite Dependencies
Add the necessary dependencies to your pubspec.yaml file:
dependencies:
flutter:
sdk: flutter
tflite: ^1.1.2
image_picker: ^0.8.4+4
image: ^3.0.2
Explanation of the dependencies:
tflite: TensorFlow Lite plugin for Flutter.image_picker: Allows users to pick images from the gallery or camera.image: Provides image processing functionalities.
After adding the dependencies, run flutter pub get in the terminal to install them.
Step 2: Import TensorFlow Lite Model
Download a pre-trained TensorFlow Lite model (.tflite file). For example, you can use a model for image classification.
Create an assets folder in your Flutter project and place the .tflite model and a label file (labels.txt) in this folder.
Update the pubspec.yaml file to include the assets:
flutter:
assets:
- assets/model.tflite
- assets/labels.txt
Step 3: Load the Model
Load the TensorFlow Lite model in your Flutter app. Create a method to load the model when the app starts.
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:tflite/tflite.dart';
import 'dart:io';
import 'package:image_picker/image_picker.dart';
import 'package:image/image.dart' as img;
void main() => runApp(MyApp());
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
home: MyHomePage(),
);
}
}
class MyHomePage extends StatefulWidget {
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State {
File? _image;
List? _recognitions;
double? _imageWidth;
double? _imageHeight;
bool _busy = false;
Future loadModel() async {
try {
Tflite.close();
String? res = await Tflite.loadModel(
model: "assets/model.tflite",
labels: "assets/labels.txt",
);
print('Model loaded: $res');
} on PlatformException {
print('Failed to load the model.');
}
}
@override
void initState() {
super.initState();
_busy = true;
loadModel().then((val) {
setState(() {
_busy = false;
});
});
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: const Text('TensorFlow Lite Demo'),
),
body: (_busy)
? const Center(
child: CircularProgressIndicator(),
)
: Center(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: <Widget>[
_image == null
? const Text('No image selected.')
: Column(
children: [
Image.file(File(_image!.path)),
if (_recognitions != null)
Column(
children: _recognitions!.map((res) {
return Text(
"${res["label"]}: ${res["confidence"].toStringAsFixed(3)}",
);
}).toList(),
),
],
),
ElevatedButton(
onPressed: getImage,
child: const Text('Pick an image'),
),
],
),
),
);
}
Step 4: Implement Image Picking
Use the image_picker package to allow the user to pick an image from the gallery or camera. Implement the getImage method.
Future getImage() async {
final pickedFile = await ImagePicker().pickImage(source: ImageSource.gallery);
if (pickedFile == null) return;
setState(() {
_busy = true;
_image = File(pickedFile.path);
});
// Resize the image
img.Image? image = img.decodeImage(await _image!.readAsBytes());
if (image == null) return;
img.Image resizedImage = img.copyResize(image, width: 224, height: 224);
File resizedFile = await File(_image!.path).writeAsBytes(img.encodeJpg(resizedImage));
predictImage(resizedFile);
}
Step 5: Create the Predict Image Function
Create a function that takes the image as input and performs the image classification.
Future predictImage(File image) async {
if (image == null) return;
int startTime = DateTime.now().millisecondsSinceEpoch;
List? recognitions = await Tflite.runModelOnImage(
path: image.path,
imageMean: 0.0, // defaults to 117.0
imageStd: 255.0, // defaults to 1.0
numResults: 3, // defaults to 5
threshold: 0.2, // defaults to 0.1
);
setState(() {
_recognitions = recognitions;
});
int endTime = DateTime.now().millisecondsSinceEpoch;
print("Inference took ${endTime - startTime}ms");
setState(() {
_busy = false;
});
}
This method:
- Takes an image file path as input.
- Uses
Tflite.runModelOnImageto perform inference on the image. - Updates the
_recognitionslist with the results. - Calculates and prints the inference time.
Step 6: Display Predictions
Display the image and the prediction results in the Flutter UI.
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:tflite/tflite.dart';
import 'dart:io';
import 'package:image_picker/image_picker.dart';
import 'package:image/image.dart' as img;
void main() => runApp(MyApp());
class MyApp extends StatelessWidget {
@override
Widget build(BuildContext context) {
return MaterialApp(
home: MyHomePage(),
);
}
}
class MyHomePage extends StatefulWidget {
@override
_MyHomePageState createState() => _MyHomePageState();
}
class _MyHomePageState extends State {
File? _image;
List? _recognitions;
double? _imageWidth;
double? _imageHeight;
bool _busy = false;
Future loadModel() async {
try {
Tflite.close();
String? res = await Tflite.loadModel(
model: "assets/model.tflite",
labels: "assets/labels.txt",
);
print('Model loaded: $res');
} on PlatformException {
print('Failed to load the model.');
}
}
@override
void initState() {
super.initState();
_busy = true;
loadModel().then((val) {
setState(() {
_busy = false;
});
});
}
Future getImage() async {
final pickedFile = await ImagePicker().pickImage(source: ImageSource.gallery);
if (pickedFile == null) return;
setState(() {
_busy = true;
_image = File(pickedFile.path);
});
// Resize the image
img.Image? image = img.decodeImage(await _image!.readAsBytes());
if (image == null) return;
img.Image resizedImage = img.copyResize(image, width: 224, height: 224);
File resizedFile = await File(_image!.path).writeAsBytes(img.encodeJpg(resizedImage));
predictImage(resizedFile);
}
Future predictImage(File image) async {
if (image == null) return;
int startTime = DateTime.now().millisecondsSinceEpoch;
List? recognitions = await Tflite.runModelOnImage(
path: image.path,
imageMean: 0.0, // defaults to 117.0
imageStd: 255.0, // defaults to 1.0
numResults: 3, // defaults to 5
threshold: 0.2, // defaults to 0.1
);
setState(() {
_recognitions = recognitions;
});
int endTime = DateTime.now().millisecondsSinceEpoch;
print("Inference took ${endTime - startTime}ms");
setState(() {
_busy = false;
});
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: const Text('TensorFlow Lite Demo'),
),
body: (_busy)
? const Center(
child: CircularProgressIndicator(),
)
: Center(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: <Widget>[
_image == null
? const Text('No image selected.')
: Column(
children: [
Image.file(File(_image!.path)),
if (_recognitions != null)
Column(
children: _recognitions!.map((res) {
return Text(
"${res["label"]}: ${res["confidence"].toStringAsFixed(3)}",
);
}).toList(),
),
],
),
ElevatedButton(
onPressed: getImage,
child: const Text('Pick an image'),
),
],
),
),
);
}
}
Explanation:
_image: Stores the selected image file._recognitions: Stores the list of recognition results._busy: Indicates whether the model is currently processing.
Running the Application
Connect a device or emulator and run the Flutter application. You can pick an image from the gallery, and the app will display the image along with the predictions.
Advanced Techniques
Using Custom Models
For specific use cases, you might need to train your own models using TensorFlow or other ML frameworks. Once trained, convert the model to TensorFlow Lite format for integration with Flutter apps.
Real-time Predictions
For real-time predictions, such as processing camera frames, use the camera package to capture frames and process them continuously using the TensorFlow Lite model.
Performance Optimization
Optimize the performance of your ML models by:
- Using quantized models.
- Reducing the model size.
- Using hardware acceleration (GPU) if available.
Conclusion
Integrating machine learning into Flutter applications can significantly enhance user experience and functionality. By leveraging TensorFlow Lite and other tools, you can create intelligent, data-driven Flutter apps that stand out. Whether you’re classifying images, analyzing text, or making predictions, Flutter provides a versatile platform for implementing machine learning solutions.