AI / ML: Convert Keras Model to iOS ML Model

In 2017, Apple announced CoreML, a move towards on device machine learning. I was excited to jump right in, so I developed a few ML iOS apps to leverage the new technology. One of these apps, Recipik, was an app that used a Neural Network to detect food and then suggest the ingredients in that food with the goal of helping those with allergies be able to have a better idea of what might be in a food, when they don't know what the food is.

I was constrained back then, because I didn't know how to create an iOS .mlmodel, so I simply found my models from GitHub pages where people had already accumulated them.

Want to just skip the article and find a model like I did a few years ago? Then check out the resource section at the bottom of this article!

This is great for many projects, but if you have any custom requirements, or aren't able to find a model that is already out there, then you are blocked. In this post, I hope to show how you can convert Keras models to iOS .mlmodels, and by extension, how you can train your own models in Jupyter and use them in your iOS applications.

Keras Cat and Dog Model

For our example, I wanted to use a common classification CNN that can be found on Kaggle. The Keras model that we are going to be converting can be downloaded from GitHub: Keras Cat Dog CNN.

Import libraries and define constants

We will need to start with our usual imports. Naturally, as we are using a Keras model, we are going to need a few utilities from Keras. Let's also define our constants that we will use for transformation of images.

# Import Libraries
import numpy as np
import pandas as pd 
from keras.preprocessing.image import load_img, image
from keras.models import Sequential
import os

IMAGE_WIDTH=128
IMAGE_HEIGHT=128
IMAGE_SIZE=(IMAGE_WIDTH, IMAGE_HEIGHT)
IMAGE_CHANNELS=3

Define Model Architecture

This is only necessary because we are loading a model from previously saved model weights. If you are training your model from scratch, then this won't be an important step. This is only important because we need to define the model architecture of our Keras model, so that it matches with our pre trained model weights.

from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense, Activation, BatchNormalization

def createModel():
    model = Sequential()

    model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_CHANNELS)))
    model.add(BatchNormalization())
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Conv2D(64, (3, 3), activation='relu'))
    model.add(BatchNormalization())
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Conv2D(128, (3, 3), activation='relu'))
    model.add(BatchNormalization())
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Flatten())
    model.add(Dense(512, activation='relu'))
    model.add(BatchNormalization())
    model.add(Dropout(0.5))
    model.add(Dense(2, activation='softmax'))

    model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
    return model

Load model into memory

With our createModel function defined, we can now use it to generate our new model and load the weights of our model.

from keras.models import load_model

model = createModel()
model.load_weights('model.h5')

I don't want you to get stuck in the weeds with the architecture, the innerworkings of a convolutional neural network, and what is going on around the model creation. The last code snippet is simply loading the model into memory so that we can convert it. Which is exactly what we will do in our next snippet!

Test model

Just as a fun aside, with the model loaded into memory, we can test it out! This is a great checkpoint as well to make sure that everything is working when you loaded up your model correctly. The model predicts if an image is of a dog or cat, so go ahead and give it a try!

def load(filename: str):
    img = image.load_img(filename, target_size = IMAGE_SIZE)
    img = image.img_to_array(img)
    img = np.expand_dims(img, axis = 0)
    return img

img = load('dog.jpg')

result = model.predict(img)

if result[0][0] == 1:
    print('That is a cat')
else:
    print('That is a dog')

That is a dog 🐕

Convert model to .mlmodel 🚀

The conversion can be done with an ML framework developed by Apple: coremltools. We are going to use this tool to do the conversion.

Coremltools has a known issue with compatibility with TensorFlow v2. If you are running into problems, try downgrading TensorFlow to version v1.14.0.

import coremltools

new_model = coremltools.converters.keras.convert(model,
    input_names="image",
    image_input_names="image",
    image_scale=1/255.0)

new_model.save('cat_dog.mlmodel')

Success! 🥳

Conclusion

When I realized that I could convert any model to an .mlmodel format, it opened up so many ideas for me! If you are working with AI/ML and want to move some of your ML inferences to iOS, then check out converting the models so that you can have quick inferences on device!

Good luck!

Resources

• Core ML Models List - Awesome CoreML Models
• Full source code for converting model - Keras model to ios .mlmodel
• Keras documentation - Keras
• Tensorflow documentation - Tensorflow
• Coremltools documentation - coremltools