Einstein Object Detection

Einstein custom object detection enables developers to train models to recognize and count multiple distinct objects within an image, providing granular details like the size and location of each object.

DocumentationExample Dataset

Getting Started

We’ll demonstrate how easy it is to create your own object detection model by creating a model which can recognize different cereal boxes. In the end this model will detect Alpine oat, bran, corn flake cereals while providing their x/y coordinates. Download the example dataset above for additional context.

Upload A Dataset

This command creates a dataset called alpine_data from the specified zip file. This zip file contains various images of Alpine oat, bran, and corn flake cereals as well as a csv file containing bounding box information for each cereal type.

$ curl -X POST  https://api.einstein.ai/v2/vision/datasets/upload/sync -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F name=alpine_data -F "type=image-detection" -F "path=https://einstein.ai/images/alpine.zip"
{
  "id": 1004848,
   "name": "alpine_dataset",
   "createdAt": "2017-11-30T02:06:39.000+0000",
   "updatedAt": "2017-11-30T02:06:39.000+0000",
   "labelSummary": {
    "labels": [
     {
      "id": 39292,
      "datasetId": 1004848,
      "name": "Alpine - Oat Cereal",
      "numExamples": 32
     },
     {
      "id": 39293,
      "datasetId": 1004848,
      "name": "Alpine - Corn Flakes",
      "numExamples": 30
     },
     {
      "id": 39294,
      "datasetId": 1004848,
      "name": "Alpine - Bran Cereal",
      "numExamples": 31
     }
    ]
   },
   "totalExamples": 33,
   "totalLabels": 3,
   "available": true,
   "statusMsg": "SUCCEEDED",
   "type": "image-detection",
   "object": "dataset"
  }

Train Your Model

Now that you’ve uploaded a labeled object detection dataset, it’s time to train your model. You will do this by using the command below and replacing the ID with the dataset you uploaded.

$ curl -X POST https://api.stg.einstein.ai/v2/vision/train -H "Authorization: Bearer <TOKEN>" -F name=<MODEL_NAME> -F datasetId=<DATASET_ID> -F type=image-detection -F epochs=30
{
  "datasetId": 1004848,
  "datasetVersionId": 0,
  "name": "alpine_model",
  "status": "QUEUED",
  "progress": 0,
  "createdAt": "2017-11-30T02:07:30.000+0000",
  "updatedAt": "2017-11-30T02:07:30.000+0000",
  "learningRate": 0,
  "epochs": 30,
  "queuePosition": 1,
  "object": "training",
  "modelId": "MQHMYBQEZIND5HUFPZMHFL5HDM",
  "trainParams": null,
  "trainStats": null,
  "modelType": "image-detection"
}

Predict

Now your model is ready to go! To test it out, send an image for prediction and the model will return labels, probabilities, and coordinates for each detected cereal.

$ curl -X POST https://api.stg.einstein.ai/v2/vision/detect -H "Authorization: Bearer <TOKEN>" -F modelId=<MODEL_ID> -F sampleLocation=https://einstein.ai/images/alpine.jpg
{
  "probabilities": [
    {
      "label": "Alpine - Oat Cereal",
      "probability": 0.98904955,
      "boundingBox": {
        "minX": 305,
        "minY": 130,
        "maxX": 390,
        "maxY": 262
      }
    },
    {
      "label": "Alpine - Oat Cereal",
      "probability": 0.97531146,
      "boundingBox": {
        "minX": 394,
        "minY": 259,
        "maxX": 485,
        "maxY": 399
      }
    },
    {
      "label": "Alpine - Corn Flakes",
      "probability": 0.9986804,
      "boundingBox": {
        "minX": 209,
        "minY": 134,
        "maxX": 302,
        "maxY": 260
      }
    },
    {
      "label": "Alpine - Corn Flakes",
      "probability": 0.99828976,
      "boundingBox": {
        "minX": 215,
        "minY": 258,
        "maxX": 305,
        "maxY": 385
      }
    },
    {
      "label": "Alpine - Bran Cereal",
      "probability": 0.9934196,
      "boundingBox": {
        "minX": 405,
        "minY": 126,
        "maxX": 503,
        "maxY": 258
      }
    },
    {
      "label": "Alpine - Bran Cereal",
      "probability": 0.9886825,
      "boundingBox": {
        "minX": 304,
        "minY": 259,
        "maxX": 386,
        "maxY": 390
      }
    }
  ]
}