LandingLens | LandingLens on Snowflake |
---|---|
(see exceptions below) |
"OK"
or "NG"
(short for “Not Good”). Because the process has two steps, the JSON schema for Cloud Deployment includes two sets of predictions:
backbonetype
and backbonepredictions
: These elements contain the data for the actual model that ran inference on the image.type
and predictions
: These elements contain the data for the Classification step.type
and predictions
elements.
However, you can run a script that adds a post-processing model or Classification step when running inference with LandingEdge or Docker. If a post-processing step is added, the JSON will have the same prediction elements that are used for Cloud Deployments. In other words, the model predictions will move to the backbonetype
and backbonepredictions
elements, and the predictions for the post-processing step will be in the type
and predictions
elements.
Element | Description |
---|---|
backbonetype (Cloud Deployment)type (LandingEdge, Docker) | The name of the prediction type. For Object Detection, this will always be ObjectDetectionPrediction . |
backbonepredictions (Cloud Deployment)predictions (LandingEdge, Docker) | This object contains the information for each region that the model predicted as an object of interest. Each prediction is a separate object nested in this element. |
xxxxxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx | This alphanumeric number is an internal identifier for the region that the model predicted as an object of interest. |
score | The confidence score for the prediction. |
labelName | The predicted class. |
labelIndex | The internal identifier of the predicted class. Each class in a project is assigned a number, starting with 0. If you delete a class, that class’s number is not re-assigned to any current or future classes in that project. |
coordinates | The x and y coordinates (in pixels) of the edges of the bounding box around the predicted object of interest. |
defect_id | The unique identifier for the predicted class. |
predictions (Cloud Deployment) | When an image is sent for inference via Cloud Deployment, it goes through the model, and then through a post-processing Classification step. The predictions object contains the results from the Classification step. This step classifies the image as "OK" or "NG" (short for “Not Good”).This step simplifies defect detection. For example, if a model checks sheet metal for scratches, the metal should be marked as defective regardless if there are one or many scratches. score : Confidence score for the Classification prediction.labelName : If the number of predictions is > 0 then "NG" , else "OK" .This element is only applicable to Cloud Deployment. Inference run via LandingEdge and Docker does not include the post-processing Classification step. |
"type": "ClassificationPrediction" (Cloud Deployment) | When an image is sent for inference via Cloud Deployment, it goes through the model, and then through a post-processing Classification step. This object is the prediction type of the post-processing Classification step. This element is only applicable to Cloud Deployment. Inference run via LandingEdge and Docker does not include the post-processing Classification step. |
metadata (LandingEdge, Docker) | This element contains nested metadata from the image. If the Image Source is Folder Watcher, some data is populated by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
image_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the file name of the image. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
inspection_station_id (LandingEdge, Docker) | This element is blank by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
location_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the directory that the image is in. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
capture_timestamp (LandingEdge, Docker) | The time and date that OCR was run on the image. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
latency | The latency object includes the detailed timing statistics of the inference call.Each key-value pair in the latency object represents a step in the inference process, and the duration of that step. All values are measured in seconds. |
model_id | The unique identifier for the model. For more information, go here. |
coordinates
object, which communicates the size and location of the bounding box around the prediction region of interest.
Each section includes the coordinates of the pixels for each edge of the bounding box. The origin for the coordinates is the top left corner of the image.
For example, let’s say that the following snippet is the coordinate
object in the JSON output for a prediction.
Name | Description | Coordinate (in pixels) |
---|---|---|
xmin | The distance from the left edge of the image to the left side of the bounding box. | 3627 |
ymin | The distance from the top of the image to the top side of the bounding box. | 979 |
xmax | The distance from the left edge of the image to the right side of the bounding box. | 3965 |
ymax | The distance from the top of the image to the bottom side of the bounding box. | 1299 |
backbonepredictions
object.
This JSON response is from Cloud Deployment.
Element | Description |
---|---|
backbonetype (Cloud Deployment)type (LandingEdge, Docker) | The name of the prediction type. For Segmentation, this will always be SegmentationPrediction . For Visual Prompting, this will always be null . |
backbonepredictions (Cloud Deployment)predictions (LandingEdge, Docker) | For Segmentation, this object contains information about the image and the model’s predictions. For Visual Prompting, this will always be null . |
imageHeight | The height of the image in pixels. |
imageWidth | The width of the image in pixels. |
encoding | The information about how LandingLens encodes Segmentation predictions. Predictions are encoded with run-length encoding (RLE). 0 is mapped to Z, and 1 is mapped to N. This section will always be: “encoding”: { "algorithm": "rle", "options": { "map": { "Z": 0, "N": 1 } } } , |
bitmaps | This object contains the information for each class that the model predicted in the image. Each predicted class is a separate object nested in the bitmaps object.LandingEdge includes data for all classes in the project, and not just the ones predicted in the image. |
xxxxxxxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx | This alphanumeric number is an internal identifier for the region that the model predicted as an object of interest. The prediction information is nested in this object. |
score | The confidence score for the prediction. |
bitmap | The prediction mask (the areas that the model predicted for a specific class) is formatted as a bitmap that is compressed with a run-length encoding (RLE) format. The Python library parses the JSON responses, which includes parsing and extracting bitmaps. We recommend using the library if you want to avoid manually parsing the bitmaps. Check out the Satellite Images and Post-Processing tutorial for an end-to-end example of running inference and visualizing the results with our Python library. You can use the decode_bitmap_rle API to decode the bitmap string into a NumPy array, which you can then convert into a PNG file. |
defect_id | The unique identifier for the predicted class. |
labelIndex | The internal identifier of the predicted class. Each class in a project is assigned a number, starting with 0. If you delete a class, that class’s number is not re-assigned to any current or future classes in that project. |
labelName | The predicted class. |
data (Cloud Deployment) | This will always be null .This element only displays in the JSON response for Cloud Deployment. |
labels (Cloud Deployment) | This will always be null .This element only displays in the JSON response for Cloud Deployment. |
num_classes (Cloud Deployment) | This is the total number of classes in the project. Segmentation and Visual Prompting projects have an internal class called “ok” . This class is included in the num_class object. So if you created 3 classes in the project, the num_class value is 4.This element only displays in the JSON response for Cloud Deployment. |
predictions (Cloud Deployment) | When an image is sent for inference via Segmentation or Visual Prompting, it goes through the model, and then through a post-processing Classification step. The predictions object contains the results from the Classification step. This step classifies the image as "OK" or "NG" (short for “Not Good”).This step simplifies defect detection. For example, if a model checks sheet metal for scratches, the metal should be marked as defective regardless if there are one or many scratches. score : Confidence score for the Classification prediction.labelName : If the number of predictions is > 0 then "NG" , else "OK" .This element is only applicable to Cloud Deployment. Inference run via LandingEdge and Docker does not include the post-processing Classification step. |
"type": "ClassificationPrediction" (Cloud Deployment) | When an image is sent for inference via Cloud Deployment, it goes through the model, and then through a post-processing Classification step. This object is the prediction type of the post-processing Classification step. This element is only applicable to Cloud Deployment. Inference run via LandingEdge and Docker does not include the post-processing Classification step. |
metadata (LandingEdge, Docker) | This element contains nested metadata from the image. If the Image Source is Folder Watcher, some data is populated by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
image_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the file name of the image. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
inspection_station_id (LandingEdge, Docker) | This element is blank by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
location_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the directory that the image is in. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
capture_timestamp (LandingEdge, Docker) | The time and date that OCR was run on the image. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
latency | The latency object includes the detailed timing statistics of the inference call.Each key-value pair in the latency object represents a step in the inference process, and the duration of that step. All values are measured in seconds. |
model_id | The unique identifier for the model. For more information, go here. |
bitmap
object, because these regions all correspond to the Chipped class.
This JSON response is from Cloud Deployment.
In the image, the purple overlay is the prediction for Chipped.
bitmaps
object.
This JSON response is from Cloud Deployment.
In the image, the purple overlay is the prediction for Shipping Containers, and the yellow overlay is the prediction for Other.
Element | Description |
---|---|
backbonetype (Cloud Deployment) | This object isn’t applicable to Classification models, because the prediction type is captured in the type object. For Classification, this will always be null .This element only displays in the JSON response for Cloud Deployment. |
backbonepredictions (Cloud Deployment) | This object isn’t applicable to Classification models, because the prediction type is captured in predictions. For Classification, this will always be null .This element only displays in the JSON response for Cloud Deployment. |
predictions | This object contains the information for the model’s prediction. |
score | The confidence score for the prediction. |
labelName | The predicted class. |
labelIndex | The internal identifier of the predicted class. Each class in a project is assigned a number, starting with 0. If you delete a class, that class’s number is not re-assigned to any current or future classes in that project. |
defectId (LandingEdge, Docker) | The unique identifier for the predicted class. This element only displays in the JSON response for LandingEdge and Docker. |
rawScores (LandingEdge, Docker) | This element only displays in the JSON response for LandingEdge and Docker. |
"type": "ClassificationPrediction" | The name of the prediction type. For Classification, this will always be ClassificationPrediction. |
metadata (LandingEdge, Docker) | This element contains nested metadata from the image. If the Image Source is Folder Watcher, some data is populated by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
image_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the file name of the image. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
inspection_station_id (LandingEdge, Docker) | This element is blank by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
location_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the directory that the image is in. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
capture_timestamp (LandingEdge, Docker) | The time and date that OCR was run on the image. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
latency | The latency object includes the detailed timing statistics of the inference call.Each key-value pair in the latency object represents a step in the inference process, and the duration of that step. All values are measured in seconds. |
model_id | The unique identifier for the model. For more information, go here. |
predictions
object.
This JSON response is from Cloud Deployment.
Element | Description |
---|---|
backbonetype (Cloud Deployment)type (LandingEdge, Docker) | The name of the prediction type. For Anomaly Detection, this will always be AnomalyDetectionPrediction . |
backbonepredictions (Cloud Deployment)predictions (LandingEdge, Docker) | This object contains the prediction information. |
score | The confidence score for the prediction. |
defect_id | The unique identifier for the predicted class. |
labelIndex | The internal identifier of the predicted class. Normal is 0. Abnormal is 1. |
labelName | The predicted class. |
predictions (Cloud Deployment) | When an image is sent for inference via Cloud Deployment, it goes through the model, and then through a post-processing Classification step. The predictions object contains the results from the Classification step. This step classifies the image as "OK" or "NG" (short for “Not Good”).This step simplifies defect detection. For example, if a model checks sheet metal for scratches, the metal should be marked as defective regardless if there are one or many scratches. score : Confidence score for the Classification prediction.labelName : If the number of predictions is > 0 then "NG" , else "OK" .This element is only applicable to Cloud Deployment. Inference run via LandingEdge and Docker does not include the post-processing Classification step. |
"type": "ClassificationPrediction" (Cloud Deployment) | When an image is sent for inference via Cloud Deployment, it goes through the model, and then through a post-processing Classification step. This object is the prediction type of the post-processing Classification step. This element is only applicable to Cloud Deployment. Inference run via LandingEdge and Docker does not include the post-processing Classification step. |
metadata (LandingEdge, Docker) | This element contains nested metadata from the image. If the Image Source is Folder Watcher, some data is populated by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
image_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the file name of the image. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
inspection_station_id (LandingEdge, Docker) | This element is blank by default. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
location_id (LandingEdge, Docker) | If the Image Source is Folder Watcher, this is the directory that the image is in. Otherwise, this object is blank. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
capture_timestamp (LandingEdge, Docker) | The time and date that OCR was run on the image. You can use scripts and web APIs to set or override the values. This element only displays in the JSON response for LandingEdge and Docker. |
latency | The latency object includes the detailed timing statistics of the inference call.Each key-value pair in the latency object represents a step in the inference process, and the duration of that step. All values are measured in seconds. |
model_id | The unique identifier for the model. For more information, go here. |
backbonepredictions
object.
This JSON response is from Cloud Deployment.