getItem
The getItem function of PDS is designed to detect the pose of a given item. To do this, PDS requires a template of the item to be detected. A template can be understood as a reference model of the object, consisting of point cloud data from one side of the object, representing the shape of the object as seen from different angles. This template serves as a basis for comparison during the pose detection process.
Please note that these templates typically allow for an angular deviation of up to 10° - 15° from the normal vector.
To detect a custom item, please contact your local ifm support or support.robotics@ifm.com for detailed instructions.
Input
Command
The input to the getItem command should be provided in JSON.
Below is an example, assuming an instantiated PDS app app0:
{
"applications": {
"instances": {
"app0": {
"configuration": {
"customization": {
"command": "getItem",
"getItem": {
"depthHint": -1,
"itemIndex": 0,
"itemOrder": "zAscending"
}
}
}
}
}
}
}
depthHint
Name |
Description |
|---|---|
|
Estimated distance between the item and the calibrated coordinate system center in meters. Set to <=0 for automatic detection. |
itemIndex
The ifm development team developed and tested this functionality on pre-configured trolley types. To detect other types of items, contact ifm support.
Name |
Description |
|---|---|
|
Type of the item to be detected. |
itemOrder
Name |
Description |
|---|---|
|
When multiple items are detected, the order of the detected items can be set based on the following properties: |
Output
The output of a getItem command is formatted in JSON.
An example JSON result, where the position of one item was identified, is shown below:
"getItem": {
"depthEstimationVoi": {
"xMax": 0,
"xMin": 0,
"yMax": 0,
"yMin": 0,
"zMax": 0,
"zMin": 0
},
"item": [
{
"angles": {
"rotX": 0,
"rotY": 0,
"rotZ": 0.05009809136390686
},
"position": {
"x": 1.5020731687545776,
"y": 0.010188048705458641,
"z": 0.1468670815229416
},
"score": 0.9384191036224365
}
],
"projectionVoi": {
"xMax": 0,
"xMin": 0,
"yMax": 0,
"yMin": 0,
"zMax": 0,
"zMin": 0
}
}
item
This component of the JSON result lists out all the detected items (up to 10). For each item, the following information is provided:
Name |
Description |
|---|---|
|
Rotations components |
|
Cartesian coordinates |
|
Detection score of the item [0..1]. Note that the score does not correspond to how well the item is detected but rather how close it is to the reference item template. |
depthEstimationVoi and projectionVoi
The depthEstimationVoi and projectionVoi components provide the volumes that were used internally to detect the item:
Name |
Description |
|---|---|
|
Volume used in the algorithm to approximate the position of the front face of the item. |
|
Area where the precise estimation of the position and rotation of the item was done. The pixels outside this area are discarded. |
Example
To initialize and configure the PDS application to execute getItem command, please see the code example below.
#!/usr/bin/env python3
###########################################
###2023-present ifm electronic, gmbh
###SPDX-License-Identifier: Apache-2.0
###########################################
"""
Setup: * Camera: O3R222, 3D on port2
* orientation: camera horizontally (FAKRA cable to the left)
* Item: item in FoV @ 1.5m distance
"""
import json
import logging
import time
import numpy as np
from ifm3dpy.device import O3R, Error
from ifm3dpy.framegrabber import FrameGrabber, buffer_id
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
# Device specific configuration
IP = "192.168.0.69"
CAMERA_PORT = "port2"
APP_PORT = "app0"
o3r = O3R(IP)
############################################
# Setup the application
############################################
try:
o3r.reset("/applications/instances")
except Error as e:
logging.error(f"Reset failed: {e}")
# Set the correct extrinsic calibration of the camera.
calibration = {
"transX": 0.0,
"transY": 0.0,
"transZ": 0.2,
"rotX": 0.0,
"rotY": 1.57,
"rotZ": -1.57,
}
logger.info(f"Setting the extrinsic calibration for camera in {CAMERA_PORT}")
o3r.set({"ports": {CAMERA_PORT: {"processing": {"extrinsicHeadToUser": calibration}}}})
# Create the application instance and set to IDLE (ready to be triggered)
logger.info(f"Create a PDS instance using camera in {CAMERA_PORT}")
o3r.set(
{
"applications": {
"instances": {
APP_PORT: {"class": "pds", "ports": [CAMERA_PORT], "state": "IDLE"}
}
}
}
)
############################################
# Setup the framegrabber to receive frames
# when the application is triggered.
############################################
fg = FrameGrabber(o3r, o3r.port(APP_PORT).pcic_port)
fg.start([buffer_id.O3R_RESULT_JSON])
# Define a callback function to be executed every time a frame is received
def item_callback(frame):
"""Callback function executed every time a frame is received.
The data is decoded and the result printed.
:param frame: A frame containing the data for all the buffer ids
requested in the start function of the framegrabber.
"""
if frame.has_buffer(buffer_id.O3R_RESULT_JSON):
json_chunk = frame.get_buffer(buffer_id.O3R_RESULT_JSON)
json_array = np.frombuffer(json_chunk[0], dtype=np.uint8)
json_array = json_array.tobytes()
parsed_json_array = json.loads(json_array.decode())
logger.info(f"Detected item: {parsed_json_array['getItem']['item']}")
fg.on_new_frame(item_callback)
############################################
# Trigger the getItem command
############################################
time.sleep(2) # Grace period after the framegrabber starts
GET_ITEM_PARAMETERS = {
"depthHint": -1, # Estimated position of the item (-1 for automatic detection)
"itemIndex": 0, # Type of item
}
logger.info("Triggering the getItem command")
o3r.set(
{
"applications": {
"instances": {
APP_PORT: {
"configuration": {
"customization": {
"command": "getItem",
"getItem": GET_ITEM_PARAMETERS,
}
}
}
}
}
}
)
# Allow time for the callback to execute before exiting
time.sleep(3)
fg.stop()