Depth Processing

The depth module contains classes that hide some of all the repeated code associated with processing depth data. The main component is the DepthProcessing class, which is used to process continuously retrieved data from the kinect. The background model used in this class is obtained through running average via the method DepthProcessing.update_background_model()

Usage Example

If code already exists to retrieve the data extracting the bounding boxes proposals can be reduced to as little as the following:

# DepthProcessing instance
depth = DepthProcessing(IMAGE_SHAPE)

while True:
    # retrieve the depth information
    depth.current_frame = cam.get_depth_matrix()
    if first_run:
        # in first run moving average start from first frame
        depth.background_model = depth.current_frame.astype(depth.background_model.dtype)
        first_run = False

    # get depth background
    depth.update_background_model(depth.current_frame)

    # get depth foreground
    depth.extract_foreground_mask_from_run_avg(depth.current_frame)

    # apply opening to remove noise
    depth.foreground_mask = bg_models.apply_opening(depth.foreground_mask,
                                                    BG_OPEN_KSIZE,
                                                    cv2.MORPH_ELLIPSE)

    depth_proposal_bbox = depth.extract_proposal_bbox(depth.ACCUMULATOR)

DepthProcessing class

class depth_processing.DepthProcessing(image_shape=(640, 480))

Depth Processing Class

extract_foreground_mask_from_run_avg(current_frame)

Extract depth foreground mask from running average computed substracting current_frame from background model where the difference is above BG_MASK_THRESHOLD

Parameters:current_frame – current frame from which extract foreground Returns:binary mask with 1 for foreground and 0 for background Return type:np.int64

extract_proposal_bbox(method=0)

Compute bounding boxes for connected components from foreground masks that remain constant for AGG_DEPTH_MAX_E frames.

To keep track of the bounding boxes over time the function uses an aggregator depending on the method specified

Parameters:method –

method used to keep track of the bounding boxes history. Methods available are:

• ACCUMULATOR: to use an image accumulator for each pixel (fastest method). The bounding boxes are extracted from the pixels accumulated AGG_DEPTH_MAX_E times.
• RECT_MATCHING/RECT_MATCHING2: to keep track of the number of times a particular bounding box occurs over time (slower method but more accurate). Two bounding boxes in different frames are considered the same if their placement and dimension remain within a tolerance threshold.

Returns:list of bounding boxes in the form of (x,y, width, height) where (x,y) is the top left corner Return type:List Raise:NotImplementedError: if a method different from ACCUMULATOR or RECT_MATCHING or RECT_MATCHING2 is specified

update_background_model(current_frame, holes_frame=<Mock object>)

Update depth background by running average

Parameters:current_frame – current frame whereby update bg model Returns:background model Return type:np.float32

depth_processing.update_depth_detection_aggregator(aggregator, foreground_current)

Update aggregator with the provided foreground. Each pixel of the image has a value that keeps the number of times it has been seen as foreground.

Parameters:

• aggregator – an image of uint8
• foreground_current – mask of the current foreground

Returns:

updated accumulator