This document provides an overview and code samples for using the Azure Kinect SDK and Azure Kinect Body Tracking SDK in C/C++. It explains how to install the SDKs, generate projects, capture and process sensor data, playback recordings, and track body poses. Code samples are provided for initializing devices, capturing images, transforming data between coordinate systems, and drawing body skeletons overlaid on images.

1. Azure Kinect DK C/C++開発概要(仮)

2. Self Introduction
杉浦 司 (Tsukasa Sugiura)
- Freelance Programmer
- Microsoft MVP for Windows Development
- Point Cloud Library Maintainer
- @UnaNancyOwen

3. Azure Kinect DK Sensor

4. Azure Kinect SDK Introduction
Azure Kinect Sensor SDK
(k4a / k4arecord)
Azure Kinect Body Tracking SDK
(k4abt)
- Color, Depth, Infrared, IMU, Point Cloud
- Open Source Library hosted on GitHub
(Depth Engine is Closed Source)
- Cross Platform (Windows, Linux)
- C API, C++ and C# Wrapper
- Body Index Map, Skeleton (26 Joints/Person)
- Deep Learning based Pose Estimation
- Closed Source Library
- Cross Platform (Windows, Linux)
- C API, (C++ Wrapper)

5. How to Install Azure Kinect SDK?
Azure Kinect Sensor SDK
- Install Pre-Built SDK using Installer
- Build and Install SDK from Source Code
Azure Kinect Body Tracking SDK
- Install SDK using Installer
- Install NVIDIA GPU Driver and Visual C++ 2015 Runtime
for ONNX Runtime (CUDA Backend)
Download Azure Kinect Body Tracking SDK | Microsoft Docs
https://docs.microsoft.com/en-us/azure/Kinect-dk/body-sdk-download
About Azure Kinect Sensor SDK | Microsoft Docs
https://docs.microsoft.com/en-us/azure/Kinect-dk/about-sensor-sdk

6. How to Generate Project with Azure Kinect SDK?
cmake_minimum_required( VERSION 3.6 )
project( Solution )
add_executable( Project main.cpp )
# Azure Kinect Sensor SDK (Official Support)
find_package( k4a REQUIRED )
find_package( k4arecord REQUIRED )
# Azure Kinect Body Tracking SDK (Un-Official Support)
set( CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}" )
find_package( k4abt REQUIRED )
if( k4a_FOUND AND k4arecord_FOUND AND k4abt_FOUND )
target_link_libraries( Project k4a::k4a )
target_link_libraries( Project k4a::k4arecord )
target_link_libraries( Project k4a::k4abt )
endif()
CMake Module for Azure Kinect Sensor SDK | GitHub Gists
https://gist.github.com/UnaNancyOwen/5ce9115ba8b71d12982e5aa9f99788f1#file-findk4abt-cmake
Include CMake and MS Build files in the MSI #370 | GitHub/Azure-Kinect-Sensor-SDK
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/issues/370

7. Azure Kinect Sensor SDK Overview
Capture Image
Playback
Device

8. Azure Kinect Sensor SDK Programming
// Azure Kinect Sensor SDK
#include <k4a/k4a.h>
#include <k4a/k4a.hpp> /* C++ Wrapper */
// Get Connected Devices
const int32_t device_count = k4a::device::get_installed_count();
if( device_count == 0 ){
throw k4a::error( "Failed to found device!" );
}
// Open Default Device
k4a::device device = k4a::device::open( K4A_DEVICE_DEFAULT );
// Start Cameras with Configuration
k4a_device_configuration_t configuration = K4A_DEVICE_CONFIG_INIT_DISABLE_ALL;
configuration.color_format = K4A_IMAGE_FORMAT_COLOR_BGRA32;
configuration.color_resolution = K4A_COLOR_RESOLUTION_1080P;
configuration.depth_mode = K4A_DEPTH_MODE_NFOV_UNBINNED;
configuration.wired_sync_mode = K4A_WIRED_SYNC_MODE_STANDALONE;
configuration.synchronized_images_only = true;
device.start_cameras( &configuration );

9. Azure Kinect Sensor SDK Programming
// Get Capture
k4a::capture capture;
const std::chrono::milliseconds timeout( K4A_WAIT_INFINITE );
const bool result = device.get_capture( &capture, timeout );
if( !result ){
break;
}
// Get Color Image
k4a::image color_image = capture.get_color_image();
cv::Mat color = k4a::get_mat( color_image );
// Get Depth Image
k4a::image depth_image = capture.get_depth_image();
cv::Mat depth = k4a::get_mat( depth_image );
- k4a::get_mat() is utility function that defined in util.h
Utility for Azure Kinect Sensor SDK | GitHub Gists
https://gist.github.com/UnaNancyOwen/9f16ce7ea4c2673fe08b4ce4804fc209#file-util-h

10. Azure Kinect Sensor SDK Programming
// Show Image
depth.convertTo( depth, CV_8U, -255.0 / 5000.0, 255.0 );
cv::imshow( "color", color );
cv::imshow( "depth", depth );
// Wait Key
const int32_t key = cv::waitKey( 30 );
if( key == 'q' ){
break;
}
// Close Device
device.close();
// Clear Handle
color_image.reset();
depth_image.reset();
capture.reset();

11. Azure Kinect Sensor SDK Programming
// Initialize Transformation
k4a::calibration calibration = device.get_calibration( configuration.depth_mode, configuration.color_resolution );
k4a::transformation transformation = k4a::transformation( calibration );
// Transform Color Image to Depth Camera
k4a::image transformed_color_image = transformation.color_image_to_depth_camera( depth_image, color_image );
cv::Mat transformed_color = k4a::get_mat( transformed_color_image );
// Transform Depth Image to Color Camera
k4a::image transformed_depth_image = transformation.depth_image_to_color_camera( depth_image );
cv::Mat transformed_depth = k4a::get_mat( transformed_depth_image );
// Transform Depth Image to Point Cloud
k4a::image xyz_image = transformation.depth_image_to_point_cloud( depth_image, K4A_CALIBRATION_TYPE_DEPTH );
//k4a::image xyz_image = transformation.depth_image_to_point_cloud( transformed_depth_image, K4A_CALIBRATION_TYPE_COLOR );
cv::Mat xyz = k4a::get_mat( xyz_image );
- Transform Image to Other Coordinate System
Color Image → Depth Camera
Depth Image → Color Camera
Depth Image → Point Cloud
Add overload functions that returns k4a::image in k4a::transformation #596 | GitHub/Azure-Kinect-Sensor-SDK
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/pull/596
// Close Transformation
transformation.destroy();

12. Azure Kinect Sensor SDK Programming
// Azure Kinect Sensor SDK
#include <k4arecord/playback.h>
#include <k4arecord/playback.hpp> /* C++ Wrapper from Azure Kinect Sensor SDK v1.2.0 */
// Open Playback File
k4a::playback playback = k4a::playback::open( "./path/to/file.mkv" );
// Get Capture
k4a::capture capture;
bool result = playback.get_next_capture( &capture );
if( !result ){
break;
}
// Close Playback
playback.close();
- NOTE: k4arecorder (Azure Kinect Sensor SDK v1.1.x) has a bug that first 2-frames are empty.
Move C++ wrapper for playback #493 | GitHub/Azure-Kinect-Sensor-SDK
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/pull/493

13. Azure Kinect Sensor SDK Programming
Azure Kinect Office Sample Recordings | Microsoft Download Center
https://www.microsoft.com/en-us/download/details.aspx?id=58385

14. Azure Kinect Body Tracking SDK Overview
Capture Queue
Playback
Device
Frame Body

15. Azure Kinect Body Tracking SDK Programming
// Azure Kinect Body Tracking SDK
#include <k4abt.h>
#include "k4abt.hpp" /* Un-Official C++ Wrapper */
//#include <k4abt.hpp> /* Official C++ Wrapper Provide from Next Release */
// Create Tracker
k4abt::tracker tracker = k4abt::tracker::create( calibration );
if( !tracker ){
throw k4a::error( "Failed to create tracker!" );
}
// Enqueue Capture
tracker.enqueue_capture( capture );
// Pop Tracker Result
k4abt::frame body_frame = tracker.pop_result();
- Official C++ wrapper is coming soon …
C++Wrapper for Azure Kinect Body Tracking SDK | GitHub Gists
https://gist.github.com/UnaNancyOwen/41b578f5d272aa6f22cf8ff6565fb71b#file-k4abt-hpp

16. Azure Kinect Body Tracking SDK Programming
// Get Body Index Map
k4a::image body_index_map_image = body_frame.get_body_index_map();
cv::Mat body_index_map = k4a::get_mat( body_index_map_image );
// Draw Body Index Map
cv::Mat body_index = cv::Mat::zeros( body_index_map.size(), CV_8UC3 );
body_index.forEach<cv::Vec3b>(
[&]( cv::Vec3b& pixel, const int32_t* position ){
const int32_t x = position[1], y = position[0];
const uint32_t index = body_index_map.at<uint8_t>( y, x );
if( index != K4ABT_BODY_INDEX_MAP_BACKGROUND ){
pixel = colors[index % colors.size()];
}
}
);
- Background Index is 255 (K4ABT_BODY_INDEX_MAP_BACKGROUND)

17. Azure Kinect Body Tracking SDK Programming
// Get Body Skeleton
std::vector<k4abt_body_t> bodies = body_frame.get_bodies();
// Get Image that used for Inference
k4a::capture body_capture = body_frame.get_capture();
k4a::image skeleton_image = body_capture.get_color_image();
cv::Mat skeleton = k4a::get_mat( skeleton_image );
// Draw Body Skeleton
for( const k4abt_body_t& body : bodies ){
for( const k4abt_joint_t& joint : body.skeleton.joints ){
k4a_float2_t position;
const bool result = calibration.convert_3d_to_2d( joint.position, K4A_CALIBRATION_TYPE_DEPTH, K4A_CALIBRATION_TYPE_COLOR, &position );
if( !result ){
continue;
}
const int32_t id = body.id;
const cv::Point point( static_cast<int32_t>( position.xy.x ), static_cast<int32_t>( position.xy.y ) );
cv::circle( skeleton, point, 5, colors[id % colors.size()], -1 );
}
}
- Get Image that used for Inference
- Convert Joint Position (3D) to 2D for Draw Circle on Image

18. Azure Kinect Body Tracking SDK Programming
// Show Image
cv::imshow( "body index", body_index );
cv::imshow( "skeleton", skeleton );
// Wait Key
const int32_t key = cv::waitKey( 30 );
if( key == 'q' ){
break;
}
// Close Tracker
tracker.destroy();
// Clear Handle
body_index_map_image.reset();
skeleton_image.reset();
body_capture.reset();
body_frame.reset();