Kinect Support on Windows through OpenCV

Update 2011/08/03: Updated the post to work with the latest trunk revision (6281 at the time of writing) of OpenCV. The old post was moved into a subsection of this post.
Update 2011/06/11: I was informed that the patch provided in this blog post was applied against the the OpenCV code-base in revision 5415.
Update 2011/04/17: You may want to read my follow-up post on limiting the effect of data flickering.

OpenCV 2.3.1 rev 6281

Since revision 4640, OpenCV, OpenNI and PrimeSense KinectMod drivers have received major updates. Here’s a short write-up on how to get it going with the latest OpenCV developer revision. The following how-to uses 32bit drivers and OpenCV builds on a 64bit Windows 7 system.

  1. Make sure the Kinect is unplugged.

    I’ve seen the Kinect failing to work when this step was skipped, so make sure you unplug the Kinect.

  2. Uninstall any previous OpenNI and PrimeSense Sensor KinectMod drivers.
  3. Install OpenNI (Version 1.3.2.3 32bit)

    Download from http://www.openni.org/downloadfiles/opennimodules/openni-binaries/20-latest-unstable

  4. Install PrimeSense Sensor KinectMod (Version 5.0.3.4 32bit)

    https://github.com/avin2/SensorKinect/tree/unstable/Bin

  5. PlugIn Kinect now.

    Wait for the device drivers to be installed. In the device manager locate

    • PrimeSense/Kinect Audio
    • PrimeSense/Kinect Camera
    • PrimeSense/Kinect Motor

    and make sure that the devices are ready to use.

  6. Checkout OpenCV rev. 6281

    https://code.ros.org/svn/opencv/trunk/opencv/

  7. Patch OpenCVFindOpenNI.cmake

    This step is necessary if you own a 64bit OS Windows. In this case OpenNI and PrimeSense Sensor KinectMod drivers get installed to c:/program files (x86), which is not searched by OpenCVFindOpenNI.cmake. The following patch might not be necessary if you use the 64bit versions of OpenNI and PrimeSense Sensor KinectMod.

    Index: OpenCVFindOpenNI.cmake
    ===================================================================
    --- OpenCVFindOpenNI.cmake	(revision 6281)
    +++ OpenCVFindOpenNI.cmake	(working copy)
    @@ -20,7 +20,7 @@
     	
     if(NOT OPENNI_LIB_DIR OR OPENNI_LIB_DIR STREQUAL "")
         if(WIN32)
    -        set(OPENNI_LIB_DIR "c:/Program Files/OpenNI/Lib" CACHE PATH ${OPENNI_LIB_DESCR})
    +        set(OPENNI_LIB_DIR "c:/Program Files (x86)/OpenNI/Lib" CACHE PATH ${OPENNI_LIB_DESCR})
         elseif(UNIX OR APPLE)
             set(OPENNI_LIB_DIR "/usr/lib" CACHE PATH ${OPENNI_LIB_DESCR})
         endif()
    @@ -28,7 +28,7 @@
     	
     if(NOT OPENNI_INCLUDE_DIR OR OPENNI_INCLUDE_DIR STREQUAL "")
         if(WIN32)
    -        set(OPENNI_INCLUDE_DIR "c:/Program Files/OpenNI/Include" CACHE PATH ${OPENNI_INCLUDE_DESCR})
    +        set(OPENNI_INCLUDE_DIR "c:/Program Files (x86)/OpenNI/Include" CACHE PATH ${OPENNI_INCLUDE_DESCR})
         elseif(UNIX OR APPLE)
             set(OPENNI_INCLUDE_DIR "/usr/include/ni" CACHE PATH ${OPENNI_INCLUDE_DESCR})
         endif()
    @@ -36,7 +36,7 @@
     	
     if(NOT OPENNI_PRIME_SENSOR_MODULE_BIN_DIR OR OPENNI_PRIME_SENSOR_MODULE_BIN_DIR STREQUAL "")
         if(WIN32)
    -        set(OPENNI_PRIME_SENSOR_MODULE_BIN_DIR "c:/Program Files/Prime Sense/Sensor/Bin" CACHE PATH ${OPENNI_PRIME_SENSOR_MODULE_BIN_DESCR})
    +        set(OPENNI_PRIME_SENSOR_MODULE_BIN_DIR "c:/Program Files (x86)/PrimeSense/SensorKinect/Bin" CACHE PATH ${OPENNI_PRIME_SENSOR_MODULE_BIN_DESCR})
         elseif(UNIX OR APPLE)
             set(OPENNI_PRIME_SENSOR_MODULE_BIN_DIR "/usr/lib" CACHE PATH ${OPENNI_PRIME_SENSOR_MODULE_BIN_DESCR})
         endif()
    
    
  8. Start CMake with OpenCV directory

    Select your compiler of choice and set the address model is 32bit.

    Make sure to enable WITH_OPENNI
    Make sure to enable BUILD_EXAMPLES
    Click Configure and then Generate. You might need to click Configure several times.

  9. Open Solution OpenCV.sln

    Build: Release > Build all > INSTALL
    Build: Debug > Build All > INSTALL

  10. Verify

    Run the kinect_maps example to test Kinect support and get a feeling for how to use the API.

OpenCV 2.2.0 rev 4640

This weekend I’ve been busy integrating the Kinect sensor into our vision framework at work, which requires a few steps to get it up and running. I decided to put them online, so anyone who cares can benefit from it.

First, I decided to go with the offical PrimseSense drivers that integrate into the OpenNI framwork. Second, the current OpenCV trunk shows efforts to integrate the drivers into its video capture framework. Since I don’t want to reinvent the wheel, I will stick with OpenCV for the remainder of this post.

Install Prerequisites

You need to install the OpenNI Framework and PrimeSense drivers. At the time of writing the current versions are

Install both in the given order. Note, if you are on Win64, make sure both get installed to c:\program files instead of c:\program files (x86), or you will need to patch the OpenCV CMake files.

Checkout OpenCV

Checkout the HEAD revision, or stick with the revision I’ve used in my tests 4640, of the trunk using Subversion.

Patch OpenCV

Note: Starting with revision 5415 this patch has been incorporated into the OpenCV code-base. So in case you are sticking with revision 5415 or later this patch is no longer necessary.

I needed tweak OpenCV slightly to fit my needs:

  • Fix temporaray file generation on Windows.
  • Align 3D point coordinates and color information.

Both tweaks are summed up in the following patch

--- cap_openni.cpp	(revision 4640)
+++ cap_openni.cpp	(working copy)
@@ -183,8 +183,7 @@
         // Write configuration to the temporary file.
         // This is a hack, because there is a bug in RunXmlScript().
         // TODO: remove hack when bug in RunXmlScript() will be fixed.
-        char xmlFilename[100];
-        tmpnam( xmlFilename );
+        char xmlFilename[] = "kinect_xml_tmp.xml";
         std::ofstream outfile( xmlFilename );
         outfile.write( XMLConfig.c_str(), XMLConfig.length() );
         outfile.close();
@@ -213,6 +212,8 @@
         CV_Assert( depthGenerator.SetMapOutputMode( depthOutputMode ) == XN_STATUS_OK );
         CV_Assert( imageGenerator.SetMapOutputMode( imageOutputMode ) == XN_STATUS_OK );
 
+        depthGenerator.GetAlternativeViewPointCap().SetViewPoint(imageGenerator); // align xyz with rgb.
+
         //  Start generating data.
         status = context.StartGeneratingAll();
         if( status != XN_STATUS_OK )

Compile and Run

Setting the correct definitions in CMake is crucial. Make sure you enable

  • BUILD_EXAMPLES
  • WITH_OPENNI

Build OpenCV and the examples that come along with OpenCV. Run the kinect_maps example to test Kinect support and get a feeling for how to use the API.

Note, the following screenshot is showing parts of our vision system at work, not the OpenCV example.

Advertisements

28 thoughts on “Kinect Support on Windows through OpenCV

  1. Pingback: Kinect Support on Windows through OpenCV

  2. Hello,

    I follow the steps one by one just the way you post it but I am having some issues when I try to compile OpenCV, this is the only error that I get, any help would be really appreciated:

    Creando biblioteca ..\..\lib\Release\opencv_highgui220.lib y objeto ..\..\lib\Release\opencv_highgui220.exp

    Vinculando…
    Creando biblioteca ..\..\lib\Release\opencv_highgui220.lib y objeto ..\..\lib\Release\opencv_highgui220.exp
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnOSMemSet sin resolver al que se hace referencia en la función “public: __cdecl xn::OutputMetaData::OutputMetaData(unsigned char const * *)” (??0OutputMetaData@xn@@QEAA@PEAPEBE@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnOSFreeAligned sin resolver al que se hace referencia en la función “public: void __cdecl xn::OutputMetaData::Free(void)” (?Free@OutputMetaData@xn@@QEAAXXZ)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnGetIntProperty sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::ProductionNode::GetIntProperty(char const *,unsigned __int64 &)const ” (?GetIntProperty@ProductionNode@xn@@QEBAIPEBDAEA_K@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnGetRealProperty sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::ProductionNode::GetRealProperty(char const *,double &)const ” (?GetRealProperty@ProductionNode@xn@@QEBAIPEBDAEAN@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnSetViewPoint sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::AlternativeViewPointCapability::SetViewPoint(class xn::ProductionNode &)” (?SetViewPoint@AlternativeViewPointCapability@xn@@QEAAIAEAVProductionNode@2@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnSetMapOutputMode sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::MapGenerator::SetMapOutputMode(struct XnMapOutputMode const &)” (?SetMapOutputMode@MapGenerator@xn@@QEAAIAEBUXnMapOutputMode@@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnGetDepthMetaData sin resolver al que se hace referencia en la función “public: void __cdecl xn::DepthGenerator::GetMetaData(class xn::DepthMetaData &)const ” (?GetMetaData@DepthGenerator@xn@@QEBAXAEAVDepthMetaData@2@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnGetDeviceMaxDepth sin resolver al que se hace referencia en la función “public: unsigned short __cdecl xn::DepthGenerator::GetDeviceMaxDepth(void)const ” (?GetDeviceMaxDepth@DepthGenerator@xn@@QEBAGXZ)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnConvertProjectiveToRealWorld sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::DepthGenerator::ConvertProjectiveToRealWorld(unsigned int,struct XnVector3D const * const,struct XnVector3D * const)const ” (?ConvertProjectiveToRealWorld@DepthGenerator@xn@@QEBAIIQEBUXnVector3D@@QEAU3@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnGetImageMetaData sin resolver al que se hace referencia en la función “public: void __cdecl xn::ImageGenerator::GetMetaData(class xn::ImageMetaData &)const ” (?GetMetaData@ImageGenerator@xn@@QEBAXAEAVImageMetaData@2@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnInit sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::Context::Init(void)” (?Init@Context@xn@@QEAAIXZ)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnContextRunXmlScriptFromFile sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::Context::RunXmlScriptFromFile(char const *,class xn::EnumerationErrors *)” (?RunXmlScriptFromFile@Context@xn@@QEAAIPEBDPEAVEnumerationErrors@2@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnShutdown sin resolver al que se hace referencia en la función “public: void __cdecl xn::Context::Shutdown(void)” (?Shutdown@Context@xn@@QEAAXXZ)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnStartGeneratingAll sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::Context::StartGeneratingAll(void)” (?StartGeneratingAll@Context@xn@@QEAAIXZ)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnStopGeneratingAll sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::Context::StopGeneratingAll(void)” (?StopGeneratingAll@Context@xn@@QEAAIXZ)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnWaitAndUpdateAll sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::Context::WaitAndUpdateAll(void)” (?WaitAndUpdateAll@Context@xn@@QEAAIXZ)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnCreateDepthGenerator sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::DepthGenerator::Create(class xn::Context &,class xn::Query *,class xn::EnumerationErrors *)” (?Create@DepthGenerator@xn@@QEAAIAEAVContext@2@PEAVQuery@2@PEAVEnumerationErrors@2@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnCreateImageGenerator sin resolver al que se hace referencia en la función “public: unsigned int __cdecl xn::ImageGenerator::Create(class xn::Context &,class xn::Query *,class xn::EnumerationErrors *)” (?Create@ImageGenerator@xn@@QEAAIAEAVContext@2@PEAVQuery@2@PEAVEnumerationErrors@2@@Z)
    cap_openni.obj : error LNK2019: símbolo externo __imp_xnGetStatusString sin resolver al que se hace referencia en la función “public: __cdecl CvCapture_OpenNI::CvCapture_OpenNI(void)” (??0CvCapture_OpenNI@@QEAA@XZ)
    ..\..\bin\Release\opencv_highgui220.dll : fatal error LNK1120: 19 externos sin resolver

    Thanks in advance.

    • Manuel,

      it seems like the linker cannot pick up the required OpenNI libraries. Where did you install OpenNI to? OpenCV searches for that libraries in c:\Program Files. In case your installation went to different directory, check the CMakeLists.txt in OpenCVs main directory. Search for if(WITH_OPENNI) and patch directories in that section as necessary. Also make sure that CMake finds the required .lib files, such as OPENNI_LIBRARY.

      Best regards,
      Christoph

  3. Hello,

    I followed your instruction but when I tried to run the kinect_maps sample it always turns out to be:

    Kinnect openning …
    Done.
    Can not open a capture object.

    It seems like the program cannot finnd the camera. But I am quite sure that the kinect camera has been connected to the computer because I can successfully run some openni samples. I was wondering would you please give me some hints about why this problem happens or how to fiix it.

    By the way, I am using the windows 7 64 bit OS and visual studio 2010 express version.

    Best wishes,
    Jim

    • Hi Jim,

      do you have multiple cameras connected? Maybe opencv code assumes some device index 0? Did you install the drivers (OpenNI and Primesense) as mentioned in my post? We have successfully tried the code on Windows7 (64bit/32bit), XP(32bit). We are compiling for 32bit.

      Best regards,
      Christoph

  4. Hi Christoph,
    What should I have to do if I just get the depth map and not the rgb
    image? I get an alternative view for the depth but the rgb image is just
    black. If I diplay the images separately I will get both of them.

    It seems to that OpenCV can not merge the two images by depthGenerator.GetAlternativeViewPointCap().SetViewPoint(imageGenerator);

    Best regards

  5. Hi Christoph,
    thank you for your reply =)
    Think it helps a lot.

    So I have to use a point cloud map … Do I have to use OpenGL respectively DivX for displaying the point cloud, too? Or is it possible to illustrate the output just with OpenCV?
    Regards,
    Klaus

    • Klaus,

      you’re welcome. To illustrate the point cloud you cannot use OpenCV, at least I’m not aware of any 3D support in OpenCV. You could use OpenGL / DirectX directly or get used to some higher level abstraction framework such as VTK or OSG.

      Best regards,
      Christoph

  6. Hi, I have done step-by-step everything like discribed above. When trying to build the Solution in Visual Studio 2010 (Win 7 Pro x64) i get 19 Errors. Here are some examples:

    Error 77 error LNK2019: unresolved external symbol __imp_xnWaitAndUpdateAll referenced in function “public: unsigned int __cdecl xn::Context::WaitAndUpdateAll(void)” (?WaitAndUpdateAll@Context@xn@@QEAAIXZ) C:\Users\EliteBook\Documents\OpenCV\build\modules\highgui\cap_openni.obj opencv_highgui

    Error 64 error LNK2019: unresolved external symbol __imp_xnSetMapOutputMode referenced in function “public: unsigned int __cdecl xn::MapGenerator::SetMapOutputMode(struct XnMapOutputMode const &)” (?SetMapOutputMode@MapGenerator@xn@@QEAAIAEBUXnMapOutputMode@@@Z) C:\Users\EliteBook\Documents\OpenCV\build\modules\highgui\cap_openni.obj opencv_highgui

    Error 65 error LNK2019: unresolved external symbol __imp_xnInit referenced in function “public: unsigned int __cdecl xn::Context::Init(void)” (?Init@Context@xn@@QEAAIXZ) C:\Users\EliteBook\Documents\OpenCV\build\modules\highgui\cap_openni.obj opencv_highgui

    What am I missing? I think is a reference but to what???

    • Alex,

      I think the linker is missing the reference to OpenNI. Make sure that CMake is able to find the correct lib file to link. Especially make sure that WITH_OPENNI is set. To verify the directories and files gather by CMAKE have a look at the following variables: OPENNI_INCLUDE_DIR, OPENNI_LIB_DIR and OPENNI_PRIME_SENSOR_MODULE_BIN_DIR. In my case they refer to c:/Program Files/OpenNI/Include, c:/Program Files/OpenNI/Lib and c:/Program Files/Prime Sense/Sensor/Bin respectively.

      Best regards,
      Christoph

      • Hi,

        When I try to run the openCV “kinects_map” example, it showed some errors like this:

        ‘kinect_maps.exe’: Loaded ‘C:\OpenCV2.2\build\bin\Debug\kinect_maps.exe’, Symbols loaded.
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\ntdll.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\kernel32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\KernelBase.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\OpenCV2.2\build\bin\Debug\opencv_core229d.dll’, Symbols loaded.
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\msvcp100d.dll’, Symbols loaded.
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\msvcr100d.dll’, Symbols loaded.
        ‘kinect_maps.exe’: Loaded ‘C:\OpenCV2.2\build\bin\Debug\opencv_highgui229d.dll’, Symbols loaded.
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\user32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\gdi32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\lpk.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\usp10.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\msvcrt.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\advapi32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\sechost.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\rpcrt4.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\sspicli.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\cryptbase.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\ole32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\OpenCV2.2\build\bin\Debug\opencv_imgproc229d.dll’, Symbols loaded.
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\winsxs\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.7601.17514_none_ec83dffa859149af\comctl32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\avifil32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\winmm.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\msacm32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\msvfw32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\shell32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\shlwapi.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\avicap32.dll’, Symbols loaded (source information stripped).
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\version.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\OpenNI\Bin\OpenNI.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\ws2_32.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\nsi.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\setupapi.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\cfgmgr32.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\oleaut32.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\devobj.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\winsxs\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4974_none_50940634bcb759cb\msvcr90.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\olepro32.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\imm32.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\msctf.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\SysWOW64\uxtheme.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\OpenNI\Bin\nimRecorder.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\OpenNI\Bin\nimMockNodes.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\OpenNI\Bin\nimCodecs.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Bin\XnVFeatures.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Windows\winsxs\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4974_none_50940634bcb759cb\msvcp90.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Bin\XnVFeatures_1_3_1.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Bin\XnVHandGenerator.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Bin\XnVHandGenerator_1_3_1.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnDeviceFile.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnCore.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnDDK.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnFormats.dll’, Cannot find or open the PDB file
        ‘kinect_maps.exe’: Loaded ‘C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnDeviceSensorV2KM.dll’, Cannot find or open the PDB file
        First-chance exception at 0x6d10e3e5 in kinect_maps.exe: 0xC0000005: Access violation reading location 0x00000008.
        Unhandled exception at 0x77c115ee (ntdll.dll) in kinect_maps.exe: 0xC0000005: Access violation reading location 0x00000008.
        First-chance exception at 0x77c0017e (ntdll.dll) in kinect_maps.exe: 0x00000000: The operation completed successfully.
        Unhandled exception at 0x77c115ee (ntdll.dll) in kinect_maps.exe: 0x00000000: The operation completed successfully.
        First-chance exception at 0x77c0017e (ntdll.dll) in kinect_maps.exe: 0x00000000: The operation completed successfully.
        Unhandled exception at 0x77c115ee (ntdll.dll) in kinect_maps.exe: 0x00000000: The operation completed successfully.
        The program ‘[3968] kinect_maps.exe: Native’ has exited with code 0 (0x0).

        I dont know why, please help me.

        Best regards,
        Huu Loc.

          • Hi,

            I try another way to use openCV to retrieve Kinect data as the code below:

            #include
            #include “XnCppWrapper.h”
            #include
            #include
            #include

            void main()
            {

            XnStatus nRetVal = XN_STATUS_OK;

            xn::Context context;
            nRetVal = context.Init();
            // TODO: check error code

            // Create a depth generator
            xn::DepthGenerator depth;
            nRetVal = depth.Create(context);
            // TODO: check error code

            // Set it to VGA maps at 30 FPS
            XnMapOutputMode mapMode;
            mapMode.nXRes = XN_VGA_X_RES;
            mapMode.nYRes = XN_VGA_Y_RES;
            mapMode.nFPS = 30;
            nRetVal = depth.SetMapOutputMode(mapMode);
            // TODO: check error code

            // Start generating
            nRetVal = context.StartGeneratingAll();
            // TODO: check error code

            // Calculate index of middle pixel
            XnUInt32 nMiddleIndex =
            XN_VGA_X_RES * XN_VGA_Y_RES/2 + // start of middle line
            XN_VGA_X_RES/2; // middle of this line

            IplImage *img = cvCreateImage(
            cvSize(
            XN_VGA_X_RES,
            XN_VGA_Y_RES
            ),IPL_DEPTH_16U,1
            );

            cvNamedWindow(“test”, 1 );
            CvMat* depthMetersMat = cvCreateMat(480, 640, CV_16UC1 );
            CvMat* imageMetersMat = cvCreateMat(480, 640, CV_16UC1 );
            IplImage *kinectDepthImage = cvCreateImage( cvSize(640,480),16,1);

            const XnDepthPixel* pDepthMap = depth.GetDepthMap();
            //CvCapture_openNI

            while (TRUE)
            {
            if( cvWaitKey( 100 ) == 27 ) break;
            // Update to next frame
            nRetVal = context.WaitOneUpdateAll(depth);
            // TODO: check error code
            const XnDepthPixel* pDepthMap = depth.GetDepthMap();
            //img->imageData = (unsigned char*)depth.GetDepthMap();
            /*for (int i = 0; i imageData + i * img->widthStep);
            for( int j = 0; j < XN_VGA_X_RES; j++ )
            {
            ptr[j] = pDepthMap[i * XN_VGA_X_RES + j];
            }
            cvShowImage("test", img );
            }*/
            for (int y=0; y<XN_VGA_Y_RES; y++){
            for(int x=0;xdata.s[y * XN_VGA_X_RES + x ] = 10 * pDepthMap[y * XN_VGA_X_RES + x];
            }
            }

            cvGetImage(depthMetersMat, kinectDepthImage);
            cvShowImage(“test”, kinectDepthImage );

            }
            cvDestroyWindow( “test” );
            // Clean up
            context.Shutdown();
            exit(0);

            }

            But when the program run, it showed the dialog box which is “the program cannot start because opencv_core299d.dll is missing from your computer. Try reinstalling the program to fix the problem.”.

            I have used Cmake to rebuild the latest openCV source code.

            Best regards,
            Huu Loc.

          • Hi,

            make sure you have the debug variants of OpenCV installed. By default an installed OpenCV version only ships with release builds. Either compile OpenCV in debug mode or, or compile your program in release mode.

            Best regards,
            Christoph

          • Hi,

            Sorry, I have fixed the dll problems. Can you show me the source code to retrieve RGB and depth data on Kinects. It’s very helpful too. Thanks.

            Best regards,
            Huu Loc.

          • Hi Chris,

            Although I could use openCV to retrieve the image and depth on Kinect, but it seemed to be very slow in comparing with 30fps. Did you feel it? I dont know the reason. When I used Brekel Kinect application, it’s strongly faster.

          • Are you fetching the depth image and the rgb image, or the point cloud directly via CV_CAP_OPENNI_POINT_CLOUD_MAP? In the latter case, the calibration is performed via OpenNI API which is slower than rolling your own calibration.

  7. Pingback: Preventing Kinect Flickering « Christoph Heindl

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s