Jump to content

  • start
  • about
  • forum
  • videos
  • downloads
  • faq

  • Photo

    PCL, OpenNI and Camboard Nano


    • Please log in to reply
    61 replies to this topic

    #21 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 31 July 2012 - 09:57 AM

    Hey Renaud,

    thx a lot!
    I downloaded the file and attached it to my post. I hope that's OK for you :)

    Best regards,
    Alex

    Attached File  CamboardPCLSample.zip   18.25KB   60 downloads
    Press [Ctrl]+[Alt]+[Del] to continue...

    #22 Rouno

    Rouno

      Member

    • Members
    • PipPip
    • 21 posts

    Posted 31 July 2012 - 10:14 AM

    Hi Alex,

    No problem for me :)
    You're welcome

    Regards,
    Renaud

    #23 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 31 July 2012 - 11:26 AM

    Hey everybody,

    I tried out the Kinect Fusion example with the Nano grabbed by OpenNI and it works :D



    So I decided to upload the OpenNI stuff I already implemented.
    Attached File  NiPMD - PCL.zip   17.1KB   48 downloads

    I know that there are still bugs in the code and it's not my nicest piece of code, but it works :)
    If you have any improvements, feel free to post.

    (Btw. if you click on "More reply options" you should be able to attach files to your post. This is not
    limited to images, you can also upload zip files for example)

    Best regards,
    Alex
    Press [Ctrl]+[Alt]+[Del] to continue...

    #24 mrtm3050

    mrtm3050

      Member

    • Members
    • PipPip
    • 17 posts

    Posted 31 July 2012 - 01:43 PM

    Awesome guys, I'll test it out.

    #25 mrtm3050

    mrtm3050

      Member

    • Members
    • PipPip
    • 17 posts

    Posted 31 July 2012 - 02:04 PM

    Hey Alex, do you have the interface code you used to grab the images and pipe it through to KinFu? It's okay otherwise, I'm sure I can figure it out. Just thought it'd save some time. In looking at this though, I assume PmdHelper is the class I use to grab everything?

    #26 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 31 July 2012 - 02:30 PM

    Hey mrtm3050,

    yes the PMDHelper is the class responsible for all the grabbing. I know that the structure isn't that
    nice, but the original code was something I made during my lunch breaks :D
    The code can be used as an OpenNI module for PCL. I just compiled it and inserted the line into the modules.xml file.

    Best regards,
    Alex
    Press [Ctrl]+[Alt]+[Del] to continue...

    #27 mrtm3050

    mrtm3050

      Member

    • Members
    • PipPip
    • 17 posts

    Posted 31 July 2012 - 03:26 PM

    Got it. I added:

    <Module path="C:\Programmierung\OpenNI\Samples\Bin\Debug\NiPMD.dll" />
    

    to my modules.xml file.

    What about the openni_capture.h and capture.cpp files for KinFu? In PmdHelper, I assume I call the EnumerateProductionTrees(...) and Create(...) as is done in capture.cpp? Followed by Init() and StartGenerating()?

    #28 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 31 July 2012 - 03:42 PM

    I changed nothing from the KinFu code. I just checked out the svn version of the PCL and build it.
    After the build was finished I had the kinfu_app_release.exe file in the bin folder.
    I assume that it just uses the first OpenNI Kinect that it finds. To be honest I never took a look
    into the kinfu_app source code ^^

    I just know that it worked out of the box :D

    Best regards,
    Alex
    Press [Ctrl]+[Alt]+[Del] to continue...

    #29 mrtm3050

    mrtm3050

      Member

    • Members
    • PipPip
    • 17 posts

    Posted 31 July 2012 - 04:27 PM

    Ah, I see. Well, then I assume it should work if I open up NiViewer or any other program that expects a Kinect. I must be missing a step. KinFu and NiViewer is saying that it can't find the device. Any thoughts?

    In the capture.cpp file of KinFu, it appears to crash on the *** line in the code segment below (I couldn't bold it) when I debug:

    pcl::gpu::CaptureOpenNI::open (int device)
    {
    impl_.reset ( new Impl () );
    XnMapOutputMode mode;
    mode.nXRes = XN_VGA_X_RES;
    mode.nYRes = XN_VGA_Y_RES;
    mode.nFPS = 30;
    XnStatus rc;
    rc = impl_->context.Init ();
    if (rc != XN_STATUS_OK)
    {
    	 sprintf (impl_->strError, "Init failed: %s\n", xnGetStatusString (rc));
    	 REPORT_ERROR (impl_->strError);
    }
    xn::NodeInfoList devicesList;
    rc = impl_->context.EnumerateProductionTrees ( XN_NODE_TYPE_DEVICE, NULL, devicesList, 0 );
    if (rc != XN_STATUS_OK)
    {
    	 sprintf (impl_->strError, "Init failed: %s\n", xnGetStatusString (rc));
    	 REPORT_ERROR (impl_->strError);
    }
    xn::NodeInfoList::Iterator it = devicesList.Begin ();
    for (int i = 0; i < device; ++i)
    	 it++;
    NodeInfo node = *it;
    rc = impl_->context.CreateProductionTree ( node, impl_->node );
    if (rc != XN_STATUS_OK)
    {
    	 sprintf (impl_->strError, "Init failed: %s\n", xnGetStatusString (rc));
    	 REPORT_ERROR (impl_->strError);
    }
    XnLicense license;
    const char* vendor = "PrimeSense";
    const char* key = "0KOIk2JeIBYClPWVnMoRKn5cdY4=";
    sprintf (license.strKey, key);
    sprintf (license.strVendor, vendor);
    rc = impl_->context.AddLicense (license);
    if (rc != XN_STATUS_OK)
    {
    	 sprintf (impl_->strError, "licence failed: %s\n", xnGetStatusString (rc));
    	 REPORT_ERROR (impl_->strError);
    }
    ************************************************************
    rc = impl_->depth.Create (impl_->context); // Crashes here
    if (rc != XN_STATUS_OK)
    {
    	 sprintf (impl_->strError, "Depth generator failed: %s\n", xnGetStatusString (rc));
    	 REPORT_ERROR (impl_->strError);
    }
    ************************************************************
    ....
    }
    


    #30 mrtm3050

    mrtm3050

      Member

    • Members
    • PipPip
    • 17 posts

    Posted 31 July 2012 - 08:24 PM

    Alex, would you mind posting your modules.xml file? Also, I had trouble getting PMD integrated in without putting the following two files in the Debug/Release directory of the exe files:

    camboradnano.W32.pap
    camboradnanoproc.W32.ppp

    #31 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 01 August 2012 - 08:45 AM

    Hey mrtm3050,

    I forgot to mention that ... I have the pmdaccess2.dll and both plugin files in the same folder as
    my module dll.

    My modules.xml looks like this :

    <Modules>
        <Module path="E:\projects\OpenNI\bin\nimRecorder.dll" />
        <Module path="E:\projects\OpenNI\bin\nimMockNodes.dll" />
        <Module path="E:\projects\OpenNI\bin\nimCodecs.dll" />
    <Module path="E:\projects\OpenNI\Samples\Bin\Release\NiPMD.dll" />
    <!--<Module path="E:\projects\OpenNI\Samples\Bin\Debug\NiPMD.dll" />-->
    <!--<Module path="E:\projects\PMDSampleModule\Release\PMDSampleModule.dll" />-->
        <Module path="E:\projects\NITE\Features_1_3_0\Bin\XnVFeatures_1_3_0.dll" configDir="E:\projects\NITE\Features_1_3_0\Data" />
        <Module path="E:\projects\NITE\Features_1_3_1\Bin\XnVFeatures_1_3_1.dll" configDir="E:\projects\NITE\Features_1_3_1\Data" />
        <Module path="E:\projects\NITE\Features_1_4_1\Bin\XnVFeatures_1_4_1.dll" configDir="E:\projects\NITE\Features_1_4_1\Data" />
        <Module path="E:\projects\NITE\Features_1_4_2\Bin\XnVFeatures_1_4_2.dll" configDir="E:\projects\NITE\Features_1_4_2\Data" />
        <Module path="E:\projects\NITE\Features_1_5_2\Bin\XnVFeatures_1_5_2.dll" configDir="E:\projects\NITE\Features_1_5_2\Data" />
        <Module path="E:\projects\NITE\Hands_1_3_0\Bin\XnVHandGenerator_1_3_0.dll" configDir="E:\projects\NITE\Hands_1_3_0\Data" />
        <Module path="E:\projects\NITE\Hands_1_3_1\Bin\XnVHandGenerator_1_3_1.dll" configDir="E:\projects\NITE\Hands_1_3_1\Data" />
        <Module path="E:\projects\NITE\Hands_1_4_1\Bin\XnVHandGenerator_1_4_1.dll" configDir="E:\projects\NITE\Hands_1_4_1\Data" />
        <Module path="E:\projects\NITE\Hands_1_4_2\Bin\XnVHandGenerator_1_4_2.dll" configDir="E:\projects\NITE\Hands_1_4_2\Data" />
        <Module path="E:\projects\NITE\Hands_1_5_2\Bin\XnVHandGenerator_1_5_2.dll" configDir="E:\projects\NITE\Hands_1_5_2\Data" />
    </Modules>
    

    When I execute the NiViewer.exe example from OpenNI it opens up my module (NiPMD.dll) and displays the CamBoard nano data.
    I assume that the PCL OpenNI examples also use this modules.xml file so I could just execute the kinfu app.

    Does the sample code work for you when you execute the OpenNI samples?

    Best regards,
    Alex
    Press [Ctrl]+[Alt]+[Del] to continue...

    #32 mrtm3050

    mrtm3050

      Member

    • Members
    • PipPip
    • 17 posts

    Posted 02 August 2012 - 04:00 PM

    Hrm, here's mine:

    <Modules>
        <Module path="C:\Program Files (x86)\OpenNI\Bin\nimRecorder.dll" />
        <Module path="C:\Program Files (x86)\OpenNI\Bin\nimMockNodes.dll" />
        <Module path="C:\Program Files (x86)\OpenNI\Bin\nimCodecs.dll" />
        <Module path="C:\Program Files (x86)\OpenNI\Samples\Bin\Release\NiPMD.dll" />
        <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Bin\XnVFeatures_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Data" />
        <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Bin\XnVFeatures_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Data" />
        <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_0\Bin\XnVFeatures_1_4_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_0\Data" />
        <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Bin\XnVHandGenerator_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Data" />
        <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Bin\XnVHandGenerator_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Data" />
        <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_0\Bin\XnVHandGenerator_1_4_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_0\Data" />
    </Modules>
    

    I'm not sure what's stopping this from working. When I run the viewer, it just says the device is not connected. In the mean time, I extracted the parts of your code and piped everything through by hand to KinFu and have some similar results to what you posted. For whatever reason, it's incredibly hard to control. Any camera movement seems to destroy the registration process. There are some parameters you can tweak, via this post:

    http://www.pcl-users...-td3828972.html

    Though everything still seems to bit a jittery. Are you piping the camera lens parameters (e.g. focal length) of the PMD into OpenNI?

    Renaud, any luck on your side? Have you tried Alex's code on your machine?

    Thanks for your help!

    #33 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 03 August 2012 - 08:04 AM

    Hey mrtm3050,

    did you change the paths to the plugins in the PMDHelper.cpp file?

    if (pmdOpen (&m_handle, "E:/projects/build32/Release/camboardnano", "", "E:/projects/build32/Release/camboardnanoproc", "") != PMD_OK)
    

    Totally forgot to warn about that in my last post :unsure: Sry!

    But it's great to hear that you've got it working anyways :thumbsup:
    I think it's the noise that is destroying the registration process :(
    Maybe for larger regions one should use the CamCube.

    Another thing we could try is to (as mentioned before) smooth the data. This could
    help getting the registration process more stable.

    Currently I'm not really piping the lens parameters to the algorithm, no.

    Best regards,
    Alex

    Edit: The thread you posted seems to be really interesting ^^
    Maybe tuning the parameters for smaller objects (since we are nearer) could also help in the process!
    Press [Ctrl]+[Alt]+[Del] to continue...

    #34 Rouno

    Rouno

      Member

    • Members
    • PipPip
    • 21 posts

    Posted 03 August 2012 - 09:23 AM

    Hi guys,

    I have quicly tried Alex code on my machine but I still have the Openni Exception on the vendor_id. I didn't investigate so there is nothing I can tell you about that.

    Right now, I still can't get a depth map of a sufficient quality for what I want to do so I am working on this.
    I will see about the lens calibration in kinfu too.

    If you have any advice or information they are welcome :)

    Regards,
    Renaud

    #35 mrtm3050

    mrtm3050

      Member

    • Members
    • PipPip
    • 17 posts

    Posted 05 August 2012 - 11:53 PM

    Alex,

    I imagine that the static link to the files is the problem, though I haven't quite tried it yet. I've been working on trying to get the Nano to work a little better with KinFu. Though, I've started to run out of things to try. I've been playing with the parameters, though I've seen no visible changes. There is just a lot of noise coming out the Nano, as you can clearly see in the KinFu surface.

    Here is a video:



    I'll probably post on PCL here to see if anyone in the community has any thoughts. I've since made a couple changes that have made the Nano perform slightly better (e.g., it doesn't reset now when I move laterally like it does at the end of the video). Let me know if you guys have any other comments or ideas of things to try.

    Thanks!

    #36 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 06 August 2012 - 09:01 AM

    Hey mrtm3050,

    I think the video looks great! Have you already implemented some sort of
    smoothing? Have you tried playing with the parameters of the bilateral filter like
    SetBilateralFilterSigmaSpatial

    SetBilateralFilterSigmaRange
    ?

    Have you tried averaging the data?
    SetAveraging On/Off

    SetAveragingFrames Nr

    Best regards,
    Alex
    Press [Ctrl]+[Alt]+[Del] to continue...

    #37 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 04 September 2012 - 12:36 PM

    Hey mrtm3050,

    could you try adding the following commands to your code :

    pmdProcessingCommand (hnd, err, 128, "SetAveraging on");
    pmdProcessingCommand (hnd, err, 128, "SetAveragingFrames 20");
    pmdProcessingCommand (hnd, err, 128, "SetAdaptiveAveraging on");
    

    For the last command to work you need the latest CamBoard nano CD 1.3.2 that you can find in the download section.


    Best regards,
    Alex
    Press [Ctrl]+[Alt]+[Del] to continue...

    #38 chyphen

    chyphen

      Member

    • Members
    • PipPip
    • 18 posts

    Posted 11 October 2012 - 08:22 PM

    Hi, Renaud:

    Thank you for sharing your code. I was trying to compile your code on Windows 7-64 bit with PCL 1.6.0, but got the error (detailed in an attached file).

    2>C:\Program Files\PCL 1.6.0\3rdParty\Boost\include\boost/exception/detail/exception_ptr.hpp(17): fatal error C1189: #error : This header requires exception handling to be enabled.

    Any suggestion about how to get around with the error?

    Thanks,

    CC

    #39 JimP4nsen

    JimP4nsen

      PMD[Team]

    • Administrators
    • 221 posts
    • LocationSiegen/Germany

    Posted 12 October 2012 - 06:19 AM

    Hi CC,

    have you enabled C++ exceptions in the Visual Studio project?
    It's under Properties->C/C++->Code Generation->Enable C++ Exceptions
    (I just assume you're using Visual Studio :) )

    Best regards,
    Alex
    Press [Ctrl]+[Alt]+[Del] to continue...

    #40 chyphen

    chyphen

      Member

    • Members
    • PipPip
    • 18 posts

    Posted 12 October 2012 - 07:28 PM

    Hi, Alex:

    I downloaded and built your code NiPMD - PCL.zip, moved NiPMD.dll as well as pmdaccess2.dll, camboardnano.W32.pap, and camboardnanoproc.W32.ppp to C:\Program Files (x86)\OpenNI\Samples\Bin\Release, and modified modules.xml as instructed. However, when I tried to run NiViewer.exe from C:\Program Files (x86)\OpenNI\Samples\Bin\Release, I got an error messag stating "Open fialed: The device is not connected". (The Camboard Nano did work with with lightvis"). Did I miss anything or did something wrong.

    Thanks,

    CC

    <Modules>
    <Module path="C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnDeviceFile.dll" />
    <Module path="C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnDeviceSensorV2KM.dll" configDir="C:\Program Files (x86)\PrimeSense\SensorKinect\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Bin\XnVFeatures_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Bin\XnVFeatures_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_1\Bin\XnVFeatures_1_4_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_1\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_2\Bin\XnVFeatures_1_4_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_2\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_5_2\Bin\XnVFeatures_1_5_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_5_2\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Bin\XnVHandGenerator_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Bin\XnVHandGenerator_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_1\Bin\XnVHandGenerator_1_4_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_1\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_2\Bin\XnVHandGenerator_1_4_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_2\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_5_2\Bin\XnVHandGenerator_1_5_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_5_2\Data" />
    <Module path="C:\Program Files (x86)\PrimeSense\Sensor\Bin\XnDeviceFile.dll" />
    <Module path="C:\Program Files (x86)\PrimeSense\Sensor\Bin\XnDeviceSensorV2.dll" configDir="C:\Program Files (x86)\PrimeSense\Sensor\Data" />
    <Module path="C:\Program Files (x86)\OpenNI\Bin\nimRecorder.dll" />
    <Module path="C:\Program Files (x86)\OpenNI\Bin\nimMockNodes.dll" />
    <Module path="C:\Program Files (x86)\OpenNI\Bin\nimCodecs.dll" />
    <Module path="C:\Program Files (x86)\OpenNI\Samples\Bin\Release\NiPMD.dll" />
    </Modules>




    0 user(s) are reading this topic

    0 members, 0 guests, 0 anonymous users