Sign in to follow this  
Followers 0
Rouno

PCL, OpenNI and Camboard Nano

62 posts in this topic

Hey mrtm3050,

yes the PMDHelper is the class responsible for all the grabbing. I know that the structure isn't that

nice, but the original code was something I made during my lunch breaks :D

The code can be used as an OpenNI module for PCL. I just compiled it and inserted the line into the modules.xml file.

Best regards,

Alex

Share this post


Link to post
Share on other sites

Got it. I added:


<Module path="C:\Programmierung\OpenNI\Samples\Bin\Debug\NiPMD.dll" />

to my modules.xml file.

What about the openni_capture.h and capture.cpp files for KinFu? In PmdHelper, I assume I call the EnumerateProductionTrees(...) and Create(...) as is done in capture.cpp? Followed by Init() and StartGenerating()?

Share this post


Link to post
Share on other sites

I changed nothing from the KinFu code. I just checked out the svn version of the PCL and build it.

After the build was finished I had the kinfu_app_release.exe file in the bin folder.

I assume that it just uses the first OpenNI Kinect that it finds. To be honest I never took a look

into the kinfu_app source code ^^

I just know that it worked out of the box :D

Best regards,

Alex

Share this post


Link to post
Share on other sites

Ah, I see. Well, then I assume it should work if I open up NiViewer or any other program that expects a Kinect. I must be missing a step. KinFu and NiViewer is saying that it can't find the device. Any thoughts?

In the capture.cpp file of KinFu, it appears to crash on the *** line in the code segment below (I couldn't bold it) when I debug:


pcl::gpu::CaptureOpenNI::open (int device)
{
impl_.reset ( new Impl () );
XnMapOutputMode mode;
mode.nXRes = XN_VGA_X_RES;
mode.nYRes = XN_VGA_Y_RES;
mode.nFPS = 30;
XnStatus rc;
rc = impl_->context.Init ();
if (rc != XN_STATUS_OK)
{
sprintf (impl_->strError, "Init failed: %s\n", xnGetStatusString (rc));
REPORT_ERROR (impl_->strError);
}
xn::NodeInfoList devicesList;
rc = impl_->context.EnumerateProductionTrees ( XN_NODE_TYPE_DEVICE, NULL, devicesList, 0 );
if (rc != XN_STATUS_OK)
{
sprintf (impl_->strError, "Init failed: %s\n", xnGetStatusString (rc));
REPORT_ERROR (impl_->strError);
}
xn::NodeInfoList::Iterator it = devicesList.Begin ();
for (int i = 0; i < device; ++i)
it++;
NodeInfo node = *it;
rc = impl_->context.CreateProductionTree ( node, impl_->node );
if (rc != XN_STATUS_OK)
{
sprintf (impl_->strError, "Init failed: %s\n", xnGetStatusString (rc));
REPORT_ERROR (impl_->strError);
}
XnLicense license;
const char* vendor = "PrimeSense";
const char* key = "0KOIk2JeIBYClPWVnMoRKn5cdY4=";
sprintf (license.strKey, key);
sprintf (license.strVendor, vendor);
rc = impl_->context.AddLicense (license);
if (rc != XN_STATUS_OK)
{
sprintf (impl_->strError, "licence failed: %s\n", xnGetStatusString (rc));
REPORT_ERROR (impl_->strError);
}
************************************************************
rc = impl_->depth.Create (impl_->context); // Crashes here
if (rc != XN_STATUS_OK)
{
sprintf (impl_->strError, "Depth generator failed: %s\n", xnGetStatusString (rc));
REPORT_ERROR (impl_->strError);
}
************************************************************
....
}

Share this post


Link to post
Share on other sites

Alex, would you mind posting your modules.xml file? Also, I had trouble getting PMD integrated in without putting the following two files in the Debug/Release directory of the exe files:

camboradnano.W32.pap

camboradnanoproc.W32.ppp

Share this post


Link to post
Share on other sites

Hey mrtm3050,

I forgot to mention that ... I have the pmdaccess2.dll and both plugin files in the same folder as

my module dll.

My modules.xml looks like this :


<Modules>
<Module path="E:\projects\OpenNI\bin\nimRecorder.dll" />
<Module path="E:\projects\OpenNI\bin\nimMockNodes.dll" />
<Module path="E:\projects\OpenNI\bin\nimCodecs.dll" />
<Module path="E:\projects\OpenNI\Samples\Bin\Release\NiPMD.dll" />
<!--<Module path="E:\projects\OpenNI\Samples\Bin\Debug\NiPMD.dll" />-->
<!--<Module path="E:\projects\PMDSampleModule\Release\PMDSampleModule.dll" />-->
<Module path="E:\projects\NITE\Features_1_3_0\Bin\XnVFeatures_1_3_0.dll" configDir="E:\projects\NITE\Features_1_3_0\Data" />
<Module path="E:\projects\NITE\Features_1_3_1\Bin\XnVFeatures_1_3_1.dll" configDir="E:\projects\NITE\Features_1_3_1\Data" />
<Module path="E:\projects\NITE\Features_1_4_1\Bin\XnVFeatures_1_4_1.dll" configDir="E:\projects\NITE\Features_1_4_1\Data" />
<Module path="E:\projects\NITE\Features_1_4_2\Bin\XnVFeatures_1_4_2.dll" configDir="E:\projects\NITE\Features_1_4_2\Data" />
<Module path="E:\projects\NITE\Features_1_5_2\Bin\XnVFeatures_1_5_2.dll" configDir="E:\projects\NITE\Features_1_5_2\Data" />
<Module path="E:\projects\NITE\Hands_1_3_0\Bin\XnVHandGenerator_1_3_0.dll" configDir="E:\projects\NITE\Hands_1_3_0\Data" />
<Module path="E:\projects\NITE\Hands_1_3_1\Bin\XnVHandGenerator_1_3_1.dll" configDir="E:\projects\NITE\Hands_1_3_1\Data" />
<Module path="E:\projects\NITE\Hands_1_4_1\Bin\XnVHandGenerator_1_4_1.dll" configDir="E:\projects\NITE\Hands_1_4_1\Data" />
<Module path="E:\projects\NITE\Hands_1_4_2\Bin\XnVHandGenerator_1_4_2.dll" configDir="E:\projects\NITE\Hands_1_4_2\Data" />
<Module path="E:\projects\NITE\Hands_1_5_2\Bin\XnVHandGenerator_1_5_2.dll" configDir="E:\projects\NITE\Hands_1_5_2\Data" />
</Modules>

When I execute the NiViewer.exe example from OpenNI it opens up my module (NiPMD.dll) and displays the CamBoard nano data.

I assume that the PCL OpenNI examples also use this modules.xml file so I could just execute the kinfu app.

Does the sample code work for you when you execute the OpenNI samples?

Best regards,

Alex

Share this post


Link to post
Share on other sites

Hrm, here's mine:


<Modules>
<Module path="C:\Program Files (x86)\OpenNI\Bin\nimRecorder.dll" />
<Module path="C:\Program Files (x86)\OpenNI\Bin\nimMockNodes.dll" />
<Module path="C:\Program Files (x86)\OpenNI\Bin\nimCodecs.dll" />
<Module path="C:\Program Files (x86)\OpenNI\Samples\Bin\Release\NiPMD.dll" />
<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Bin\XnVFeatures_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Data" />
<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Bin\XnVFeatures_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Data" />
<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_0\Bin\XnVFeatures_1_4_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_0\Data" />
<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Bin\XnVHandGenerator_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Data" />
<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Bin\XnVHandGenerator_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Data" />
<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_0\Bin\XnVHandGenerator_1_4_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_0\Data" />
</Modules>

I'm not sure what's stopping this from working. When I run the viewer, it just says the device is not connected. In the mean time, I extracted the parts of your code and piped everything through by hand to KinFu and have some similar results to what you posted. For whatever reason, it's incredibly hard to control. Any camera movement seems to destroy the registration process. There are some parameters you can tweak, via this post:

http://www.pcl-users.org/KinectFusion-failed-to-scan-small-physical-objects-td3828972.html

Though everything still seems to bit a jittery. Are you piping the camera lens parameters (e.g. focal length) of the PMD into OpenNI?

Renaud, any luck on your side? Have you tried Alex's code on your machine?

Thanks for your help!

Share this post


Link to post
Share on other sites

Hey mrtm3050,

did you change the paths to the plugins in the PMDHelper.cpp file?


if (pmdOpen (&m_handle, "E:/projects/build32/Release/camboardnano", "", "E:/projects/build32/Release/camboardnanoproc", "") != PMD_OK)

Totally forgot to warn about that in my last post :unsure: Sry!

But it's great to hear that you've got it working anyways :thumbsup:

I think it's the noise that is destroying the registration process :(

Maybe for larger regions one should use the CamCube.

Another thing we could try is to (as mentioned before) smooth the data. This could

help getting the registration process more stable.

Currently I'm not really piping the lens parameters to the algorithm, no.

Best regards,

Alex

Edit: The thread you posted seems to be really interesting ^^

Maybe tuning the parameters for smaller objects (since we are nearer) could also help in the process!

Share this post


Link to post
Share on other sites

Hi guys,

I have quicly tried Alex code on my machine but I still have the Openni Exception on the vendor_id. I didn't investigate so there is nothing I can tell you about that.

Right now, I still can't get a depth map of a sufficient quality for what I want to do so I am working on this.

I will see about the lens calibration in kinfu too.

If you have any advice or information they are welcome :)

Regards,

Renaud

Share this post


Link to post
Share on other sites

Alex,

I imagine that the static link to the files is the problem, though I haven't quite tried it yet. I've been working on trying to get the Nano to work a little better with KinFu. Though, I've started to run out of things to try. I've been playing with the parameters, though I've seen no visible changes. There is just a lot of noise coming out the Nano, as you can clearly see in the KinFu surface.

Here is a video:

I'll probably post on PCL here to see if anyone in the community has any thoughts. I've since made a couple changes that have made the Nano perform slightly better (e.g., it doesn't reset now when I move laterally like it does at the end of the video). Let me know if you guys have any other comments or ideas of things to try.

Thanks!

Share this post


Link to post
Share on other sites

Hey mrtm3050,

I think the video looks great! Have you already implemented some sort of

smoothing? Have you tried playing with the parameters of the bilateral filter like

SetBilateralFilterSigmaSpatial

SetBilateralFilterSigmaRange

?

Have you tried averaging the data?

SetAveraging On/Off

SetAveragingFrames Nr

Best regards,

Alex

Share this post


Link to post
Share on other sites

Hey mrtm3050,

could you try adding the following commands to your code :


pmdProcessingCommand (hnd, err, 128, "SetAveraging on");
pmdProcessingCommand (hnd, err, 128, "SetAveragingFrames 20");
pmdProcessingCommand (hnd, err, 128, "SetAdaptiveAveraging on");

For the last command to work you need the latest CamBoard nano CD 1.3.2 that you can find in the download section.

Best regards,

Alex

Share this post


Link to post
Share on other sites

Hi, Renaud:

Thank you for sharing your code. I was trying to compile your code on Windows 7-64 bit with PCL 1.6.0, but got the error (detailed in an attached file).

2>C:\Program Files\PCL 1.6.0\3rdParty\Boost\include\boost/exception/detail/exception_ptr.hpp(17): fatal error C1189: #error : This header requires exception handling to be enabled.

Any suggestion about how to get around with the error?

Thanks,

CC

Share this post


Link to post
Share on other sites

Hi CC,

have you enabled C++ exceptions in the Visual Studio project?

It's under Properties->C/C++->Code Generation->Enable C++ Exceptions

(I just assume you're using Visual Studio :) )

Best regards,

Alex

Share this post


Link to post
Share on other sites

Hi, Alex:

I downloaded and built your code NiPMD - PCL.zip, moved NiPMD.dll as well as pmdaccess2.dll, camboardnano.W32.pap, and camboardnanoproc.W32.ppp to C:\Program Files (x86)\OpenNI\Samples\Bin\Release, and modified modules.xml as instructed. However, when I tried to run NiViewer.exe from C:\Program Files (x86)\OpenNI\Samples\Bin\Release, I got an error messag stating "Open fialed: The device is not connected". (The Camboard Nano did work with with lightvis"). Did I miss anything or did something wrong.

Thanks,

CC

<Modules>

<Module path="C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnDeviceFile.dll" />

<Module path="C:\Program Files (x86)\PrimeSense\SensorKinect\Bin\XnDeviceSensorV2KM.dll" configDir="C:\Program Files (x86)\PrimeSense\SensorKinect\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Bin\XnVFeatures_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_0\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Bin\XnVFeatures_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_3_1\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_1\Bin\XnVFeatures_1_4_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_1\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_2\Bin\XnVFeatures_1_4_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_4_2\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Features_1_5_2\Bin\XnVFeatures_1_5_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Features_1_5_2\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Bin\XnVHandGenerator_1_3_0.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_0\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Bin\XnVHandGenerator_1_3_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_3_1\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_1\Bin\XnVHandGenerator_1_4_1.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_1\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_2\Bin\XnVHandGenerator_1_4_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_4_2\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_5_2\Bin\XnVHandGenerator_1_5_2.dll" configDir="C:\Program Files (x86)\PrimeSense\NITE\Hands_1_5_2\Data" />

<Module path="C:\Program Files (x86)\PrimeSense\Sensor\Bin\XnDeviceFile.dll" />

<Module path="C:\Program Files (x86)\PrimeSense\Sensor\Bin\XnDeviceSensorV2.dll" configDir="C:\Program Files (x86)\PrimeSense\Sensor\Data" />

<Module path="C:\Program Files (x86)\OpenNI\Bin\nimRecorder.dll" />

<Module path="C:\Program Files (x86)\OpenNI\Bin\nimMockNodes.dll" />

<Module path="C:\Program Files (x86)\OpenNI\Bin\nimCodecs.dll" />

<Module path="C:\Program Files (x86)\OpenNI\Samples\Bin\Release\NiPMD.dll" />

</Modules>

Share this post


Link to post
Share on other sites

Hi, Alex:

Thank you for the suggestion of enabling C++ Exceptions. (Yes, I am using Visual Studio 2010.) Previous errors disappear. However, I am getting another errors, and could not figure out why.

2>DepthDevice.obj : error LNK2019: unresolved external symbol "public: __cdecl openni_wrapper::OpenNIException::OpenNIException(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &,unsigned int,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (??0OpenNIException@openni_wrapper@@QEAA@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@0I0@Z) referenced in function "void __cdecl openni_wrapper::throwOpenNIException(char const *,char const *,unsigned int,char const *,...)" (?throwOpenNIException@openni_wrapper@@YAXPEBD0I0ZZ)

2>DepthDevice.obj : error LNK2001: unresolved external symbol "public: virtual __cdecl openni_wrapper::OpenNIException::~OpenNIException(void)" (??1OpenNIException@openni_wrapper@@UEAA@XZ)

2>DepthDevice.obj : error LNK2001: unresolved external symbol "public: virtual char const * __cdecl openni_wrapper::OpenNIException::what(void)const " (?what@OpenNIException@openni_wrapper@@UEBAPEBDXZ)

2>D:\ccSoftware\CamboardNano\PCLSample\Debug\openni_grabber.exe : fatal error LNK1120: 3 unresolved externals

2>

2>Build FAILED.

Those functions seem to have been defined in openni_exception.h.

OpenNIException (const std::string& function_name, const std::string& file_name, unsigned line_number, const std::string& message) throw ();

Any idea? Suggestion? Thanks.

CC

Share this post


Link to post
Share on other sites

Hi, Alex:

Please ignore my post regarding OpenNIException(). I solved the problem by adding openni_exception.cpp into the project (though I though it should have been in the linked PCL). However, when I tried to run it, I got similar error as one from running NIPDM (posted in another post).

Generation creation failed: The device is not connected. As mention in my earlier post regarding NiPDM, I did copy two plugin files and dll into the folder where Niviewer is and modify modules.xml as suggested. Double checked with running Camboard Nano with lightvis to make sure the senor was working.

I must have missed something. Any suggestion and help will be highly appreciated.

Thanks,

Chien

Share this post


Link to post
Share on other sites

Hmmmm, did you try to start the NiViewer example?

And could you post the content of the modules.xml and the samplesconfig.xml file?

When it comes to configuring devices and modules OpenNI isn't very intuitive imho :/

Best regards,

Alex

Share this post


Link to post
Share on other sites

I tried to run NiViewer from C:\Program Files\OpenNI\Samples\Bin\Release iwth pmdaccess2.dll, NiPMD.dll, camboardnano.W32.pap and camboardnanoproc.W32.ppp in it, but got an error "Open fialed: The device is not connected!

here is modules.xml:

<Modules>

<Module path="C:\Program Files\OpenNI\Bin\nimRecorder.dll" />

<Module path="C:\Program Files\OpenNI\Bin\nimMockNodes.dll" />

<Module path="C:\Program Files\OpenNI\Bin\nimCodecs.dll" />

<Module path="C:\Program Files\OpenNI\Samples\Bin\Release\NiPMD.dll" />

<Module path="C:\Program Files\PrimeSense\SensorKinect\Bin\XnDeviceFile.dll" />

<Module path="C:\Program Files\PrimeSense\SensorKinect\Bin\XnDeviceSensorV2KM.dll" configDir="C:\Program Files\PrimeSense\SensorKinect\Data" />

</Modules>

and samplesconfig.xml:

<OpenNI>

<Licenses>

<!-- Add licenses here

<License vendor="vendor" key="key"/>

-->

</Licenses>

<Log writeToConsole="false" writeToFile="false">

<!-- 0 - Verbose, 1 - Info, 2 - Warning, 3 - Error (default) -->

<LogLevel value="3"/>

<Masks>

<Mask name="ALL" on="true"/>

</Masks>

<Dumps>

</Dumps>

</Log>

<ProductionNodes>

<Node type="Depth" name="Depth1">

<Configuration>

<Mirror on="true"/>

</Configuration>

</Node>

<Node type="Image" name="Image1" stoponerror="false">

<Configuration>

<Mirror on="true"/>

</Configuration>

</Node>

<!--

<Node type="Audio" name="Audio1"/>

-->

</ProductionNodes>

</OpenNI>

By the way NiViewer works with ASUS Xtion Pro.

Best Regards,

CC

Share this post


Link to post
Share on other sites

Hi, Alex:

After reading through your earlier posts regarding the pathes to two pluggins in pmdHelper.cpp, I changed pmdOpen()'s arguments to

if (pmdOpen (&m_handle, "C:/Program Files/OpenNI/Samples/Bin/Release/camboardnano", "", "C:/Program Files/OpenNI/Samples/Bin/Release/camboardnanoproc", "") != PMD_OK)

{

return (XN_STATUS_DEVICE_NOT_CONNECTED);

}

I got different error: "Open failed: This operation is invalid! Any thoughts?

Best regards,

CC

Share this post


Link to post
Share on other sites

Hi CC,

could you try to comment out the Primesense sensors in the modules.xml like this


<!-- <Module path="C:\Program Files\PrimeSense\SensorKinect\Bin\XnDeviceFile.dll" />
<Module path="C:\Program Files\PrimeSense\SensorKinect\Bin\XnDeviceSensorV2KM.dll" configDir="C:\Program Files\PrimeSense\SensorKinect\Data" /> -->

Best regards,

Alex

Share this post


Link to post
Share on other sites

I commented out those two lines (in fact I also tried to outright deleted them), but it didn't chagned anythinig. I added some print out statements in pmdHelper.cpp. NiViewer went throught PMDHelper::init(), and then spit out

Open failed: This operation is invalid.

without calling PMDHelper::StartGenerating(). I don't know which operation it refers to.

Best regards,

CC

Share this post


Link to post
Share on other sites

Hey CC,

to be honest I'm not sure what's wrong with your setup :(

What you could try is to change these lines in the samplesconfig.xml file


<ProductionNodes>
<Node type="Depth" name="Depth1">
<Configuration>
<Mirror on="true"/>
</Configuration>
</Node>
<Node type="Image" name="Image1" stoponerror="false">
<Configuration>
<Mirror on="true"/>
</Configuration>
</Node>
<!--
<Node type="Audio" name="Audio1"/>
-->
</ProductionNodes>

to


<ProductionNodes>
<Node type="Depth" name="Depth1">
</Node>
</ProductionNodes>

Maybe anyone else has other suggestions?

Best regards,

Alex

Share this post


Link to post
Share on other sites

Hi, Alex:

Thanks for the suggestion. It works!!!

Do you (or does anyone) know if there is ROS driver available for Camboard Nano?

Chien

Share this post


Link to post
Share on other sites

That's great to hear :D

As I said, OpenNI can be quite cumbersome when it comes to

configuring modules ...

I know that there is a ROS driver for the PMD CamCube, but I'm not sure

if it already has been adapted for the CamBoard nano. But if the original implementation

uses the PMDDataDescription (e.g. for width, height and orientation of sensor) and these

values aren't hardcoded the adaption should boil down to changing

the plugin names in the source code.

Best regards,

Alex

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0