Photogrammetry Testing 11: VisualSFM & OpenMVS (and COLMAP+openMVS)

Part 1: Photoscan | Part 2: VisualSFM+MeshRecon | Part 3: VisualSFM+PMVS+Meshlab | Part 4: OpenMVG + MVE | Part 5: MVE | Part 6: ReMake | Part 7: SMVS (+MVE) | Part 8: COLMAP | Part 9: 3DF Zephyr Free | Part 10: MicMac | Part 11: VisualSFM + openMVS, & COLMAP + openMVS | Part 12: Revisiting OpenMVG (with OpenMVS) [updated] | Part 13: Revisiting 3DFZephyr Free (v4.001) | Part 14: AliceVision Meshroom

Current Recommendation: Part 14, AliceVision Meshroom

Here’s the original post, with links to all post

This post was updated 27/5/17 – see end of post; turns out using COLMAP for sparse reconstruction let openMVS create an awesome mesh.

Normally I see OpenMVS mentioned in the same breath as OpenMVG, but I’m still having significant troubles with OpenMVG behaving with this dataset, so I’ll run the matching with VisualSFM.

Download VisualSFM here: http://ccwu.me/vsfm/

And Download OpenMVS binaries here: http://cdcseacave.github.io/openMVS/ (or build it yourself from source.)

It’s a while since I’ve used VisualSFM, preferring COLMAP recently.  My word it is fast!

95 seconds to a full reconstruction of all 53 images:

vsfm_openMVS1.JPG

Then SfM->Save nvmatch.

I’ve saved it in a folder called “work”, and named the file “Styrac.nvm”.  My openMVS binaries are in a folder at the same level as work, called openMVS_binaries.

I’m running the following commands through powershell in windows.

Convert Scene:

..\openMVS_binaries\InterfaceVisualSFM.exe -i .\styrac.nvm

This was super quick.

Densify point cloud:

..\openMVS_binaries\DensifyPointCloud.exe -I styrac.mvs

 

Here’s what the dense cloud looks like:

vsfm_openMVS2.JPG

It’s… well, it’s pretty noisy.

Time taken: 17 minutes 6 seconds.

Mesh reconstruction

We can run this on either the sparse or the dense point cloud.

..\openMVS_binaries\ReconstructMesh.exe styrac_dense.mvs

I also ran this on the sparse cloud just for interest sake, and because the dense cloud was so noisy:

vsfm_mvs3.JPG

Mesh Refinement

After reconstructing the mesh, we refine it.  Again, I’ve done this on both the sparse and dense cloud:

..\openMVS_binaries\RefineMesh.exe styrac_dense_mesh.mvs –resolution-level 2

It’s worth noting that on my 16Gb ram machine, I ran out of memory while refining the mesh for the sparse cloud so needed to add the “resolution-level 2″ to reduce memory usage.

vsfm_mvs4.JPG

The sparse cloud has actually resulted in a mesh not too dissimilar to the dense cloud.

Texturing:

Use the command:

..\openMVS_binaries\TextureMesh.exe styrac_dense_mesh_refine.mvs

vsfm_mvs5.JPG

Concluding remarks:

Well, sadly I didn’t manage to get amazing results.  It looks like things fell apart on the Densify Point Cloud stage, and perhaps with a bit more digging I’ll be able to improve the results.

 

 

What is noteworthy is that meshing the sparse cloud actually resulted in a reasonably nice model, especially when textured, and could perhaps be useful for “quick’n’dirty” models.

As always, if time permits I’ll play a little more and see if I can improve these results.  For now though, ColMap remains my Goto.

UPDATE 27/5/17

Turns out what you use for the sparse reconstruction matters.  Even though VisualSFM looked good at the camera matching stage, it was the reason the final mesh from openMVS wasn’t great. The thread with the author of openMVS, at the sketchfab model above inspired me to try a different matching software.  So I ran COLMAP and exported the cameras + sparse cloud as a *.nvm file, then did exactly the same with openMVS as described above.  This was the result:

 

I’m sure you’ll agree that this is probably about the best model produced yet in this series!

33 thoughts on “Photogrammetry Testing 11: VisualSFM & OpenMVS (and COLMAP+openMVS)

Add yours

  1. Great work comparing and documenting all these photogrammetry solutions! Regarding this post in particular, I can complement it with some minor comments:
    – indeed, OpenMVS is usually mentioned together with OpenMVG, thought the two projects have nothing in common, except that the creators of the two projects are friends
    – OpenMVS is responsible only for the dense reconstruction part (dense point-cloud, mesh, texture) of the pipeline, and the quality of these outputs depend a lot on the accuracy of the camera calibration supplied as input; in this case, the problem is the VisualSFM input as discovered and discussed in more detail here: https://skfb.ly/6rwO9
    – OpenMVS project was designed and developed to deliver high quality textured meshes as the end result (for many reasons, not here the place to discuss); the dense point-cloud is only an intermediate step towards that goal and it was never meant to be used alone; do not be fulled by it’s noisiness, it is know to be like that and that does not affect the end result

    1. Yeah, the dense cloud only as an intermediary caught me out, having been used to pipelines where the dense point cloud is generally clean and an end in and of itself.

      Thank you!

    1. Thanks. Is that just camera matching/sparse reconstruction? I’m pretty burned out from all this for now (and I need to spend some time actually processing real data, instead of just testing), but perhaps in the future I’ll take a look.

      1. sparse + depth map reconstruction. Yeah, it seems like you found the perfect photogrammetry pipeline for now (Colmap + openMVS) … Thanks for the testing/overview btw.

    1. I’d say something with a GUI that takes you through the whole process, such as Autodesk ReMake. If you want something totally free, either VisualSFM or COLMAP are going to be the most straightforward.

  2. I tried using COLMAP->openMVS as you describe above, but I couldn’t figure out how to do the second step in the openMVS part. When I “convert scene” from the nvm file from COLMAP, no mvs file is produced for the next step. Am I missing something here?

    1. Sometimes this happens if there’s no matches in the nvm file. Try loading COLMAP and then importing the nvm file to see if it contains calibrated cameras.

      1. I don’t see where you can import an nvm file in COLMAP. I tried “import model”, but that does accept .nvm files, instead it requires a folder with cameras.bin, images.bin, points3D.bin and project.ini files in it. There is also “import model from”, but that only accepts a .ply file. I believe I got a decent sparse reconstruction in COLMAP, but here’s my .nvm file if you want to take a look:

        https://www.dropbox.com/s/wv5hygoqsrzupla/FH3D.nvm?dl=0

        Also, here’s a photo of my sparse reconstruction in COLMAP:

        I appreciate any help you can give me!
        -Ben

  3. Nevermind, I think I figured it out. The images need to be in the same folder the whole time. Either that or openMVS didn’t like name image names, not sure which. I just moved them all the the same folder where I would be saving the nvm file and I renamed them all 1, 2 3, etc before starting COLMAP and making the sparse reconstruction. I exported that nvm to the image folder and now openMVS seems to be working now. Hopefully the final mesh will look good!

      1. Thanks for all your help, the model turned out great! I wanted to mention something I noticed – you said you had to reduce the resolution in the refinement step of openMVS becasue you ran out memory, but actually that shouldn’t necessarily stop you from running the refinement at max resolution.

        I have a 32GB RAM machine, but I still ran out of memory when I did a second project with more images (a little over 100). However, I realized that even when no more RAM is available, it continues to process at a reasonable rate using the paging file. My paging file is on an SSD, so I’m sure that helps, but most rigs use an SSD for the OS and applications these days anyway, so I thought that might be the case for you as well.

        When my RAM ran out (and of course my disk usage spiked), I noticed my processor usage for RefineMesh.exe only dropped from 95-100% to about 55-60%, implying that it was still churning along at around 60% of the max speed. The total time for the refinement was around 6 hours (using all 4 cores on a 3.5GHz i5), which isn’t too bad if you’re letting it run overnight anyway. Of course, if you’re trying to simultaneously use your computer for other tasks, things will be quite slugging with no available RAM. It might be nice if there was an option to cap the applications memory usage, similar to how you can reduce the number of cores it uses, but nonetheless it still produces great models!

        Also, I think I determined that it was the filenames that was messing me up originally, so I includes a file-renaming script in my pipeline and haven’t had any problems since. Not sure if you still need the images in the same folder, but I’m keeping them there for now anyway (if it ain’t broke, don’t fix it!)

        TL;DR – Running out of memory doesn’t slow down refinement that much if you have an SSD, and some filenames are a problem for openMVS.

  4. Have you tried using openMVG and openMVS?
    Both of them have tons of parameters?
    You can make openMVG get same result (if not better) than COLMAP.

    1. I have, but I just can’t get openMVG to reliably match my photos. I don’t know if it’s the way I do things (I take photos pretty fast and loose and mix variable focal lengths), or just the way I’ve got openMVG set up, but I’ve literally never been able to get decent matches with anything but the sample data that’s in the openMVG repository.

  5. Do you know if openMVS (for Windows) also uses CUDA on GPUs or does it only use CPU processing? If it also uses GPUs can you than accelerate the computations?

    1. As far as I’m aware, it does not, sadly. In fact, I don’t think I’ve found a free/open source solution that handles dense reconstruction/meshing using the GPU (could be wrong though)

  6. I am using Mac and kind of stuck with OpenMVS or OpenMVG, neither seems to be working when I tried to build 😦

    Currently using this method: CMVS PMVS

    1. Yeah, I find macs just aren’t cutting it these days for scientific software, especially fast moving (developmentally) stuff like photogrammetry software that generally uses CUDA. It can be done, but not something I’m familiar with.

      1. I ended up using CloudCompare which is available as binary app for Mac. I am hoping that Mac is catching up with their own iOS devices to generate point cloud and meshing. Fingers crossed!

  7. Whenever I run DensifyPointCloud on Windows, it is VERY noisy and looks uncalibrated and unusable, meanwhile my sparse NVM export is clean and very accurate, with 5 views per point in some cases. Whenever I run DensifyPointCloud on Linux, it is very clean. I should note my Windows machines is AMD while my Linux machine is using Intel for the CPU which provides AVX & SSE. This is odd because I believe cDc mentioned on the openmvs page that Windows is the targeted platform.

    1. Yeah, that’s normal – you need to go all the way to mesh. The densified point cloud is just an intermediate stage, it’s not an end in itself.

      If you’re just wanting a dense point cloud, stick with COLMAP, or use VisualSFM and PMVS/CMVS (Or Meshroom)

  8. Hello, at the end of the tutorial you use a kind of widget on the website to view the 3D model. What widget or canvan is it?
    And thanks for the tutorial

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Website Built with WordPress.com.

Up ↑

%d bloggers like this: