Photogrammetry: Does shooting RAW or JPG make a difference?

This is one of those questions that I see cropping up regularly in discussions online and off; Should I shoot and process RAW photos, or will I get the same results with out-of-camera JPGs?

There is absolutely a school of thought among photogrammetry practitioners that everything should be absolutely top-quality: Prime lenses, shooting in RAW, DSLR only. I tend to play it a bit more fast and loose:

Clearly, using the best lens, and the best camera, assuming they’re used correctly, is going to get you better photos (crisper, more detail), which will be able to bring out more features in a photogrammetric model. And you’re not going to get great models of small objects if you don’t have a decent macro lens. But is there really enough difference between a RAW image and a JPG? There’s certainly a lot of scope for recovering details from RAW images – brightening dark areas, underexposing bright areas etc, but outside of relatively simple batch processing, you don’t really want to be manually adjusting hundreds of images – I’ll discuss this later.

I wanted to do a simple test of RAW vs out-of-camera JPG datasets, just to see if there were any difference in either processing time or quality of the final model (both mesh and texture). I set up my Extended Automated Photogrammetry Setup, and used only my Sony Nex 6 with it’s 18-55mm lens (I was zoomed in to about 50mm). I collected photo sets of two fossils – a fairly boring (in terms of features) dinosaur vertebra (Iguanaodon I think?), and a really nice Ammonite I’ve had since I was a child. I set it up to take 100 photos on each revolution, and did three orientations for each specimen, combining all images for each specimen into single photo sets. The camera was set to shoot RAW and JPG, so every time it was triggered, it saved the photo in both formats.(due to a couple of minor hiccups, the total number of photos wasn’t 300, it was 317 for the Ammonite, and 321 for the vertebra).

All processing was done on my home desktop (16Gb DDR3, i7 4790K@4.4GHz, Nvidia 2060 super).

I ran all the data through Meshroom using a powershell batch script (below) that processed each dataset sequentially and recorded timings. You need a blank meshroom graph, which you can make by opening Meshroom, changing the settings you want, then save as.

"Iggy_JPG"
measure-command -expression {E:\Programs\Meshroom-2019.2.0\meshroom_photogrammetry.exe --input E:\Photog\Iggy_jpg\ --cache E:\Photog\iggjpgcache --pipeline .\BlankGraph.mg}
"IGGY_RAW"
measure-command -expression {E:\Programs\Meshroom-2019.2.0\meshroom_photogrammetry.exe --input E:\Photog\Iggy_raw\ --cache E:\Photog\iggrawcache --pipeline .\BlankGraph.mg}
"AMM_JPG"
measure-command -expression {E:\Programs\Meshroom-2019.2.0\meshroom_photogrammetry.exe --input E:\Photog\AmmoniteJPG\ --cache E:\Photog\AMMJPGCACHE --pipeline .\BlankGraph.mg}
"AMM_RAW"
measure-command -expression {E:\Programs\Meshroom-2019.2.0\meshroom_photogrammetry.exe --input E:\Photog\AmmoniteRAW\ --cache E:\Photog\AMMRAWCACHE --pipeline .\BlankGraph.mg}

Save the above as Runall.ps1, then you can open powershell and run $.\Runall.ps1 > output.log, and it’ll give you the time taken for each step. This script sees each photo set in E:\Photog, and runs my meshroom graph ‘BlankGraph.mg’, and uses a new cache folder for each so I can go back and check different stages if I want.

Meshroom happily accepts RAW files and processes them natively, just as it does JPG.The Meshroom template used default settings, except I enabled Guided Matching in the Feature Matching node, and I set texture algorithm to LSCM to produce a single texture file 4096×4096 (see my explanation of parameters here). I didn’t do any decimation or retopology in Meshroom.

I was going to duplicate this test in Metashape, but unfortunately Metashape can’t accept .ARW files directly – I’d have to convert them to digital negatives first.

Speed:

So, did processing RAW photos take any more or less time than JPGs?

Turns out, RAW was slightly quicker:

Not massively, maybe a 10-20% difference in time taken. That might be important to you on big datasets, but it’s not huge. For this example, it matters little to me if I’m tying my computer up for 5 hours or 6 hours – either way it needs to run over night if I want to use my computer during the day.

Model Quality

First up, a quantitative measure – number of triangles:

I provide the above numbers just for a quick reference, but as we all know, poly count isn’t a particularly accurate indicator of model quality. Interesting that the RAW models are both a couple of hundred thousand faces smaller.

Here’s the vertebra side-by-side in blender, rendered with a simple grey material (we’ll come to textures shortly):

The models are almost identical. A couple of odd dimples on the front face of the RAW model, and some triangles with reversed faces at the top.

The same holds true for the Ammonite:

Texture Quality

There were some major differences in the contrast of the textures. Let’s start with the vertebra:

Clearly, the RAW photos produced a texture that is darker. I like it more aesthetically, but is it more accurate? Hard to say, because the colour of the specimen in-hand differs in different light conditions. I’d say the RAW model is closer to what the specimen looks like when I’m holding it in a normal-lit room, and the JPG-generated texture more closely resembles what the specimen looked like in the light box. I guess in that regard, the JPG texture is more accurate?

Both are very well detailed textures as can be seen in these closeups (of the middle of the bone, as show in the image above):

Below are the texture files. Unfortunately, the texture map was calculated differently for RAW and JPG, so they are hard to compare.

And here’s the Ammonite:

Just as for the vertebra, the RAW images produced a darker texture, possibly a bit muddier in this example, but ultimately just about as detailed:

I think I’d say the RAW texture is noisier.

Again, here’s the texture files directly:

Caveats:

Before I summarise, I want to cover my back and address a couple of glaring issues. First, this test is just on close-range photogrammetry. I don’t imagine it would be that different, but Drone or large-scale photogrammetry might behave differently. Secondly, The set-up was reasonably good, and I took half decent photos to start with; both specimens were in a light box, and had good contrast against the background. Dark objects against dark backgrounds (or light against light), or poor lighting set-ups with low contrast might benefit from shooting RAW more. Thirdly, this test might only be applicable to Sony cameras – different vendors treat JPG conversion differently, and may lose or retain more detail. Finally, my textures are high res, but not extreme. If you’re producing 8k or 16k textures, you might find the differences more pronounced.

As I mentioned at the beggining, I wasn’t able to load the RAW files directly into Metashape, so can’t confirm if what we’re seeing here is unique to Meshroom.

Summary, and the Pros/Cons of shooting RAW vs JPG

Ok, so this simple test shows RAW photos process a bit quicker, make meshes of about the same quality, and produce textures that maybe have a touch more noise than models made by JPGs, but perhaps a nicer colour.

Advantages of shooting RAW:

The main advantage of shooting RAW, as I see it, is that you can recover more details from photos that lack contrast, or were poorly lit. Ideally through batch processing, but individually if necessary (though you can do this, albeit to a lesser extent, with jpgs). Your original data is also potentially more robust – it’s lossless so you’re not losing something every time you open and save, though this isn’t a huge issue for JPGs if you’re not editing them.

Processing models from RAW images seemed to be noticeably quicker. Whether that 10-20% speed up is worth it to you in exchange for the downsides listed below is up for debate.

Disadvantages of shooting RAW:

File space. That’s about it, but it’s a big one. The JPG datasets for the Ammonite and Vertebra were 918 MB and 0.99GB respectively. The RAW datasets were 4.9GB and 4.98GB, so about 5x bigger. I have hundreds of gigs of photogrammetry datasets going back a decade, and if they were all RAW files, I just wouldn’t have the space to store them. I know that storage space is constantly increasing, but so is the speed of photogrammetry software, so I’m taking more photos. I’ve made the case twice in the literature (once in Davies et al 2017, and again in Falkingham et al 2018) that when publishing photogrammetric data, the model and the original photos need archiving. Many of my JPG datasets are already pushing the limits of Figshare, Zenodo, and other data repositories, and are already difficult to share with colleagues over home broadband connections.

It’s also not just the dataset – the cache folder Meshroom produces was also several gigs larger for the RAW datasets.

Handling, processing and archiving datasets 5x larger for a very minor increase in speed and the ability to recover details if photos are murky isn’t worth it to me. Far better, I think, to shoot JPG and just make sure you take decent photos to start with.

I say this in full knowledge that future Peter may well look back on this day, surrounded as he is by a bunch of irreplaceable photo sets beset by JPG compression artefacts, and curse past Peter for not saving everything in RAW.

You can compare the models here:

And finally, here’s a little render of the Ammonites, because why not?

[Edit] Llion Evans commented below on what the difference between the meshes looks like. I put them both in cloud compare to see. I haven’t scaled the model, but you can see that differences between meshes are on the order of 0.0002, if the ammonite is 0.4 units across.

And here’s the Vert, same scale of difference between meshes: