Night shot with 5D3 + ML, how to avoid noise?

Started by adrjork, August 06, 2019, 03:34:36 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

adrjork

Hi everybody,
newbie question here: I'm going to take a night shot with 5D3 + ML (the scene is a lady walking on a sidewalk, the camera is moveless on tripod, the only lights are a street lamp and the moon).
I made a test with Sigma 35mm at f/1.4, T/30 and ISO 3200. The result was a very noisy shot because the high ISO, but even setting ISO 1600 doesn't help because image is darker so noise appears anyway.
I'm a bit frustrated...

I've thought a couple of solutions:

1. I could try to shoot not-yet-in-night, when the sky just start to be darker but is not black, and then lowering brightness in post to obtain "night effect". But the truth is that all the shadows created by the street lamp are visible only when sky is "noisily" black;

2. I could take a shot of the walking lady when the sky is not-so-black, and later I could take a "clean" timelapse of the street during night, and then masking the walking lady in post in order to add her over the "clean" street in timelapse. Interesting effect perhaps, but again the lady will not have consisten lights and shadows on her in coparison with the night street.

So, how could I solve it?

Thanks a lot

KirbyLikes525

Photo or video? Are you shooting raw? What's your post processing workflow?
5D Mark II
10 bit 12 bit Experimental Build

adrjork

Quote from: KirbyLikes525 on August 06, 2019, 03:47:46 AM
Photo or video? Are you shooting raw? What's your post processing workflow?
Thanks for your reply Kirby.

Video: more precisely, my first solution is thought as video only, while the second is a combination of video (walking lady) over the timelapse (photos).

Uncompressed RAW 14bit.

My workflow is MLV-to-cDNG through Switch, then directly into Davinci. Nothing more.

KirbyLikes525

My next question is how did you expose in camera? @ +/-0 or some other value? 

Maybe someone else with a cDNG workflow can chime in with how they reduce noise. I have a Pro Res workflow that gives me little to no noise.
5D Mark II
10 bit 12 bit Experimental Build

yourboylloyd

Quote from: adrjork on August 06, 2019, 03:34:36 AM
I'm going to take a night shot with 5D3 + ML.

I made a test with Sigma 35mm at f/1.4, T/30 and ISO 3200. The result was a very noisy shot because the high ISO, but even setting ISO 1600 doesn't help because image is darker so noise appears anyway.

So, how could I solve it?


Why are you shooting RAW in low light? 5D3 isn't too bad at low light in regular settings. Just shoot in MOV/H264

Also. Try using fps override. If you shoot at 12 frames per second then you can drop your shutter down to let in double the light. Just have her do her actions in slow motion if you have to! (You could also try 6fps and could let in quadruple the light). You can then bring it into an editor and adjust the speed. Just make sure she moves SUPER slow. I can imagine it will be hard to get sharp focus if you're at 1.4 too and all that motion blur.

Just shoot in H.264. RAW shouldn't really need to be used honestly in my opinion.
Join the ML discord! https://discord.gg/H7h6rfq

a1ex

If highlight clipping is not an issue, and you have already maxed out aperture and exposure time, feel free to use ISO 6400. On 5D3's 1080p regular (non-crop) video mode, I'd expect it to be a bit cleaner than 3200 (by 0.5 stops) or 1600 (by 1.15 stops).

It won't do miracles, though. Half FPS will let you double the exposure time, i.e. another full stop. Temporal denoising could help, too.

adrjork

Thank you all for your replies.
Quote from: a1ex on August 06, 2019, 07:05:36 AM
feel free to use ISO 6400.
Very interesting topic! I didn't know it. You are the wizard here, Alex. Anyway – please forgive my skepticism – even just at 3200 ISO I see a lot of noise (noticeable worsening than 1600), so I'm afraid of using directly 6400 ISO...
Perhaps my issue could be related to the ratio I use? I usually shoot at 5:3 (1920x1152). Do you think that shooting with 16:9 ratio could improve S/Noise performance?

a1ex

Quote from: adrjork on August 06, 2019, 03:38:14 PM
even just at 3200 ISO I see a lot of noise (noticeable worsening than 1600), so I'm afraid of using directly 6400 ISO...

QuoteI made a test with Sigma 35mm at f/1.4, T/30 and ISO 3200. The result was a very noisy shot because the high ISO, but even setting ISO 1600 doesn't help ...

There's your issue - all else being equal, higher ISO is going to give *lower* noise. That is, 1/30 f/1.4 ISO 1600 is going to be noisier than 1/30 f/1.4 ISO 3200 (by 0.65 stops). In the same way, 1/30 f/1.4 ISO 3200 is going to be noisier than 1/30 f/1.4 ISO 6400 (by 0.5 stops), assuming you will be normalizing the exposure in post to get comparable renderings.

So, as long as highlights are not clipped, increasing ISO from 1600 to 6400 is going to give lower noise in shadows - by 1.15 stops (yes, a little more than one full stop). Of course, this shadow improvement will cost you exactly 2 stops of highlights.

The above numbers apply to 5D3 in 1080p mode (3x3 binning). Things will be different in 1:1 crop mode (ISO 6400 no longer providing a noticeable benefit, but that's not applicable here), or on other cameras (where the DR figures may be slightly different).

Don't take my numbers from granted - feel free to do the following test, on the same static scene:
1) 1/30 f/1.4 ISO 1600 (this will be best in highlights)
2) 1/30 f/1.4 ISO 3200
3) 1/30 f/1.4 ISO 6400 (expecting to be best in shadows)
4) 1/15 f/1.4 ISO 1600 with FPS override (expecting shadows as good as #3 and highlights as good as #2)

In PAL land, you will need 1/25 and 1/12.5 for the above test, to avoid flicker. NTSC would require 1/30 (your initial setting) and 1/15.

You may cut one second from each of these 4 clips, for evaluation. I'm curious about the results, as my recommendation comes from synthetic tests, not from real-world experience.

adrjork

Quote from: a1ex on August 06, 2019, 03:57:14 PMThe above numbers apply to 5D3 in 1080p mode (3x3 binning). Things will be different in 1:1 crop mode (ISO 6400 no longer providing a noticeable benefit, but that's not applicable here), or on other cameras (where the DR figures may be slightly different).
This could be the point! I'll surely test 6400 ISO in 16:9 1080p (perhaps the problem was the 5:3 ratio I used mainly to have room of re-framing in post).
Quote from: a1ex on August 06, 2019, 03:57:14 PM
[...] with FPS override [...]
Very-newbie question: even if I shoot always in 25fps, in ML menu I usually keep FPS override always activated (both Canon menu and ML override are set on 25fps). Is that a mistake? Should I deactivate ML override when shooting at 25fps?

KirbyLikes525

Quote from: adrjork on August 06, 2019, 03:38:14 PM
Thank you all for your replies.Very interesting topic! I didn't know it. You are the wizard here, Alex. Anyway – please forgive my skepticism – even just at 3200 ISO I see a lot of noise (noticeable worsening than 1600), so I'm afraid of using directly 6400 ISO...
Perhaps my issue could be related to the ratio I use? I usually shoot at 5:3 (1920x1152). Do you think that shooting with 16:9 ratio could improve S/Noise performance?

I used to be scared of high iso as well until I learned that it's just a matter of letting in enough light. I regularly shoot at night for a weekly outdoor social event and I took a chance and rented a T/1.5 cine lens (same principle as f stops, different measurement) and expanded my ISO to 12800 to get a 0 exposure (but have since starting 'exposing to the right' with the ettr module). I get no noise in my post processing but I am of the unpopular opinion that noise is inherit in the cDNG workflow.
5D Mark II
10 bit 12 bit Experimental Build

adrjork

Quote from: KirbyLikes525 on August 06, 2019, 04:38:08 PM
[...] but I am of the unpopular opinion that noise is inherit in the cDNG workflow.
Possible? Is there a technical reason?

Kharak

There is no difference in noise between shooting 16x9 or 3x2. You are shooting raw, the ratio only determines how much of the sensor raw data you want.

If you are shooting at 6400 iso and everything is noisy, you simply have too little light.. add lights or find a new location.

And for lowlight, i would never ever shoot h264 instead of raw.. well basically always but you want as much information as possible for temporal noise reduction, 14 bit raw denoises at another level compared to h264, where 95% of the data has been discarded.

I recommend Neat Video, it is simply the best.
once you go raw you never go back

Levas

For less noise, you need more light (like many already have said here).
So I would definitely recommend your own suggestion of faking the shot, start shooting as soon the street lamps are on.
If you want the sky in the background to become black anyway(in post), shoot with the sun, or lighter part of the sky in your back so your subject reflects more light to your sensor.

I assume you are gonna setup your tripod at the opposite site of the street, so for more light, also look for a street with street lights on both sides of the street and set up your tripod directly under one of the street lights for more light on your subject. Or choose a street with (brightly lit) shops on one side, and set up your tripod at the side of the road with shops and shoot the other side of the street, this way, you are using the lights of the shops in the back to get more light.

Levas

Also wondering, what noise are we talking about, is it mostly the grainy noise, or do you have vertical lines/banding noise, or do you have static, bright coloured pixels across the frame ?

adrjork

Thanks a lot for your reply Levas.

Quote from: Levas on August 06, 2019, 07:51:05 PM
Also wondering, what noise are we talking about, is it mostly the grainy noise, or do you have vertical lines/banding noise, or do you have static, bright coloured pixels across the frame ?
Mainly grainy noise, but I noticed also vertical lines!

Quote from: Levas on August 06, 2019, 07:45:35 PM
So I would definitely recommend your own suggestion of faking the shot, start shooting as soon the street lamps are on.
Yes, that was my first solution. But what about the second? (Compositing of sidewalk-timelapse + woman-normally-walking?) In this way only woman will be noisy. But what worries me is the fact that camera position is near ground and I see woman "through" a lot of out-of-focus grass and stuff, so I don't know if I will be able to make a realistic comp in post...

P.S. What about Override in your opinion? Any advantage in keeping it active at the same exact FPS (25) of Canon menu?

Quote from: Kharak on August 06, 2019, 06:57:08 PM
There is no difference in noise between shooting 16x9 or 3x2. You are shooting raw, the ratio only determines how much of the sensor raw data you want.
Is this true also between "normal" 16:9 and crop-zoom 5X? I mean, I can suppose that in 5X crop mode the amount of light is reduced, then I should have more noise... or not?

Levas

Quote from: adrjork on August 06, 2019, 08:26:42 PM
Yes, that was my first solution. But what about the second? (Compositing of sidewalk-timelapse + woman-normally-walking?) In this way only woman will be noisy. But what worries me is the fact that camera position is near ground and I see woman "through" a lot of out-of-focus grass and stuff, so I don't know if I will be able to make a realistic comp in post...
Sounds like a lot of work, especially when there is lots of out of focus stuff in the foreground.

Quote from: adrjork on August 06, 2019, 08:26:42 PM
P.S. What about Override in your opinion? Any advantage in keeping it active at the same exact FPS (25) of Canon menu?
Doesn't have any effect on noise, so no advantage or disadvantage.

Quote from: adrjork on August 06, 2019, 08:26:42 PM
Is this true also between "normal" 16:9 and crop-zoom 5X? I mean, I can suppose that in 5X crop mode the amount of light is reduced, then I should have more noise... or not?
Yup, 5x zoom gives more noise, because effectively using smaller portion of the sensor, so indeed less light to gather.

I think it could be done without using composite tricks, but it mostly depends on your post processing skills I guess.
What software are you using for post process, or what is your workflow ?

Another thing or 'trick' to consider, cut and lift the shadows, so all noisy dark parts of the frame become a noise free grey color.
See this example, shot on 6d, ISO 6400, 1/33th shutter time, F2.0. Already got rid of the chroma noise in this example, but still lots of (line) noise to see:


Now with cut and lift, see curve and see how I changed it (left at the bottom of the curve)


Look, no more noise  ;)  also all detail in the shadows is lost  :P

Kharak

Instead of mucking about with comp+rotoscoping+6400 iso noise, i say buy a pair of cheap LED construction lights, add diffuse filters and gain +2 stops of light for your scene. Also buy some tiny LED flashlights and place them behind objects in the scene with some diffuse on and you can "fake" more light in the scene.

3x crop will have different noise than 3x3, possibly more hot pixels and different fpn as its 1:1 ratio of the sensor, but I would not necesearily say it has less light. The light hitting the sensor still hits the entire sensor, you are just cropping out a part of it.

i believe that 3x3 readout has a cleaner output because of the interpolation, but slightly unsharper.

And stick to fps override and shutter fine tuning, always.

Edit: also look in to mlv app's Darkframe subtraction, it does wonders when you go >1600 iso
once you go raw you never go back

KirbyLikes525

Quote from: adrjork on August 06, 2019, 04:58:21 PM
Possible? Is there a technical reason?

With cDNG, there's no raw processing just transcoding (going from one file format to another). But if you transcode to a lossy format like h264 or ProRess then you can "bake in" raw processing such as exposure, white balance*, and specifically the color space or color profile. The color profile tells the raw what the range of color values to use and the noise is colors that are out of gamut (range). Rec.709 is the standardized color space/profile for HD Video including the H264 codec (.mp4/.mov). If you encode a value that is not within the Rec.709 range it will decode as noise when viewed. So when you go to cDNG, if there's values out of range of 709 it can't be corrected hence the noise when editing and exporting video.
*=manual white balance gets applied to raw in camera, auto white balance does not [5D Mark II]

There is 1 con to this, but I look at it as a trade-off. You have to go from raw to lossy to set the color profile meaning you're going to lose quality in the sense that you're working with less colors (14bit=4billion colors down to 10bit=1billion colors) but still more than the 16k colors in 8bit H264 which will be your final output. This is the workflow of every digital television show and movie you've ever watched - it's an industry standard workflow.

That's as objective as I can put it.

Now, subjectively, I have little to no noise when I process my raw video with Rec.709, adjusting the exposure (to 0 or less), uncheck "use camera matrix" or select "Don't Use Camera Matrix", export as ProRes 422 and resize the export to my desired dimensions, if needed. You can white balance the raw first if you want, I choose not to because I do all of my coloring in Adobe with the ProRes. I stopped having issues with noise with this workflow as long as I had a proper exposure.

This video was shot at 1600 and 12800 ISO with a Rokinon Cine DS 35mm. I pushed 1 shot past 0 and got some noise, can you tell which one?
https://youtu.be/gRVj5_OIsso
5D Mark II
10 bit 12 bit Experimental Build

adrjork

Thank you all for your very good advices!

Quote from: Levas on August 06, 2019, 09:13:41 PM
Sounds like a lot of work, especially when there is lots of out of focus stuff in the foreground.
Well, perhaps, instead of masking only the lady (a lot of work indeed) I could mask the entire "corridor" of her walk (...perhaps...) and in this way avoiding all the transparency issues of the out-of-focus grass and stuff in front of me... We will see...

Quote from: Levas on August 06, 2019, 09:13:41 PM
What software are you using for post process, or what is your workflow ?
Switch (for MLV to cDNG) + Davinci. That's it.

Quote from: Levas on August 06, 2019, 09:13:41 PM
Another thing or 'trick' to consider, cut and lift the shadows, so all noisy dark parts of the frame become a noise free grey color.
Damn good!!! Thanks a lot for this trick!

Quote from: Kharak on August 06, 2019, 09:26:16 PM
i say buy a pair of cheap LED construction lights, add diffuse filters and gain +2 stops of light for your scene. Also buy some tiny LED flashlights and place them behind objects in the scene with some diffuse on and you can "fake" more light in the scene.
Eh... Not bad indeed! I'll try surely!

Quote from: Kharak on August 06, 2019, 09:26:16 PM
i believe that 3x3 readout has a cleaner output because of the interpolation, but slightly unsharper.
Then... I'm an idiot (damn me): all my shots are 5:3 (1920 X 1150) ONLY to have some reframing room in post... So, if I correctly understand, 1920X1080 gives me less noise thanks to the interpolation? (Honestly I tried this evening to shoot a semi-dark room both in 16:9 and 5:3, and the result was actually pretty similar...)

Quote from: Kharak on August 06, 2019, 09:26:16 PM
And stick to fps override and shutter fine tuning, always.
My English is very poor... "Stick to" means "don't touch"? (Again, I must be an idiot...) I've always keep FPS override activated, for all my shots... But always keeping the same fps ("exact") value of the Canon menu (25). Is it possible that I've made my shots worse somehow?

Quote from: Kharak on August 06, 2019, 09:26:16 PM
Edit: also look in to mlv app's Darkframe subtraction, it does wonders when you go >1600 iso
Never used MLV App and I don't know what is darkframe subtraction... But Inoticed "darkframe average automation" in Switch.
Q: Is it the same thing? How does it work?

Quote from: KirbyLikes525 on August 06, 2019, 09:46:01 PM
The color profile tells the raw what the range of color values to use and the noise is colors that are out of gamut
So, if I correctly understand, I could anyway take my shots in RAW and then converting them in Prores only when I find  some out-of-gamut artifacts (like too much noise). Or this operation must be done in-camera?

Again, thank you all a lot!

adrjork

A detail: in Canon menu I deactivated all the Noise Reductions. I did so in order to apply Denoise only in post (to control it "manually").
Q: Do you think it's better to let the camera doing this automatically? And if so, which values?

Thanks

Walter Schulz

And here we go again:

High ISO noise reduction does not affect RAW
Long exposure noise reduction is applied to RAW, too but only if exposure time is 1 sec or longer.

Pretty well covered and not even new. You may want to use a search engine of your choice.

adrjork

Thanks Walter, and sorry for my last dumb question  :-\
Quote from: Walter Schulz on August 07, 2019, 06:33:49 AM
Long exposure noise reduction is applied to RAW, too but only if exposure time is 1 sec or longer.
And if my exposure is 1 sec or longer, is it advisable in-camera NR or better doing it in post?

adrjork

Sorry but... I've always used MLV Lite 1.1 instead of MLV 2.0, and today I tested MLV 2.0 for the first time. Now, same shutter speed, same aperture, same everything, then MLVFS and then Davinci... is it normal that MLV Lite file is brighter (minimum 1 stop) than MLV 2.0 file? ??? Possible?
Is it an "interpretation" of MLVFS? Or really is the file?

Levas

There can't be any difference in exposure. (Quality should be the same)
The MLV file contains all sorts of data info like which lens was used, iso setting, exposure time etc.
Some software use more of this data than others and (pre)adjust image to that.

Could it be that you shoot in 12 bit lossless raw option ? Instead of standard 14 bit ?
Some software do correct exposure adjustments for the magic lantern lower bit options like 10 and 12 bit raw.

adrjork

Thanks for your reply, Levas.
Quote from: Levas on August 09, 2019, 11:02:56 AM
Could it be that you shoot in 12 bit lossless raw option ? Instead of standard 14 bit ?
No, MLV App Info says 14-bit for both, and exactly same ISO/Aperture/etc. for both, Levels for both are 2047 black, 16200 white.
Perhaps stupid question: why does black is not 0 and white is not 16383? (Should I set it manually? Or there is a meaning in those numbers?)