ETTR with full-res silent pictures

Started by barepixels, July 27, 2014, 03:56:56 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

barepixels

So I tried doing Holy Grails last night   :P  I had ETTR always on

Initially it seem to settle into right exposure  but it did not regulate and adjust exposure from previous shot.  So as the sky gets darker all of my frames stay at F22  :-\

I think I need to have global draw on and raw histogram on  correct?

Your thought?

Give me some suggestions and I will have another run at it later this evening
5D2 + nightly ML

a1ex


barepixels

thnx Alex.  I will go refresh my reading on ETTR again.  If you have any recommendation for camera settings, it would be greatly appreciated
5D2 + nightly ML

brapodam

Quote from: barepixels on July 27, 2014, 04:15:40 PM
thnx Alex.  I will go refresh my reading on ETTR again.  If you have any recommendation for camera settings, it would be greatly appreciated
I guess if you're doing day to night or night to day, just shoot wide open and ETTR should take care of the ISO and shutter speed. I'm not sure about white balance though. I don't do time lapses, but I do occasionally experiment with it when there are developments in silent pics

josepvm

Quote from: barepixels on July 27, 2014, 03:56:56 PM
Give me some suggestions and I will have another run at it later this evening

If you want to shoot a sunset, or transition from day to night, start choosing and aperture and ISO that gives you, measuring with ETTR, an exposure time of 1/10 sec when starting the timelapse.  1/10 sec is the shortest exposure possible for silent pics, I think.

When the scene starts to get darker, ETTR will increase the exposure time.

My procedure is: with intervalometer and silent pics disabled, I go to liveview, and set Auto ETTR on, with trigger mode set as "Half Shutter Double click",  (or as "Press SET", if you prefer it)
Then I measure the exposure manually triggering ETTR with a half shutter double click, I read the exposure time,  and change the aperture value or ISO and trigger ETTR again until I get !/10 or !/8 sec.

Then I change ETTR triger mode to "Always on", set Silent Pics On, and at last, adjust the intevalometer to the desired values, setting the intervalometer "Start trigger" mode as "Leave menu". When done, a half shutter press starts the timelapse.

 

barepixels

Well look like ETTR desided not to work last night.  I am pretty sure I had ETTR always on....  I just check, I haven't touch the camera settings since last night, AutoETTR is still being sellected in ML menu along with exposure override



210 frames latter  EXIF never change


I threw the rest out cause it just too dark to be useable.

Here is the weird thing.  Am pretty sure I started at 1/15 seconds for the first pic  and

in autoETTR i have
trigger mode: Always On
slowest shutter 16"
hightlight 1%
mid SNR 6ev
shadow SNR 2 EV
link to dual ISO off


PS:  I really wish I can use ML ISO 25 in shoot mode and without pink highlight.  I wouldn't have to be F23 to start

PS PS: @Josep, you can do faster then 1/10 with FullRes-SP.  But as you go faster you will darker gradient at top of the frame.

PS PS PS:  I did a successful flicker free ETTR Holy Grails two weeks ago (not FullRes-SP)



Note to self:  keep an eye on the live view every once in a while to make sure ETTR is working.
5D2 + nightly ML


josepvm

I have been experimenting with fullres silent timelapses with ETTR during day to night transition, and today also with the advanced intervalometer, a very powerful and flexible tool, indeed.

And now I have an idea to suggest: would be possible to add to the advanced intervalometer the ability to do a ramping with ETTR's target exposure value?

I explain the reasons to suggest it. ETTR does a very good job adjusting the exposure to changing light conditions. So good, that all the sunset darkening effect is lost. I need to reintroduce it in postprocessing. I have done it successfuly aplying with Darktable a gradual reduction in exposure across the whole DNG frame sequence. With a exposure reduction of 4 EV from the begining to the end of the timelapse, I get a quite natural effect. It's laborious, doing it manually, but possible. And I suppose this could be automated with a LUA script.

But I wonder if this progressive darkening could be done in camera, ramping the target exposure value in ETTR module. This, apart from reducing postprocessing work, I think could have two other advantages:

- reducing unnecessary long exposures in the last part of the timelapse.
- allowing a longer sequence of exposures, with a better use of the available range of exposure times (that is a narrower range with silent pics)

One could say that I can use a fixed exposure to get this effect. But I think this is not a natural effect, because our eye adapts naturally to changing light, and camera does not. A fixed exposure leads to very high differences in lightness through the sequence. But I think the 3.5 EV range that could be achieved adjusting ETTR target exposure could do a nice effect.

Well, that's my idea. Any opinions?

barepixels

What happen when you try Fullres silent and ETTR MLV alone?  I could not get it to work on my 5D2 build.  It keeps heading to 1/8000 speed.  Am hoping it me/user error cause I really want it to works.

As for advanced intervalometer, what settings did you use.  I will try it tonight.
5D2 + nightly ML

josepvm

Quote from: barepixels on July 29, 2014, 09:48:08 PM
What happen when you try Fullres silent and ETTR MLV alone?  I could not get it to work on my 5D2 build.  It keeps heading to 1/8000 speed.  Am hoping it me/user error cause I really want it to works.

You need to start with an aperture and ISO setting that allows you to get an exposure time lower than 1/10 sec. With daylight, even at sunset, you could need a strong neutral density filter to achieve it. I use a 8x ND Hoya filter for sunsets and a 64x ND during the day.

If the exposure time needed is shorter than 1/10 sec, ETTR will read every exposure done and see that it is too bright, and then ETTR will try to set a shorter time exposure value. And with silent pics this will not have any effect, because a shorter exposure is not possible. An then, for the next exposure, ETTR will try to set an even shorter value, an so on, until you get the shortest camera value (1/4000) and at this point ETTR will fail

Quote from: barepixels on July 29, 2014, 09:48:08 PM
As for advanced intervalometer, what settings did you use.  I will try it tonight.

Advanced intervalometer is useful here to do an aperture value ramping, Start with a very narrow aperture to keep exposure time below 1/10 sec, and as the scene gets darker, open the diaphragm more.

It's not easy for me to explain in few words how to use it. Go to the "readme" link in the advanced intervalometer thread. You need to create a first keyframe (with "Loop after" value =1) with the camera adjusted to the initial aperture value (very narrow aperture) and then create more keyframes with wider aperture values.

dmilligan

Quote from: josepvm on July 29, 2014, 09:25:28 PM
But I wonder if this progressive darkening could be done in camera, ramping the target exposure value in ETTR module. This, apart from reducing postprocessing work, I think could have two other advantages:

- reducing unnecessary long exposures in the last part of the timelapse.
- allowing a longer sequence of exposures, with a better use of the available range of exposure times (that is a narrower range with silent pics)

You are basically asking for noisier photos.

Ramping exposure target down in camera would not give you any advantage in "reducing unnecessary long exposures" over simply setting some slowest shutter speed in the ETTR menu.  In fact, doing this with 'slowest shutter speed' setting is going to provide superior results because it's going to use higher ISOs to compensate, and they have less noise (per electron). Besides, you've already set some interval in the intervalometer, what does it matter if your exposures take up more of that interval? Your timelapse is still going to take the same amount of total time.

In general ML does not try to do things that can easily be done in post. Doing so is a waste of time and effort, if there's some bit of code that can be written for ML to do some legwork for post production, then that same code could simply be written to run on a computer and be used in post, and it would be much easier to write => we have nice documented APIs, high level languages, debuggers, etc. unlike ML development.

There might be some other valid reasons for needing to ramp the exposure target, but I don't see this as one => it can easily be done and post, and you're losing image quality in the process.

josepvm

Ok, thanks dmilligan, I understand your points.

I will invest time in finding a way to automate this exposure ramping in post with Darktable.

I said "unnecessary long exposures" refering those frames that the camera tries to lighten increasing exposure time and I know I will darken again ... but no problem. It's true that is a lot worse to have to lighten in post frames underexposed in camera, you only get tons of noise.

Limiting the exposure time in ETTR module and allowing it to increase ISO does not give a darker picture, as far I understand. But setting the exposure target to -2EV , -3EV or -4EV yes, you get a darker image. But I can do this more acurately in postprocessing, true.

And to optimize the limited range of shutter speeds in silent pics, I will play more with aperture value ramping.

Thanks, dmilligan, and the other developers, for the amazing work you are doing.

barepixels

Josep  I don't know what software avail on Linux but would this work for ramping?

but if you have after effects, maybe

top layer normal  clip                    100% opacity ................................................................................... 0% opacity
bottom layer  darken clip               0% opacity    .................................................................................. 100% opacity
5D2 + nightly ML

barepixels

So I tried Fullres silent and ETTR MLV again tonight  LOL  am determine to get it right

My auto ETTR setting were

Trigger: Always On
Slowest shutter: 16"
Highlight ignore: 0.1%
Midtone SNR:  6EV
Shadow SNR:  2EV
Link 2 Dual ISO: ON <-------------- oops

I darken my variable ND and got  ISO: 100  F2.8  1/10 second.  Set my interval for 20 seconds

And this is what I get.  Every images is 2 stop under.  I raise each to 2 stop brighter in Lightroom and you can see the after below

1) here you see it was shot at 1/2 second  although i initial set it at 1/10


2) then few minutes later ETTR raised Tv to 1 second exposure.  ETTR never raised it again even though I have slowest set for 16 seconds in auto ETTR


3) here you can see ETTR raise ISO  from 100  to 200


4) then it settle at ISO 1600


5) and remain at 1600 to end


Am I wrong for expecting ETTR to incremental raise shutter to 16 seconds before raising ISO to next stop.
Or is it because I screwed up and have link to Dual ISO on (dual iso module were never loaded)

Is ISO 1600 the max value for ETTR

the end result video is noisy and a lot of flickering.  oh yeah time stamp is off by couple hours ... prob because I don't have latest update:

EXIF shutter speed should now be the time of FA_CreateTestImage (the time you see displayed on the LV screen), not the value set in the Canon menu (which is wrong). This value may be slightly longer than the true exposure time, but it's more correct than Canon menu shutter speed.

5D2 + nightly ML

a1ex

Just tried on 5D2, but could not reproduce the bug - I've got 2.5, 3, 4 ... 16 second exposures, and then it started to increase ISO.

The rest are expected behavior.

- underexposure: your images have large areas of highlights that you can safely clip (much larger than 0.1%)
- furthermore, the sky color is not essential here, so you can rely on aggressive highlight recovery - configure ETTR so it knows that (allow clipping on the green channel, for example)
- flicker: you need to use a deflicker algorithm (either the deflick module, or dmilligan's deflicker script)
- ISO: if you can prove there's a better choice than ISO 1600 on 5D2 for that really dark scene, I'll change ETTR to use that instead.
- dual ISO link has no effect if dual ISO is turned off

barepixels

I will try again tomorrow and report back

Am still a nOOb with ML.  Can you give me some suggestion to set for auto ETTL

As for ISO 1600, I didn't know the cap so that is why I ask.  I wasn't challenging you on the decision.

5D2 + nightly ML

josepvm

Quote from: barepixels on July 30, 2014, 04:56:17 AM
Josep  I don't know what software avail on Linux but would this work for ramping?


I use Darktable as my main photographic workflow app. And I adjust the DNG's from timelapses with Darktable, too. It's fast and accurate (32 bit float color depth), non-destructive processing, and with it is easy to apply an adjustement to a large group of images.

But Darktable does not use adjustment layers, in the way Photoshop does. Instead it applies a pipeline of cascaded modules, and for every module you can define an opacity value (to adjust effects strenght) and a mask (drawn or parametric or both) with different possible blending modes,

I have done the ramping simply adjusting the exposure, because it is easy to quantify and to apply gradually through the whole sequence of images. If I want to aply a 4 EV darkening from the beginning to the end of a timelapse composed by 170 images, then I can lower the exposure by 0.1EV to the frames 5 to 8, by 0.2 to frames 9 to 12, etc.

And this way I think it wouldn't be difficult to automate, using a LUA script inside Darktable, or a bash script to do the same through command line commands with dcraw or ufraw.

josepvm

Quote from: barepixels on July 30, 2014, 09:14:46 AM
So I tried Fullres silent and ETTR MLV again tonight  LOL  am determine to get it right

My auto ETTR setting were

Trigger: Always On
Slowest shutter: 16"
Highlight ignore: 0.1%
Midtone SNR:  6EV
Shadow SNR:  2EV
Link 2 Dual ISO: ON <-------------- oops

I darken my variable ND and got  ISO: 100  F2.8  1/10 second.  Set my interval for 20 seconds

And this is what I get.  Every images is 2 stop under.  I raise each to 2 stop brighter in Lightroom and you can see the after below

2) then few minutes later ETTR raised Tv to 1 second exposure.  ETTR never raised it again even though I have slowest set for 16 seconds in auto

Am I wrong for expecting ETTR to incremental raise shutter to 16 seconds before raising ISO to next stop.
Or is it because I screwed up and have link to Dual ISO on (dual iso module were never loaded)


I do not understand this behaviour. For me, ETTR efectively changes the exposure time, without touching the ISO value.

A dumb question: are you using the camera in Manual mode ?

And a midtone SNR of 6EV is quite high. When not using DualISO, in a high dynamic range escene, as the bright sky against a post-sunset landscape, ETTR with this SNR setting will clip highligths in the sky, that is the most interesting part of the picture.

So try to lower the midtone SNR to 4EV.


barepixels

awesome.  and yes I was in manual.  i think my meta is wrong becuase i do not have the latest update.  will have to wait till next week for nightly build

what would you suggest for highlight and shadow

Highlight ignore: ????? %
Midtone SNR:  4EV
Shadow SNR:  ????? EV
5D2 + nightly ML

josepvm

Quote from: barepixels on July 30, 2014, 10:46:47 AM
i think my meta is wrong becuase i do not have the latest update.  will have to wait till next week for nightly build

The "silent.mo" using "SeekSkipFile" that dmilligan linked to solve problems with > 2GB files already includes the changes for saving in meta more accurate shutter values.

And a "Highlight ignore" = 0.1% and "Shadow SNR" = 2EV are Ok.

a1ex

@josepvm:

Quote from: barepixels on July 30, 2014, 09:14:46 AM
Every images is 2 stop under.  I raise each to 2 stop brighter in Lightroom ...
...
the end result video is noisy

=> the SNR settings are alright, he needs to allow ETTR to overexpose a little more.

If there is noise in the daylight exposures, also consider dual ISO. To reduce noise at night, you need to capture more light (remove the ND filter, try a wider aperture), and/or darken the image in post.

josepvm

Quote from: a1ex on July 30, 2014, 11:00:33 AM
@josepvm:

=> the SNR settings are alright, he needs to allow ETTR to overexpose a little more.

If there is noise in the daylight exposures, also consider dual ISO. To reduce noise at night, you need to capture more light (remove the ND filter, try a wider aperture), and/or darken the image in post.

Yes, I didn't notice he was looking for brighter images. In this case, Midtone SNR of 6 EV is fine.

barepixels

Shoud I put Highlight ignor at 0.5%  or 5%  <-- am throwing these number because am not experience
5D2 + nightly ML

a1ex


josepvm

Quote from: barepixels on July 30, 2014, 11:09:31 AM
Shoud I put Highlight ignor at 0.5%  or 5%  <-- am throwing these number because am not experience

Increasing the "highligt ignore" value could be useful if you have very bright zones in the image: lightbulbs, etc. This way this spots are burned but you can get a higher exposure for the remaining scene.

barepixels

Just want to report that when I press the shutter deflicker module create a sidecard.  But if I use the intervalometer it doesn't.  Am assuming for now, it treats multiple frames MLV as movie.  Maybe in the future we can have XMP for Full-size SP MLV mode?



Good news is I think I have achieved Holy Grail with Full-res SP-MLV.  Am exporting them now as 16bit TIFF to deflick in Adobe Bridge.... Stay tune...

For Alex, you are right, the meta is wrong... As I observe shutter was going from 1/5 to 1/6 gradually etc ...  but meta says most frames were 1/2 seconds

EDIT:  here is the video from last night

Ran deflicker in Abode Bridge like 20x  then ran it in GBDeflicker for about 6 more times.  Still have flickers.  Will try again tonight. Out of 286 frames  one were all black.
5D2 + nightly ML

josepvm

Deflicker data is not saved for silent pics if you use the intervalometer, either with MLV format or with individual DNGs.

For now, to have a deflickered timelapse I need to stay with the traditional method, wearing my shutter for every frame  :(

barepixels

Tried another Holy Grail tonight but it crash.  Manage to get 177 frames.  This is in the log

ASSERT: GetMemoryAddressOfMemoryChunk( GetFirstMemChunk( pMem1AllocateListItem->hMemSuite ) ) == pMessage->pAddress
at SrmActionMemory.c:984, task RscMgr
lv:0 mode:3

EDIT:  It crashed as soon as ISO jump to 800



Frame 176 didn't get to be extracted.  Prob cause if the crash
5D2 + nightly ML

a1ex

Can you find a way to reproduce?

Currently there is a race condition (like 0.1 seconds or so) when switching to review mode - during that time, Canon code unlocks the buttons, so if you press the shutter halfway in that moment, it will return to LiveView and crash. But this one seems to be something else, because you were running on intervalometer...

barepixels

I tired again and everything works great.  I even did not get a single black frame.  So, sadly, I wasn't able to repeat the error.  BTW I went back and redit previous video with my new work flow.  LRTimelapse and dmilligan amazing Adobe Bridge Script

5D2 + nightly ML

88oak88

hello simple a question how many pictures in a row I could firing mode silent and dual image without damaging the camera sensor. if this question has already been made still could not find it because the forum this increasing. and thank you to the developers without the contribution of you our videos and our photographs estriam a little poorer. earned great job

translate for google.

ola uma pergunta simple quantas fotografias seguidas eu conseguiria atirando com modo imagem silenciosa e dual, sem prejudicar o sensor de camera. caso essa pergunta ja tenha sido feita ainda não consegui encontra-la visto que o forum esta cada vez maior. e muito obrigado aos desenvolvedores sem a contribuição de vocês nossos videos e  nossas fotografias estriam um pouco mais pobres. valeu grande trabalho

budafilms

El sensor no se degrada.
Se degrada e mecanismo que sube y baja el espejo.
Con lo cual, Silente Picture, protege ese mecanismo al no ser utilizado.

Brigado.