Magic Lantern Forum

Using Magic Lantern => General Help Q&A => Topic started by: garry23 on January 10, 2017, 06:54:36 PM

Title: FRSP
Post by: garry23 on January 10, 2017, 06:54:36 PM
Forgive this potentially dumb question.

When saving FRSP in individual DNGs or DNGs in an MLV folder, I thought the ISO and Tv were saved now.

But it looks like I'm wrong.

Or am I right?

BTW I'm using the EOSM latest experimental build.
Title: Re: FRSP
Post by: koenkooi on January 10, 2017, 07:10:18 PM
I don't think the 'dng' branch has been merged yet, that has the needed code for saving that info.
Title: Re: FRSP
Post by: garry23 on January 10, 2017, 07:16:50 PM
:( and a photo trip coming up where this would really help me for timelapses :(
Title: Re: FRSP
Post by: g3gg0 on January 10, 2017, 08:10:29 PM
as a workaround you could use MLV which should save metadata.
of course, please doublecheck before.
Title: Re: FRSP
Post by: garry23 on January 10, 2017, 08:35:42 PM
@g3gg0

Sadly it appears to be only a 50% solution on the EOSM.

The ISO seems to be captured.

The Tv seems to always be 1s.

I'm using a manual lens (Rokinon 14mm) and strangely the aperture, rather than reporting the 'usual' F/00, reports F/8.

Strange stuff: at least for me.

Cheers

Garry
Title: Re: FRSP
Post by: dfort on January 10, 2017, 11:42:53 PM
Quote from: koenkooi on January 10, 2017, 07:10:18 PM
I don't think the 'dng' branch has been merged yet, that has the needed code for saving that info.

I've been testing that branch for a while. The DNG module (https://bitbucket.org/hudson/magic-lantern/pull-requests/603/dng-module/diff) pull request has been open for a couple of years and there is an even older pull request for a DNG module (https://bitbucket.org/hudson/magic-lantern/pull-requests/456/move-the-dng-library-to-a-self-contained/diff) that is also still open.

I'd like to add EXIF tags to DNG files for lens information for the Assign lens focal length and name for non cpu lenses (https://www.magiclantern.fm/forum/index.php?topic=18083.0) project though it might exceed the scope of the manual_lens_info (https://bitbucket.org/hudson/magic-lantern/branch/manual_lens_info) branch. There is no lens information saved at all in the main (unified) branch on DNG's even for cpu lenses--and yeah, it is working in the ml_dng branch.

@garry23 -- Where did you get a build of the ml_dng branch? Have you set up a Magic Lantern development environment?

It might be a good idea to create a feature request (https://www.magiclantern.fm/forum/index.php?board=24.0) for this and also post the issues you discovered to the DNG module pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/603/dng-module/diff).

BTW--What are you looking for in the Tv information? Could you post an example from a CR2 file showing how it is supposed to look? I usually set the camera either full auto+ mode or all manual, depending on my state of mind while I'm shooting.
Title: Re: FRSP
Post by: garry23 on January 11, 2017, 12:01:22 AM
Dan

I appreciate a 'full scope' solution may be a challenge.

For those of us playing with timelapses, using FRSP, 'all' we need to capture is the ISO and the shutter time value. The FL and the aperture, which (usually) will not change in a timelapse, can be added in post, eg I use LensTagger in LR.

For holy grail, I use LRTimelapse, you need to have the ISO and shutter time data on a frame by frame basis, ie because it will change in steps, either through ETTR or manual intervention.

I'm afraid, at the moment, I just not able to put the time in to get a compile environment up and running. But I do want to do this: but after my forthcoming trip to Iceland  ;)

My 'big wish', and only on the EOSM at the moment (which I will use for timelapses on my trip) is to see the intervalometer working with FRSP (and ideally ETTR) and recording ISO and Tv data. That's all  :) ;)  ;D

On the EOSM I'm only using the latest nightly experimental build for the manual lens (but I'm not trying to use the manual lens feature, I'm just assuming this represents the 'best' build).

Cheers

Garry
Title: Re: FRSP
Post by: a1ex on January 11, 2017, 12:14:48 AM
The shutter speed is recorded in the MLV files in silent.dng, but it looks like there's a bug in current mlv_dump (exposure time doesn't get copied to DNG).

However, I do not recommend using the shutter metadata from the FRSP for deflicker purposes. Why? Because it's approximated as the total image capture time, so it's overestimated. Therefore, an image tagged as 2" will not be twice as bright as an image tagged as 1".

Instead, you may want to try a deflicker based on image contents. Both ML deflicker (https://www.magiclantern.fm/forum/index.php?topic=10496.0) module and this script (http://www.magiclantern.fm/forum/index.php?topic=8850.0) use robust statistics (median or some other percentile), so they don't need exposure metadata.
Title: Re: FRSP
Post by: garry23 on January 11, 2017, 12:39:04 AM
@A1ex

All good points and understood: thank you.

I thought the deflicker approach on stats wouldn't be able to handle holy grail steps? Or am I wrong?

Cheers

Garry
Title: Re: FRSP
Post by: a1ex on January 11, 2017, 01:06:55 AM
It should work just fine; even exposure jumps of 1-2 stops should not cause problems.

However, you need to be careful to avoid overexposure (maybe configure ETTR to underexpose a little more than usual).

Also, keep in mind that FRSP exposures are actually gradients (http://www.magiclantern.fm/forum/index.php?topic=12523.msg121962#msg121962), and changing exposures will also change the gradient ramp. The entire sensor starts capturing, waits for some delay, then the capture is stopped line by line, as the image is read out, causing a difference in exposure times on top and bottom lines. That difference remains constant.

I don't know how much this would impact a deflicker algorithm yet...

I can imagine a way to capture metadata that will give exact exposure time for each line (so one can undo the gradient), although it would be tedious and model-dependent. It would require logging the video timer registers while Canon code is performing the capture; during a long exposure, the sensor is reconfigured several times. Then, interpreting this data would probably give accurate exposure time for every single line from the image (I hope).

Or, the above things can be approximated from capture time (currently saved as metadata). This capture time includes:
- a dark frame (every time you take a still picture, Canon code takes a dark frame before, although it's not at full resolution; just a few hundred lines)
- delays from inter-task communications (semaphores, message queues); repeatability here is a few tens of milliseconds
- the actual capture time, until the last line is read out
- extra overheads (sensor setup, DMA, whatever else Canon code does)

All this extra stuff can probably be modeled as an overhead, to be subtracted from capture time (which is currently saved as metadata).

Not sure all of this is actually worth the trouble...
Title: Re: FRSP
Post by: dmilligan on January 11, 2017, 01:12:20 AM
MLVFS also has a deflicker option built in that is also based on the same robust statistics (depends on your raw processor honoring a specific tag, "ExposureOffset" IIRC), and I also ported the Bridge script to Lightroom: https://github.com/davidmilligan/LrDeflick
Title: Re: FRSP
Post by: mothaibaphoto on January 11, 2017, 06:14:48 AM
Hi, garry23. I do a lot of FRSP timelapses with  5D MarkII & MarkIII.
Your main concern is to get familiar with ETTR on your particular camera.
You need to spend a lot of time training to get correct exposure in different lighting conditions before your trip.
Once you captured correctly exposed timelapse, postprocess is rather simply:
Convert to DNG, process DualISO(if it was shot with DualISO).
Tweak first image to taste in ACR.
Sync settings.
Tweak last image WB and exposure.
Ramp multiple(dmilligan's bridge script) Temperature and Tint.
Deflicker(dmilligan's bridge script).
Thats enough.

Title: Re: FRSP
Post by: garry23 on January 11, 2017, 07:55:57 AM
@all

Thanks for all the useful feedback. I'll spend the time I have before my forthcoming trip 'experimenting and practicing'.

Once again, many thanks.

Cheers

Garry
Title: .xmp help
Post by: garry23 on January 19, 2017, 05:47:02 PM
I'm still trying to get a 'solution' for my FSRP timelapse capture.

I'm using the latest experimental Non-CPU lens info version.

From what I see, the FRSP DNG still does not capture the shutter or ISO (I'm using a 10-20mm Sigma with AF).

If this is the case, is there a way to force a .xmp file to be output with every image, ie capturing ISO and shutter?

I'm not so worried about FL or aperture, as I can add these back in later in post. But, as I'm doing holy grail timelapses, the shutter and ISO are varying all the time.

Hope someone can help.

Cheers

Garry
Title: Re: .xmp help
Post by: dmilligan on January 19, 2017, 06:03:26 PM
Simply use the MLV format instead of DNG, or this: https://bitbucket.org/hudson/magic-lantern/pull-requests/603
Title: Re: .xmp help
Post by: mothaibaphoto on January 19, 2017, 06:10:18 PM
You don't need shutter or ISO or even FRSP at all (you intend to shoot timelapses with EOSM, it's mirrorless, right?)
You do need: sturdy tripod, polariser, intervalometer, ETTR, DualISO (optional).
Title: Re: .xmp help
Post by: garry23 on January 19, 2017, 06:18:02 PM
@mothaibaphoto

mirrorless does not mean shutterless  ;) Plus I need the EXIF as I'm using LRTimelapse.

@dmilligan

As you pointed me there, I remembered I had the same problem before. ISO, aperture and FL all captured in the FRSP MLV, but, irrespective of shutter time, shutter always reports at 1.0s. :-(
Title: Re: .xmp help
Post by: mothaibaphoto on January 19, 2017, 06:37:05 PM
FRSP is very limiting in exposures. Shooting in bright light is a real pain.
You need ND filters and/or closed apertures. I've spent a lot of time healing sensor dust visible due to shot at F16.
Its much more complicated with timelapses than with ordinal shots: every frame may looks fine, but as long, as you start the movie, you see them popping...
With EOSM I don't see any reason to bother with FRSP for timelapses.
And why to use that LRTimelapse? Equalising image exposure based on EXIF data? Really? Is that could even work?
Title: Re: .xmp help
Post by: garry23 on January 19, 2017, 06:41:27 PM
@mothaibaphoto

I agree with you :-)

Especially as a 2nd hand EOSM is so cheap. I just picked one up for 58GBP with a 6000 shutter count.

I'm also trying to get the EOSM working with my Radian 2, ie triggering via the ML audio remote: but I need to wait for  3.5 to 2.5 mm cable.

Thanks for the feedback.

Cheers

Garry

PS the MLV 1.0s shutter reporting remain a mystery.
Title: Re: .xmp help
Post by: a1ex on January 19, 2017, 07:23:24 PM
Anything wrong with the advice from here (http://www.magiclantern.fm/forum/index.php?topic=18655.0)?

Quote from: garry23 on January 19, 2017, 06:41:27 PM
PS the MLV 1.0s shutter reporting remain a mystery.

Really? (https://bitbucket.org/hudson/magic-lantern/commits/29542681035266d060b01464a36c7e809311e08e)
Title: Re: FRSP
Post by: garry23 on January 19, 2017, 07:42:01 PM
@A1ex

Sorry, I'm really getting myself 'confused' over what is in a nightly and an experimental version.

I wait until some tells me what version to use  ;)

Cheers and, once again, sorry for being a pain
Title: Re: FRSP
Post by: Danne on January 19, 2017, 07:48:23 PM
garry23, are you on mac?
Coud you test this version?
http://www.magiclantern.fm/forum/index.php?topic=5601.msg178746#msg178746
Title: Re: FRSP
Post by: garry23 on January 19, 2017, 08:03:34 PM
@Danne

I'll test anything  ;)

But I'm on a PC, does that matter?

BTW I'm using ML_converter.exe
Title: Re: FRSP
Post by: Danne on January 19, 2017, 08:15:44 PM
Then this should work
https://builds.magiclantern.fm/utilities.html
Title: Re: FRSP
Post by: glubber on January 19, 2017, 08:25:52 PM
Hey garry,

I've done FRSP-Timelapses processed with LRTimelapse some months ago. Iirc It works by using the Visual workflow and changing some advanced settings. There is a option to do visual ramping instead of using Exif-data. Of course it takes longer than the basic workflow.
But as stated above the gradient of the FRSP is sometimes clearly visible, so i would recommend using Cr2 and autoettr for getting best results.
With exposure time limited to 1/10 it's hard to avoid blown highlights with sunny weather, even with ND-Filter.
Title: Re: FRSP
Post by: garry23 on January 19, 2017, 08:35:09 PM
All

Many thanks for you helpful and educational responses. I really enjoy 'talking' on this forum.

I've come to the same conclusion as many, i.e. I'll stick with .cr2s for my 'difficult', holy grail timelapses.

Once again, thanks to all of you  :)

Cheers

Garry