Magic Lantern Forum

Developing Magic Lantern => General Development Discussion => Topic started by: dlrpgmsvc on May 22, 2013, 12:33:18 PM

Title: [DONE] 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 22, 2013, 12:33:18 PM
Lets concentrate here all our efforts to port raw zebras & C. to 550d !

Current state-of-the-art : all the menus are enabled for raw visualizations of zebras & C., and now they all works correctly, thanks to "xaint" and his invaluable work !
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: Greg on May 22, 2013, 01:14:33 PM
The 500D works raw histogram when live view.

(http://img28.imageshack.us/img28/5006/vram0.png)
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 22, 2013, 02:23:40 PM
On 550d, RAW canon mode, M mode on main dial, luma histogram, ETTR hints ON, use RAW histogram off :

(https://dl.dropboxusercontent.com/u/46130896/no_raw.BMP)


On 550d, RAW canon mode, M mode on main dial, luma histogram, ETTR hints ON, use RAW histogram ON :

(https://dl.dropboxusercontent.com/u/46130896/yes_raw.BMP)

Like you can see... no differences and no ETTR hints displayed...

Same scene, same camera parameters, same illumination, same lens

For Greg: make a comparison like mine... and report, please ! Thanks !
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 22, 2013, 02:41:14 PM
dlrpgmsvc: I think it only works in RGB mode. Am I right?
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: 1% on May 22, 2013, 02:50:40 PM
Raw histo and all that jazz works on 600D and EOS-M too.

Check states in photo mode.
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 22, 2013, 07:08:37 PM
dlrpgmsvc: I think it only works in RGB mode. Am I right?

I don't know, so I made a test, and it's the same: no difference ! Look here :

NO RAW :
(https://dl.dropboxusercontent.com/u/46130896/noraw_rgb.BMP)

YES RAW:
(https://dl.dropboxusercontent.com/u/46130896/yesraw_rgb.BMP)

So, what is the clue here ? Am I not able to use this function correctly or am I not in the correct conditions to run this test (photo mode M and RAW from canon menu and main dial) ?

And where should the ETTR hints appear ? I see none...
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 22, 2013, 07:11:47 PM
Raw histo and all that jazz works on 600D and EOS-M too.

Check states in photo mode.

Many thanks 1% ! I updated the first status message of this thread including these models !
What means "check states in photo mode" ? Sorry for my question, but if you can explain I can do this check, many thanks 1%  !!!!
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 22, 2013, 11:17:41 PM
Following the porting hints stated in the beginning of this thread, I can see that all the modifications needed are already there for the 550d, and the #ifdef's about 550d are all there with their parameters. So I think someone putted them with knowledge. Or not ? My dubts are: do it should work for 550d also ? Or they are guess values yet to be fine-tuned ? Or do am I doing something wrong into using raw histogram and the other raw visualizations aids (you can see my tests above to check) ? By dissipating these doubts, you will help me into pointing in the right direction in order to solve this problem, many thanks !  ;)
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 23, 2013, 03:53:26 PM
Check states in photo mode.

Do you mean the file "state-object.c" inside "src" directory ?
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 23, 2013, 05:08:44 PM
Whoa ! A little step for a programmer... a giant leap for 550D Magic Lantern !

I managed to enable the RAW HISTOGRAM on 550D !  ;D

The pull request is yet sent and sitting, waiting to be merged in the next nightly build !  :P

(https://dl.dropboxusercontent.com/u/46130896/raw_histogram.BMP)
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: 1% on May 23, 2013, 05:14:47 PM
Quote
Do you mean the file "state-object.c" inside "src" directory ?
Yep.
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: zenlenz on May 23, 2013, 05:49:07 PM
I managed to enable the RAW HISTOGRAM on 550D !  ;D

The pull request is yet sent and sitting, waiting to be merged in the next nightly build !  :P


Excellent stuff, dlrpgmsvc  ;D

I look forward to seeing this implemented on the 550D. Many thanks!

Alas, I'm no coder myself. But I have been looking at the DNG bursts, with some sample shots today and the results are amazing, thanks for your work on that too. I hope I'm not sounding too greedy, to wonder how/if you have progressed on the RAW video capabilities as yet? It's kinda ironic that the poll reveals sooo many users of the 550D, yet it seems to be the last to get code ported!
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 23, 2013, 05:51:26 PM
One question: why, if raw histogram enabled, the live view shows raw histo and then the quick review show the normal histo ? Do this is normal ? Do we must enable some option to have raw histo ALSO on the quick review ? Many thanks in advance for your replies !
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 23, 2013, 06:00:40 PM
I hope I'm not sounding too greedy, to wonder how/if you have progressed on the RAW video capabilities as yet? It's kinda ironic that the poll reveals sooo many users of the 550D, yet it seems to be the last to get code ported!

Unfortunately not, I have not progressed for raw video  :'(
I'm "making my bones" on more simple things like this, just to get aquainted with the system. Then, if no one will port the raw video, I will pass to it... even if I hope someone will port it before waiting for me  ;)

Yes, you are right: it's so ironic and sad your consideration about how many people have a 550D and it's Always the last to be ported... but perhaps all the users in the poll are simply users and not programmers... so ... I'm trying to be their "Robin Hood" coder ! I have surely more followers than for 5DMK3 and company ! This is for sure !  ::)
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: a1ex on May 23, 2013, 06:14:27 PM
For photo histograms, you need to enable CONFIG_RAW_PHOTO in internals.h (look at 5D2/5D3). Then, you need to intercept the raw image buffer from Canon's state object (this is the task that develops the image). The state machine is just like 5D2 one.

You will also have to guess the photo EDMAC. They are only 32 of them (maybe less). It's likely to be the same as on 5D2.

Then, find the offsets until you get raw zebras aligned correctly (1% knows a program that autodetects those, I did them by trial and error). Be careful with bayer grid alignment (top-left pixel should be red, otherwise you have to skip one line and/or column).
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 23, 2013, 11:52:59 PM
Then, you need to intercept the raw image buffer from Canon's state object (this is the task that develops the image). The state machine is just like 5D2 one.

SDS_FRONT3_STATE ? It seems it is this ! I tried to duplicate the 5D2 codes snippets, but nothing... the review histogram is still not raw, but normal... I used 52D as reference like you said
And where I have to put the raw photo EDMAC ? In which file ? Raw.c ? It is already there the 550d edmac for raw photo...
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 24, 2013, 07:10:04 PM
These are the modifications I have done: what is wrong or what is missing ? Many thanks to all that would help !  ;)

https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: sutobe on May 25, 2013, 12:51:38 AM
Great work with this!

I only have a question (what the ETTL hints do should be obvious), but I can't really figure out how to read / interpret the RAW histogramm.

So as far as I can think of, it shows the actual exposure values of the RAW, not of what the camera thinks it would be in a JPEG, right?

And what does the brownish-red and grey area in the left quarter of the histogramm mean?

Thanks,
Tobi


Whoa ! A little step for a programmer... a giant leap for 550D Magic Lantern !

I managed to enable the RAW HISTOGRAM on 550D !  ;D

The pull request is yet sent and sitting, waiting to be merged in the next nightly build !  :P

(https://dl.dropboxusercontent.com/u/46130896/raw_histogram.BMP)
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 25, 2013, 11:16:54 AM
So as far as I can think of, it shows the actual exposure values of the RAW, not of what the camera thinks it would be in a JPEG, right?

And what does the brownish-red and grey area in the left quarter of the histogramm mean?

Thanks,
Tobi

Yes, your first thinking is correct.

For the red area, I think it means: higly underexposed area, and the grey means: warning, this area is moderately underexposed. However i dunno for sure, so I wait for the programmer that implemented  this to confirm my toughts... anyone that knows this, is welcome to knock here ! Many thanks !

What I know for sure is that each vertical line correspond to a 1 Ev value, and this is from the sources comments, so it should be sure !  ;)
Title: Re: 550d raw zebras, histogram, spotmeter & C. port official thread
Post by: sutobe on May 26, 2013, 12:00:28 AM
I forgot to ask about the vertical lines haha. thanks for clarifying this.

@anyone who knows

Still'd like to know what the colored areas on the left mean, thanks ;-)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 26, 2013, 08:38:14 PM
One question: why, if raw histogram enabled, the live view shows raw histo and then the quick review show the normal histo ? Do this is normal ? Do we must enable some option to have raw histo ALSO on the quick review ? Many thanks in advance for your replies !

First thing: sorry for my horrible english :)

I think,i have successfully enabled raw histogram in quick review mode.
Here are the changes i've made: (dlrpgmsvc please check! :D)

in raw.c i've add:

Code: [Select]
#if defined(CONFIG_550D)
#define RAW_PHOTO_EDMAC  0xc0f04208
#endif

same as 60D.


Then inside the raw_update_params() function
inside the else if (QR_MODE) branch... (for me it's ~309th lines)
right below this:
Code: [Select]
raw_info.buffer = (void*) raw_buffer_photo;
added:
Code: [Select]
#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif
 

again: same as 60D

Then below these, where we need to define the Raw buffer size for photos ( below the long commented thing :) )
i've added:

Code: [Select]
#if defined(CONFIG_550D)
width = 5344;
height = 3516;
skip_left = 142;
skip_right = 0;
skip_top = 50;
#endif

same as 60D i guess, but i'm not sure, I'll look into it now...


Thats it, dlrpgmsvc please check if it's working to you or not...
If so, then you can change the "Current state-of-the-art:" :)

With these modifications i can see the raw histogram in Quick Review mode:

LiveView:
(http://xaint.hu/LV.jpg)

Quick Review:
(http://xaint.hu/QR.jpg)

It's working without LiveView too.
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 26, 2013, 09:22:14 PM
@Xaint : Great ! Now I put on your modifications and I will check it throughly ! Then I will report here my confirmations ! Many thanks Xaint !  :D
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 26, 2013, 10:10:43 PM
@Xaint: You went so close to the solution but... we have only a little problem when we reach some overexposures in some channels: the liveview raw histogram, in this case, differ from the quick review histogram, but not on the graph, but on the indications, and here are the differences:

[1] The ETTR hint is different
[2] The background red area extension is different
[3] The overexposure circles are different (well... in the quickreview there aren't at all)

See this for reference :

LIVEVIEW
(https://dl.dropboxusercontent.com/u/46130896/liveview1.BMP)

QUICKREVIEW
(https://dl.dropboxusercontent.com/u/46130896/quickreview1.BMP)

We are so close ! Compliments anyway ! Do you have an idea on where to make modifications for this little problem ? I'm searching also... the first who arrive at a solution, knocks here !  ;)  Many thanks again !!  ;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 26, 2013, 10:21:52 PM
First we need to find even more correct values for this:

Code: [Select]
#if defined(CONFIG_550D)
width = 5344;
height = 3516;
skip_left = 142;
skip_right = 0;
skip_top = 50;
#endif

because as i said it was just a guess :)
btw i'm on it :)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 27, 2013, 12:38:42 AM
I found that this code you added :

#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif

Is not needed at all... it don't solve our problem, however we clean out the code a bit...
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 27, 2013, 01:03:25 PM
OK, if i comment out this:
Code: [Select]
#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif

as you mention, then we don't have raw histogram in Quick Review mode outside LiveView.
I've enabled RAW_DEBUG_DUMP, so ML saves a DNG raw file directly from the RAW image buffer.
Now if we comment out the above, and we are in LiveView, it saves a ~2Mpx DNG, i guess it's come from RAW_LV_EDMAC,
and compute the QuickReview RAW Histogram from that ~2Mpx RAW. So we have QuickReview RAW histogram if we came from LV.
But when you just take a pic, without turning on LiveView, then for me RAW_DEBUG_DUMP reports the error: Photo raw buffer null!

So as i guess if we have this:
Code: [Select]
#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif

then it doesn't matter where we are: in LiveView mode, or not in LiveView, RAW_DEBUG_DUMP saves a 18Mpx DNG from both 2 cases,
and we have the RAW Histogram in Quick Review mode.

I hope I make myself clear :D :D
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 27, 2013, 02:08:21 PM
then it doesn't matter where we are: in LiveView mode, or not in LiveView, RAW_DEBUG_DUMP saves a 18Mpx DNG from both 2 cases,
and we have the RAW Histogram in Quick Review mode.

I hope I make myself clear :D :D

Whoa ! My fault ! I was so concentrated by using only live view, that I didn't considered the case when we don't use liveview ! Great, so we will keep this piece of code, it's vital ! Many thanks !

Also, have you managed to follow the hints in the link I posted on the first post of this thread ? I lost myself because I'm not so aquainted with Linux o/s ... so I cannot manage to launch all that mess commands in order to guess the values we need... did you succeeded in some degree ?  ;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: 3pointedit on May 27, 2013, 02:56:56 PM
Sorry, does this 18MP dump mean we can have a shutterless image capture in RAW? At full res!
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 27, 2013, 03:04:27 PM
Sorry, does this 18MP dump mean we can have a shutterless image capture in RAW? At full res!

No, without shutter it's only ~2Mpx.
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 27, 2013, 11:16:22 PM
Also, have you managed to follow the hints in the link I posted on the first post of this thread ? I lost myself because I'm not so aquainted with Linux o/s ... so I cannot manage to launch all that mess commands in order to guess the values we need... did you succeeded in some degree ?  ;)

Huh. It will be long!!!  ;D

Yes after running img.py script as a1ex mentioned here:
https://bitbucket.org/hudson/magic-lantern/commits/51cdc8c568c1854183c6b0f4b49eec17da57850a (https://bitbucket.org/hudson/magic-lantern/commits/51cdc8c568c1854183c6b0f4b49eec17da57850a)

i've got: 5344, same as it was (like 60D) :)

However I modified it a bit, so now it looks like this:

Code: [Select]
#if defined(CONFIG_550D)
width = 5344;
height = 3516;
skip_left = 142;
skip_right = 18;
skip_top = 58;
skip_bottom = 10;
#endif

But after I read this again:

For photo histograms, you need to enable CONFIG_RAW_PHOTO in internals.h (look at 5D2/5D3). Then, you need to intercept the raw image buffer from Canon's state object (this is the task that develops the image). The state machine is just like 5D2 one.

You will also have to guess the photo EDMAC. They are only 32 of them (maybe less). It's likely to be the same as on 5D2.

Then, find the offsets until you get raw zebras aligned correctly (1% knows a program that autodetects those, I did them by trial and error). Be careful with bayer grid alignment (top-left pixel should be red, otherwise you have to skip one line and/or column).

I revert those things what i said earlier here: :)

First thing: sorry for my horrible english :)

I think,i have successfully enabled raw histogram in quick review mode.
Here are the changes i've made: (dlrpgmsvc please check! :D)

in raw.c i've add:
...

So forget it!

Then I tried to do what alex said, and as I saw here:
https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2#chg-src/state-object.c (https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2#chg-src/state-object.c)
you tried too, without succes?..

First  i've enabled CONFIG_RAW_PHOTO in 550D.109/internals.h:

Code: [Select]
/** We have access to raw data in photo mode too :) */
#define CONFIG_RAW_PHOTO

then in top of  state-object.c i've added:

Code: [Select]
#define SDS_FRONT3_STATE (*(struct state_object **)0x3840)
between the #ifdef CONFIG_550D and #endif lines, so it looks like this:

Code: [Select]
#ifdef CONFIG_550D
#define DISPLAY_STATE DISPLAY_STATEOBJ
#define INPUT_ENABLE_IMAGE_PHYSICAL_SCREEN_PARAMETER 19
#define MOVREC_STATE (*(struct state_object **)0x5B34)
#define LV_STATE (*(struct state_object **)0x4B74)
#define LVCAE_STATE (*(struct state_object **)0x51E4)
#define SDS_FRONT3_STATE (*(struct state_object **)0x3840)   <<<<<<<<<<<<<<<<<<<<<<<<<<<<
#endif


Quote
Then, you need to intercept the raw image buffer from Canon's state object

So i've added this:

Code: [Select]

#ifdef CONFIG_550D
if (old_state == 0 && input == 3 && new_state == 1)
     raw_buffer_intercept_from_stateobj();
#endif

into the static int stateobj_sdsf3_spy() function, so now it looks something like this:

Code: [Select]
#ifdef SDS_FRONT3_STATE
static int stateobj_sdsf3_spy(struct state_object * self, int x, int input, int z, int t)
{
    int old_state = self->current_state;
    int ans = StateTransition(self, x, input, z, t);
    int new_state = self->current_state;

    #ifdef CONFIG_5D2
    // SDSf3:(0)  --  3 sdsMem1toRAWcompress-->(1)
    // SDSf3:(1)  --  3 sdsMem1toJpegDevelop-->(1)
    if (old_state == 0 && input == 3 && new_state == 1)
        raw_buffer_intercept_from_stateobj();
    #endif

    #ifdef CONFIG_550D                                  <<<<<<<<<<<<<<<<<<<<<<<<<<<<
    if (old_state == 0 && input == 3 && new_state == 1) <<<<<<<<<<<<<<<<<<<<<<<<<<<<
        raw_buffer_intercept_from_stateobj();           <<<<<<<<<<<<<<<<<<<<<<<<<<<<
    #endif                                              <<<<<<<<<<<<<<<<<<<<<<<<<<<<

    return ans;
}
#endif


But we are not done here.. :)

In raw.c still need to change the RAW_PHOTO_EDMAC address from 0xc0f04A08 to 0xc0f04208,
so I would suggest to modify these lines:

Code: [Select]
#if defined(CONFIG_5D2) || defined(CONFIG_500D) || defined (CONFIG_550D) || defined(CONFIG_600D) || defined(CONFIG_650D) || defined(CONFIG_EOSM) // remove this: defined(CONFIG_550D)
#define RAW_PHOTO_EDMAC 0xc0f04A08
#endif

#if defined(CONFIG_5D3) || defined(CONFIG_6D)
#define RAW_PHOTO_EDMAC 0xc0f04808
#endif

#if defined(CONFIG_60D)
#define RAW_PHOTO_EDMAC 0xc0f04208
#endif

to these:

Code: [Select]
#if defined(CONFIG_5D2) || defined(CONFIG_500D)|| defined(CONFIG_600D) || defined(CONFIG_650D) || defined(CONFIG_EOSM)
#define RAW_PHOTO_EDMAC 0xc0f04A08
#endif

#if defined(CONFIG_5D3) || defined(CONFIG_6D)
#define RAW_PHOTO_EDMAC 0xc0f04808
#endif

#if defined(CONFIG_60D)
#define RAW_PHOTO_EDMAC 0xc0f04208
#endif

#if defined(CONFIG_550D)           <<<<<<<<<<<<<<<<<<<<<<<<<<<<
#define RAW_PHOTO_EDMAC 0xc0f04208 <<<<<<<<<<<<<<<<<<<<<<<<<<<<
#endif                             <<<<<<<<<<<<<<<<<<<<<<<<<<<<



Then inside the int raw_update_params() function,
inside the else if (QR_MODE) branch... (for me it's ~309th lines)

delete this:

Code: [Select]
#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif

as you first mentioned here:
I found that this code you added :

#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif

Is not needed at all... it don't solve our problem, however we clean out the code a bit...


So now:

[1] The ETTR hint is different
[2] The background red area extension is different


maybe still consist, but let me say that (some stupid thoughts???):

if you examine the histogram in LV, then (i guess) it's come from RAW_LV_EDMAC (in 550D ~2Mpx)

and when you viewing it in QuickReview (through LV after the shot) it's come from RAW_PHOTO_EDMAC, which is differs (maybe :D)

Maybe stupid thoughts again:

Code: [Select]
#if defined(CONFIG_550D)
width = 5344;
height = 3516;
skip_left = 142;
skip_right = 18;
skip_top = 58;
skip_bottom = 10;
#endif

From a1ex:
Code: [Select]
Be careful with bayer grid alignment (top-left pixel should be red, otherwise you have to skip one line and/or column).
In our cases, the image from the RAW_PHOTO_EDMAC DNG is just a little bit differs from the real RAW file, a very little color difference, and a little misalignment: 2 pixel to the left and 3 pixel down!



[3] The overexposure circles are different (well... in the quickreview there aren't at all)

I have overexposure circles in quick review, and Auto ETTR is working with and without LiveView too!
(Tomorrow morning i will go and do a couple more test shots)

From a1ex:
Quote
Then, find the offsets until you get raw zebras aligned correctly

We need  a1ex, who give us some more tips :) :) :)

For now it looks like this:

LV:
(http://xaint.hu/ML2/LV.jpg)

Quick Review:
(http://xaint.hu/ML2/QR.jpg)

... i think Zebras are not aligned properly

But a1ex will come and corrects things out!

(And again: I hope I make myself clear :D :D)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 03:15:56 PM
Yes, I have yet done your modifications some days ago, but the problem is still into the parameters "skip...and so on" I think. You have found the right modifications I already done. Now I inserted your NEW parameters of skippage and dimensions you have lastly found, but I still don't see the overexposure circles in qr that I see in LV, and the ETTR hint data is different from liveview to Quickreview. The problem may be :

[1] The dimensions and skip parameters
[2] The raw photo EDMAC address
[3] The value in sds 3 front state, like discussed here in my latest modifications: https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2
[4] Something else (?)

What do you think ? And: are you sure the skip and dimensions parameters are ok ? You said it's off about 2 pixels on one side and three pixels on other side, so: why you don't correct the skip or dimensions parameters accordingly ? Or is this not correctable ? It's just a curiosity, I know this don't solve our problem so much...
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: sutobe on May 28, 2013, 05:18:51 PM
I somehow can't figure out how to interpret the hints.

when it says i.e. 1.5 up and I adjust the EV ~1.5 upwards it says "Over" most of the time but the picture is really dark.

Trying in my living room while pointin the cam outside, 550d with 10-22mm Canon lens.

I thought the hints were vor overexposing in such a range you can still crank the lights down to get more informations out of a RAW?

The results I get are the other way round, picture needs to be darkened down according to the Hints.

Using M mode.
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 06:58:49 PM
I somehow can't figure out how to interpret the hints.

when it says i.e. 1.5 up and I adjust the EV ~1.5 upwards it says "Over" most of the time but the picture is really dark.

Trying in my living room while pointin the cam outside, 550d with 10-22mm Canon lens.

I thought the hints were vor overexposing in such a range you can still crank the lights down to get more informations out of a RAW?

The results I get are the other way round, picture needs to be darkened down according to the Hints.

Using M mode.

Do you base your ETTR hint readings on Live View screen or on Quick Review screen ?
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: sutobe on May 28, 2013, 08:01:37 PM
I put them on LV.

Maybe I am just not getting it right, I am no programmer or pro-photographer. I understand the concept of ETTR but can't figure out how to properly use the histgramm for some reason.

sorry for hijacking this thread but I couldnt find one specificly for the 550d
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 28, 2013, 08:34:58 PM
Yes, I have yet done your modifications some days ago, but the problem is still into the parameters "skip...and so on" I think. You have found the right modifications I already done. Now I inserted your NEW parameters of skippage and dimensions you have lastly found, but I still don't see the overexposure circles in qr that I see in LV, and the ETTR hint data is different from liveview to Quickreview. The problem may be :

[1] The dimensions and skip parameters
[2] The raw photo EDMAC address
[3] The value in sds 3 front state, like discussed here in my latest modifications: https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2
[4] Something else (?)

What do you think ? And: are you sure the skip and dimensions parameters are ok ? You said it's off about 2 pixels on one side and three pixels on other side, so: why you don't correct the skip or dimensions parameters accordingly ? Or is this not correctable ? It's just a curiosity, I know this don't solve our problem so much...


Dunno...

I did a quick test, here's the results:

Histogram on LiveView:
(http://xaint.hu/ML3/LVHISTO.jpg)

Histogram on Quick Review (LV):
(http://xaint.hu/ML3/QRHISTOFROMLV.jpg)

Then Quick Review histogram in Photo Mode (non LiveView):
(http://xaint.hu/ML3/QRINPHOTOMODE.jpg)


Now about overexposure circles:

LiveView:
(http://xaint.hu/ML3/OVEREXPO_CIRCLES.jpg)

Quick Review:
(http://xaint.hu/ML3/OVEREXPO_CIRCLES_QR.jpg)

So I have overexposure circles everywhere (LV, LV->QR, PHOTO_MODE->QR)...


And Auto ETTR is working pretty good i think:

Settings before Auto ETTR:
(http://xaint.hu/ML3/SETTINGS_BEFORE_AUTO_ETTR.jpg)

And after Auto ETTR:
(http://xaint.hu/ML3/SETTINGS_AFTER_AUTO_ETTR.jpg)

Image after Auto ETTR:
(http://xaint.hu/ML3/ETTRD_SHOT_QR.jpg)

I don't say it is perfect, but works.
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 08:52:20 PM
Ouch !  :o Here I'm missing something very hard to understand for me ! Why I'm missing the warning circles in the QR mode ?!?  :o
Can you send me your BIN, please ? Just to check ! Thanks !  :o

I checked and double-checked: I get perfectly identical histogram values in lv and qr, BUT in qr there are no warning circles !  :o

If you send your bin, then, if the problem persists, it's a matter of values and parameters inside ml or canon menus in my camera. If, otherwise, the problem is gone, then it's a difference in our sources !  ::)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 28, 2013, 09:04:17 PM
Ouch !  :o Here I'm missing something very hard to understand for me ! Why I'm missing the warning circles in the QR mode ?!?  :o
Can you send me your BIN, please ? Just to check ! Thanks !  :o

I checked and double-checked: I get perfectly identical histogram values in lv and qr, BUT in qr there are no warning circles !  :o

If you send your bin, then, if the problem persists, it's a matter of values and parameters inside ml or canon menus in my camera. If, otherwise, the problem is gone, then it's a difference in our sources !  ::)


Sure! Here you go: xaint.hu/autoexec.bin (http://xaint.hu/autoexec.bin)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 09:18:36 PM

Sure! Here you go: xaint.hu/autoexec.bin (http://xaint.hu/autoexec.bin)

Oh crap ! Your bin also it doesn't work !   :o
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 09:34:42 PM
try to go to the wrench menu and then -restore ml defaults- then shut down and restart your 550d so we have same settings, then see if warn circles appear in quick review ... in my 550d not!  :o
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 28, 2013, 09:39:15 PM
try to go to the wrench menu and then -restore ml defaults- then shut down and restart your 550d so we have same settings, then see if warn circles appear in quick review ... in my 500d not!  :o

You try it on 500d? or typo?  500D 550D -not the same

EDIT: anyway: it's working after -restore ml defaults too
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 09:41:24 PM
You try it on 500d? or typo?  500D 550D -not the same

Typo ! I mean 550d ! sorry !!!  ;D  corrected now...  ::)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 09:45:44 PM
I also tried to delete all canon settings to their defaults, and also defaulting all the canon fn(c) functions, but the problem persists ! Try also you, to double check if perhaps you have some settings inside canon parameters (normal or fn(c)) that make the warn circles working on qr mode: if by resetting them you come to my results, it's some canon parameters, otherwise the mistery goes on !  :o
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 28, 2013, 09:52:43 PM
OK, every settings in its default mode, and after I change the picture quality back to RAW, everything working again :)

Mystery!
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 28, 2013, 09:54:49 PM
So you say only the overexposure circles are missing?
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 10:07:27 PM
Yes ! But... there are news ! juggling with the settings, i discovered that by rising the ISO setting from 100 to 200, circles appears in qr ! , but are lower values than lv ! Then going to 400 iso, warn circles values are in some way equals, even if not very much. Then going to 800 iso and up, the warn values in qr are way way more Greater than in lv !!!! Do we discovered the problem ? ...  :o

Can you make also this test ?!? So we can arrive to a consolidated raw histogram for 550d !

However, it's still incredible on how your values (lv and qr) are so almost perfectly equal !!! And mine not !   :o
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 28, 2013, 11:15:20 PM
OK, i think it's depend on the subject/scene (dark room, long exposure...) What you can see in LiveView on the display, I think it's just simulated (Exp.sim). Let's say you set the shutter to 6 seconds, then what you see on the display it's not the real picture :) so i think maybe here's the difference?

oh i can not express myself... i know a lot of words, but i don't know grammar :D:D:D
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 11:20:13 PM
Yes I understand perfectly what you are saying ! And I agree... but... why you obtain so perfect match between lv and qr, and me not ? Have you made the ISO switching tests to see if you maintain the perfect match between lv and qr in all iso values ?
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 28, 2013, 11:38:06 PM
Yes, i've made the test, the values are almost the same. The histograms update value is 1 sec, so the hand shaking maybe a factor i don't know. :) Please create some sample pictures at daytime and use a tripod, then post the results here :)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 28, 2013, 11:52:37 PM
Ok, may thanks !

For the moment I made a pull request to add raw histogram on lv and qr modes, I think it's stable now, thanks to your valuable help !

The next problem is that in review mode (not quick) the histogram is normal and not raw, do it's so also for you ?
If so, this is our next problem to solve now !  ;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 29, 2013, 12:10:13 AM
Yes, I think there is no raw histogram in review mode. Right?
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 29, 2013, 12:19:40 AM
Right, there is only normal histogram if, while reviewing the image, you press the "Q" button, can you verify if we are aligned on this ?  ;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 29, 2013, 12:58:31 AM
I'm sorry I meant to say I think that the raw histogram works only in LV and QR . On every model (5D2, 5D3 , 550D and so on)..

(As I guess :P) While reviewing the image, ML creates the histogram from the image buffer. Which is a jpeg only (in preview mode), because canons software loads only the jpeg preview from the raw file...
But I don't want to say stupid things... It's just my thinking :D
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 29, 2013, 09:13:48 AM
Yes ! All you said is correct ! It is too work to load up the raw and make a histogram from it...
However, where Canon finds the jpg associated to the raw file, to load into preview mode, if in the Canon menu I select RAW (only) and not RAW+jpg ?
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: a1ex on May 29, 2013, 09:42:30 AM
dcraw -e IMG_1234.CR2

;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 29, 2013, 12:40:29 PM
Yep, it is a pre-rendered jpeg thumbnail, built into the CR2 raw file itself. And dcraw -e IMG_1234.CR2  will show it.
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 29, 2013, 01:25:12 PM
Great ! So good news : these modifications for raw histo on lv and qr mode on 550D was now merged and will be available on the next nightly build for all of you 550D lovers and proud owners ! Many thanks and credits to XAINT and his invaluable help and time !!  8)

Now lets verify raw zebras and raw spotmeter !  ;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 29, 2013, 03:15:28 PM
Ok, raw SPOTMETER seems to work great on 550D.
EV raw indicator is mean value of RGB values, when 0EV it means that center spot is oversaturated on the max, and then lesser values, like -1.5EV, -4.0EV and so  on, indicates how overexposed the center spot is. Waiting for confirmations or anomalies, if any found. Many thanks !  ;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 29, 2013, 05:57:11 PM
Also RAW Zebras (working only in QR mode) seems to work ok (the zebras area is less extended in RAW representation than in normal LV zebras, like they have to be) and they are perfectly aligned with the image, at least with my extensive tests I done. If someone have different results or problems, please chime here !
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 29, 2013, 07:38:42 PM
@Xaint : you said, some posts ago, that zebras seems not to be aligned. Can you confirm this or do they are ok ? Many thanks !  ;)
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 30, 2013, 11:49:03 AM
@Xaint : you said, some posts ago, that zebras seems not to be aligned. Can you confirm this or do they are ok ? Many thanks !  ;)

Zebras are ok!!
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: dlrpgmsvc on May 30, 2013, 02:03:57 PM
Great, xaint ! Many thanks !

I can confirm the 550D ports about the title of this thread are now merged in the nightly build !

So... if no more problems arise about this, I will wait a little and then I will mark this thread with the prefix [DONE] !  ;)

Always special thanks to "xaint" and his successful efforts !  ;D
Title: Re: 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 30, 2013, 05:16:39 PM
The glory is not mine, but all the developers.
I just wanted to help the porting, for those who uses the same body.

And dlrpgmsvc: Your efforts are even more valuable than mine, so thank YOU!
Title: Re: [DONE] 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: shawnwytch on May 31, 2013, 09:02:09 PM
how are you guys converting these files? so u can see them in premiere or resolve
Title: Re: [DONE] 550D raw zebras, histogram, spotmeter & C. port official thread
Post by: xaint on May 31, 2013, 10:11:07 PM
how are you guys converting these files? so u can see them in premiere or resolve

Which files?