[DONE] 550D raw zebras, histogram, spotmeter & C. port official thread

Started by dlrpgmsvc, May 22, 2013, 12:33:18 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

xaint

OK, if i comment out this:

#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif


as you mention, then we don't have raw histogram in Quick Review mode outside LiveView.
I've enabled RAW_DEBUG_DUMP, so ML saves a DNG raw file directly from the RAW image buffer.
Now if we comment out the above, and we are in LiveView, it saves a ~2Mpx DNG, i guess it's come from RAW_LV_EDMAC,
and compute the QuickReview RAW Histogram from that ~2Mpx RAW. So we have QuickReview RAW histogram if we came from LV.
But when you just take a pic, without turning on LiveView, then for me RAW_DEBUG_DUMP reports the error: Photo raw buffer null!

So as i guess if we have this:

#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif


then it doesn't matter where we are: in LiveView mode, or not in LiveView, RAW_DEBUG_DUMP saves a 18Mpx DNG from both 2 cases,
and we have the RAW Histogram in Quick Review mode.

I hope I make myself clear :D :D

dlrpgmsvc

Quote from: xaint on May 27, 2013, 01:03:25 PM
then it doesn't matter where we are: in LiveView mode, or not in LiveView, RAW_DEBUG_DUMP saves a 18Mpx DNG from both 2 cases,
and we have the RAW Histogram in Quick Review mode.

I hope I make myself clear :D :D

Whoa ! My fault ! I was so concentrated by using only live view, that I didn't considered the case when we don't use liveview ! Great, so we will keep this piece of code, it's vital ! Many thanks !

Also, have you managed to follow the hints in the link I posted on the first post of this thread ? I lost myself because I'm not so aquainted with Linux o/s ... so I cannot manage to launch all that mess commands in order to guess the values we need... did you succeeded in some degree ?  ;)
If you think it's impossible, you have lost beforehand

3pointedit

Sorry, does this 18MP dump mean we can have a shutterless image capture in RAW? At full res!
550D on ML-roids

xaint

Quote from: 3pointedit on May 27, 2013, 02:56:56 PM
Sorry, does this 18MP dump mean we can have a shutterless image capture in RAW? At full res!

No, without shutter it's only ~2Mpx.

xaint

Quote from: dlrpgmsvc on May 27, 2013, 02:08:21 PM
Also, have you managed to follow the hints in the link I posted on the first post of this thread ? I lost myself because I'm not so aquainted with Linux o/s ... so I cannot manage to launch all that mess commands in order to guess the values we need... did you succeeded in some degree ?  ;)

Huh. It will be long!!!  ;D

Yes after running img.py script as a1ex mentioned here:
https://bitbucket.org/hudson/magic-lantern/commits/51cdc8c568c1854183c6b0f4b49eec17da57850a

i've got: 5344, same as it was (like 60D) :)

However I modified it a bit, so now it looks like this:


#if defined(CONFIG_550D)
width = 5344;
height = 3516;
skip_left = 142;
skip_right = 18;
skip_top = 58;
skip_bottom = 10;
#endif


But after I read this again:

Quote from: a1ex on May 23, 2013, 06:14:27 PM
For photo histograms, you need to enable CONFIG_RAW_PHOTO in internals.h (look at 5D2/5D3). Then, you need to intercept the raw image buffer from Canon's state object (this is the task that develops the image). The state machine is just like 5D2 one.

You will also have to guess the photo EDMAC. They are only 32 of them (maybe less). It's likely to be the same as on 5D2.

Then, find the offsets until you get raw zebras aligned correctly (1% knows a program that autodetects those, I did them by trial and error). Be careful with bayer grid alignment (top-left pixel should be red, otherwise you have to skip one line and/or column).

I revert those things what i said earlier here: :)

Quote from: xaint on May 26, 2013, 08:38:14 PM
First thing: sorry for my horrible english :)

I think,i have successfully enabled raw histogram in quick review mode.
Here are the changes i've made: (dlrpgmsvc please check! :D)

in raw.c i've add:
...

So forget it!

Then I tried to do what alex said, and as I saw here:
https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2#chg-src/state-object.c
you tried too, without succes?..

First  i've enabled CONFIG_RAW_PHOTO in 550D.109/internals.h:

/** We have access to raw data in photo mode too :) */
#define CONFIG_RAW_PHOTO


then in top of  state-object.c i've added:

#define SDS_FRONT3_STATE (*(struct state_object **)0x3840)

between the #ifdef CONFIG_550D and #endif lines, so it looks like this:


#ifdef CONFIG_550D
#define DISPLAY_STATE DISPLAY_STATEOBJ
#define INPUT_ENABLE_IMAGE_PHYSICAL_SCREEN_PARAMETER 19
#define MOVREC_STATE (*(struct state_object **)0x5B34)
#define LV_STATE (*(struct state_object **)0x4B74)
#define LVCAE_STATE (*(struct state_object **)0x51E4)
#define SDS_FRONT3_STATE (*(struct state_object **)0x3840)   <<<<<<<<<<<<<<<<<<<<<<<<<<<<
#endif



QuoteThen, you need to intercept the raw image buffer from Canon's state object

So i've added this:


#ifdef CONFIG_550D
if (old_state == 0 && input == 3 && new_state == 1)
     raw_buffer_intercept_from_stateobj();
#endif


into the static int stateobj_sdsf3_spy() function, so now it looks something like this:


#ifdef SDS_FRONT3_STATE
static int stateobj_sdsf3_spy(struct state_object * self, int x, int input, int z, int t)
{
    int old_state = self->current_state;
    int ans = StateTransition(self, x, input, z, t);
    int new_state = self->current_state;

    #ifdef CONFIG_5D2
    // SDSf3:(0)  --  3 sdsMem1toRAWcompress-->(1)
    // SDSf3:(1)  --  3 sdsMem1toJpegDevelop-->(1)
    if (old_state == 0 && input == 3 && new_state == 1)
        raw_buffer_intercept_from_stateobj();
    #endif

    #ifdef CONFIG_550D                                  <<<<<<<<<<<<<<<<<<<<<<<<<<<<
    if (old_state == 0 && input == 3 && new_state == 1) <<<<<<<<<<<<<<<<<<<<<<<<<<<<
        raw_buffer_intercept_from_stateobj();           <<<<<<<<<<<<<<<<<<<<<<<<<<<<
    #endif                                              <<<<<<<<<<<<<<<<<<<<<<<<<<<<

    return ans;
}
#endif



But we are not done here.. :)

In raw.c still need to change the RAW_PHOTO_EDMAC address from 0xc0f04A08 to 0xc0f04208,
so I would suggest to modify these lines:

#if defined(CONFIG_5D2) || defined(CONFIG_500D) || defined (CONFIG_550D) || defined(CONFIG_600D) || defined(CONFIG_650D) || defined(CONFIG_EOSM) // remove this: defined(CONFIG_550D)
#define RAW_PHOTO_EDMAC 0xc0f04A08
#endif

#if defined(CONFIG_5D3) || defined(CONFIG_6D)
#define RAW_PHOTO_EDMAC 0xc0f04808
#endif

#if defined(CONFIG_60D)
#define RAW_PHOTO_EDMAC 0xc0f04208
#endif


to these:


#if defined(CONFIG_5D2) || defined(CONFIG_500D)|| defined(CONFIG_600D) || defined(CONFIG_650D) || defined(CONFIG_EOSM)
#define RAW_PHOTO_EDMAC 0xc0f04A08
#endif

#if defined(CONFIG_5D3) || defined(CONFIG_6D)
#define RAW_PHOTO_EDMAC 0xc0f04808
#endif

#if defined(CONFIG_60D)
#define RAW_PHOTO_EDMAC 0xc0f04208
#endif

#if defined(CONFIG_550D)           <<<<<<<<<<<<<<<<<<<<<<<<<<<<
#define RAW_PHOTO_EDMAC 0xc0f04208 <<<<<<<<<<<<<<<<<<<<<<<<<<<<
#endif                             <<<<<<<<<<<<<<<<<<<<<<<<<<<<




Then inside the int raw_update_params() function,
inside the else if (QR_MODE) branch... (for me it's ~309th lines)

delete this:


#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif


as you first mentioned here:
Quote from: dlrpgmsvc on May 27, 2013, 12:38:42 AM
I found that this code you added :

#ifdef CONFIG_550D
raw_info.buffer = (void*) shamem_read(RAW_PHOTO_EDMAC);
#endif

Is not needed at all... it don't solve our problem, however we clean out the code a bit...


So now:

[1] The ETTR hint is different
[2] The background red area extension is different


maybe still consist, but let me say that (some stupid thoughts???):

if you examine the histogram in LV, then (i guess) it's come from RAW_LV_EDMAC (in 550D ~2Mpx)

and when you viewing it in QuickReview (through LV after the shot) it's come from RAW_PHOTO_EDMAC, which is differs (maybe :D)

Maybe stupid thoughts again:


#if defined(CONFIG_550D)
width = 5344;
height = 3516;
skip_left = 142;
skip_right = 18;
skip_top = 58;
skip_bottom = 10;
#endif


From a1ex:
Be careful with bayer grid alignment (top-left pixel should be red, otherwise you have to skip one line and/or column).

In our cases, the image from the RAW_PHOTO_EDMAC DNG is just a little bit differs from the real RAW file, a very little color difference, and a little misalignment: 2 pixel to the left and 3 pixel down!



[3] The overexposure circles are different (well... in the quickreview there aren't at all)

I have overexposure circles in quick review, and Auto ETTR is working with and without LiveView too!
(Tomorrow morning i will go and do a couple more test shots)

From a1ex:
QuoteThen, find the offsets until you get raw zebras aligned correctly

We need  a1ex, who give us some more tips :) :) :)

For now it looks like this:

LV:


Quick Review:


... i think Zebras are not aligned properly

But a1ex will come and corrects things out!

(And again: I hope I make myself clear :D :D)

dlrpgmsvc

Yes, I have yet done your modifications some days ago, but the problem is still into the parameters "skip...and so on" I think. You have found the right modifications I already done. Now I inserted your NEW parameters of skippage and dimensions you have lastly found, but I still don't see the overexposure circles in qr that I see in LV, and the ETTR hint data is different from liveview to Quickreview. The problem may be :

[1] The dimensions and skip parameters
[2] The raw photo EDMAC address
[3] The value in sds 3 front state, like discussed here in my latest modifications: https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2
[4] Something else (?)

What do you think ? And: are you sure the skip and dimensions parameters are ok ? You said it's off about 2 pixels on one side and three pixels on other side, so: why you don't correct the skip or dimensions parameters accordingly ? Or is this not correctable ? It's just a curiosity, I know this don't solve our problem so much...
If you think it's impossible, you have lost beforehand

sutobe

I somehow can't figure out how to interpret the hints.

when it says i.e. 1.5 up and I adjust the EV ~1.5 upwards it says "Over" most of the time but the picture is really dark.

Trying in my living room while pointin the cam outside, 550d with 10-22mm Canon lens.

I thought the hints were vor overexposing in such a range you can still crank the lights down to get more informations out of a RAW?

The results I get are the other way round, picture needs to be darkened down according to the Hints.

Using M mode.
Canon EOS 550D / Canon EF-S 10-22mm 1:3,5-4,5 USM / Canon EF-S 18-55 mm 3.5-5.6 IS / Canon EF-S 55-250mm 4,0-5,6 IS II / Canon EF 50mm 1:1.8 II / Canon Speedlite 430EX II

dlrpgmsvc

Quote from: sutobe on May 28, 2013, 05:18:51 PM
I somehow can't figure out how to interpret the hints.

when it says i.e. 1.5 up and I adjust the EV ~1.5 upwards it says "Over" most of the time but the picture is really dark.

Trying in my living room while pointin the cam outside, 550d with 10-22mm Canon lens.

I thought the hints were vor overexposing in such a range you can still crank the lights down to get more informations out of a RAW?

The results I get are the other way round, picture needs to be darkened down according to the Hints.

Using M mode.

Do you base your ETTR hint readings on Live View screen or on Quick Review screen ?
If you think it's impossible, you have lost beforehand

sutobe

I put them on LV.

Maybe I am just not getting it right, I am no programmer or pro-photographer. I understand the concept of ETTR but can't figure out how to properly use the histgramm for some reason.

sorry for hijacking this thread but I couldnt find one specificly for the 550d
Canon EOS 550D / Canon EF-S 10-22mm 1:3,5-4,5 USM / Canon EF-S 18-55 mm 3.5-5.6 IS / Canon EF-S 55-250mm 4,0-5,6 IS II / Canon EF 50mm 1:1.8 II / Canon Speedlite 430EX II

xaint

Quote from: dlrpgmsvc on May 28, 2013, 03:15:56 PM
Yes, I have yet done your modifications some days ago, but the problem is still into the parameters "skip...and so on" I think. You have found the right modifications I already done. Now I inserted your NEW parameters of skippage and dimensions you have lastly found, but I still don't see the overexposure circles in qr that I see in LV, and the ETTR hint data is different from liveview to Quickreview. The problem may be :

[1] The dimensions and skip parameters
[2] The raw photo EDMAC address
[3] The value in sds 3 front state, like discussed here in my latest modifications: https://bitbucket.org/dlrpgmsvc/magic-lantern/commits/3c7ce12b6ec26a796be0030d70aa837783e91ee2
[4] Something else (?)

What do you think ? And: are you sure the skip and dimensions parameters are ok ? You said it's off about 2 pixels on one side and three pixels on other side, so: why you don't correct the skip or dimensions parameters accordingly ? Or is this not correctable ? It's just a curiosity, I know this don't solve our problem so much...


Dunno...

I did a quick test, here's the results:

Histogram on LiveView:


Histogram on Quick Review (LV):


Then Quick Review histogram in Photo Mode (non LiveView):



Now about overexposure circles:

LiveView:


Quick Review:


So I have overexposure circles everywhere (LV, LV->QR, PHOTO_MODE->QR)...


And Auto ETTR is working pretty good i think:

Settings before Auto ETTR:


And after Auto ETTR:


Image after Auto ETTR:


I don't say it is perfect, but works.

dlrpgmsvc

Ouch !  :o Here I'm missing something very hard to understand for me ! Why I'm missing the warning circles in the QR mode ?!?  :o
Can you send me your BIN, please ? Just to check ! Thanks !  :o

I checked and double-checked: I get perfectly identical histogram values in lv and qr, BUT in qr there are no warning circles !  :o

If you send your bin, then, if the problem persists, it's a matter of values and parameters inside ml or canon menus in my camera. If, otherwise, the problem is gone, then it's a difference in our sources !  ::)
If you think it's impossible, you have lost beforehand

xaint

Quote from: dlrpgmsvc on May 28, 2013, 08:52:20 PM
Ouch !  :o Here I'm missing something very hard to understand for me ! Why I'm missing the warning circles in the QR mode ?!?  :o
Can you send me your BIN, please ? Just to check ! Thanks !  :o

I checked and double-checked: I get perfectly identical histogram values in lv and qr, BUT in qr there are no warning circles !  :o

If you send your bin, then, if the problem persists, it's a matter of values and parameters inside ml or canon menus in my camera. If, otherwise, the problem is gone, then it's a difference in our sources !  ::)


Sure! Here you go: xaint.hu/autoexec.bin

dlrpgmsvc

If you think it's impossible, you have lost beforehand

dlrpgmsvc

try to go to the wrench menu and then -restore ml defaults- then shut down and restart your 550d so we have same settings, then see if warn circles appear in quick review ... in my 550d not!  :o
If you think it's impossible, you have lost beforehand

xaint

Quote from: dlrpgmsvc on May 28, 2013, 09:34:42 PM
try to go to the wrench menu and then -restore ml defaults- then shut down and restart your 550d so we have same settings, then see if warn circles appear in quick review ... in my 500d not!  :o

You try it on 500d? or typo?  500D 550D -not the same

EDIT: anyway: it's working after -restore ml defaults too

dlrpgmsvc

Quote from: xaint on May 28, 2013, 09:39:15 PM
You try it on 500d? or typo?  500D 550D -not the same

Typo ! I mean 550d ! sorry !!!  ;D  corrected now...  ::)
If you think it's impossible, you have lost beforehand

dlrpgmsvc

I also tried to delete all canon settings to their defaults, and also defaulting all the canon fn(c) functions, but the problem persists ! Try also you, to double check if perhaps you have some settings inside canon parameters (normal or fn(c)) that make the warn circles working on qr mode: if by resetting them you come to my results, it's some canon parameters, otherwise the mistery goes on !  :o
If you think it's impossible, you have lost beforehand

xaint

OK, every settings in its default mode, and after I change the picture quality back to RAW, everything working again :)

Mystery!

xaint


dlrpgmsvc

Yes ! But... there are news ! juggling with the settings, i discovered that by rising the ISO setting from 100 to 200, circles appears in qr ! , but are lower values than lv ! Then going to 400 iso, warn circles values are in some way equals, even if not very much. Then going to 800 iso and up, the warn values in qr are way way more Greater than in lv !!!! Do we discovered the problem ? ...  :o

Can you make also this test ?!? So we can arrive to a consolidated raw histogram for 550d !

However, it's still incredible on how your values (lv and qr) are so almost perfectly equal !!! And mine not !   :o
If you think it's impossible, you have lost beforehand

xaint

OK, i think it's depend on the subject/scene (dark room, long exposure...) What you can see in LiveView on the display, I think it's just simulated (Exp.sim). Let's say you set the shutter to 6 seconds, then what you see on the display it's not the real picture :) so i think maybe here's the difference?

oh i can not express myself... i know a lot of words, but i don't know grammar :D:D:D

dlrpgmsvc

Yes I understand perfectly what you are saying ! And I agree... but... why you obtain so perfect match between lv and qr, and me not ? Have you made the ISO switching tests to see if you maintain the perfect match between lv and qr in all iso values ?
If you think it's impossible, you have lost beforehand

xaint

Yes, i've made the test, the values are almost the same. The histograms update value is 1 sec, so the hand shaking maybe a factor i don't know. :) Please create some sample pictures at daytime and use a tripod, then post the results here :)

dlrpgmsvc

Ok, may thanks !

For the moment I made a pull request to add raw histogram on lv and qr modes, I think it's stable now, thanks to your valuable help !

The next problem is that in review mode (not quick) the histogram is normal and not raw, do it's so also for you ?
If so, this is our next problem to solve now !  ;)
If you think it's impossible, you have lost beforehand

xaint

Yes, I think there is no raw histogram in review mode. Right?