Author Topic: How to view RAW histograms after taking the image?  (Read 2516 times)

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to view RAW histograms after taking the image?
« Reply #50 on: October 04, 2017, 05:55:09 PM »
Before including the updated code in regular nightlies, may I ask for testing feedback from owners of the other cameras present in the crop_rec_4k builds? (700D, EOSM, 100D).

Reason: other sensors might clip to white in different ways, and the current heuristic makes a tight assumption: that the clipping point is harsh and spans on one or two levels (not more). I'm not sure whether this holds true on other camera models.

heyjoe

  • New to the forum
  • *
  • Posts: 41
Re: How to view RAW histograms after taking the image?
« Reply #51 on: October 04, 2017, 07:35:41 PM »
Reason: other sensors might clip to white in different ways, and the current heuristic makes a tight assumption: that the clipping point is harsh and spans on one or two levels (not more). I'm not sure whether this holds true on other camera models.

I don't know if this may help but here is a video about RD which mentions highlight histogram shapes (scroll to 2:40). The speaker says "A bell" shape is typical for Canon and also talks about the changing of saturation value at different exposure parameters (which you mentioned in an earlier reply).

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3031
  • 5D3 / M1 / 7D / 70D / SL1
Re: How to view RAW histograms after taking the image?
« Reply #52 on: October 04, 2017, 10:08:03 PM »
Before including the updated code in regular nightlies, may I ask for testing feedback from owners of the other cameras present in the crop_rec_4k builds? (700D, EOSM, 100D).

Reason: other sensors might clip to white in different ways, and the current heuristic makes a tight assumption: that the clipping point is harsh and spans on one or two levels (not more). I'm not sure whether this holds true on other camera models.

Done.
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10326
  • 5D Mark Free
Re: How to view RAW histograms after taking the image?
« Reply #53 on: October 07, 2017, 11:20:15 AM »
Tested some long exposures (30 seconds at higher ISOs) - it no longer clips harshly, so my algorithm thinks the images are not overexposed (even when it's obvious to the human eye on the histogram).

Will push a fix shortly.

heyjoe

  • New to the forum
  • *
  • Posts: 41
Re: How to view RAW histograms after taking the image?
« Reply #54 on: October 07, 2017, 12:16:22 PM »
Will push a fix shortly.
Good. Please let us know if we need to test anything again.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3406
  • Blunt and to the point
Re: How to view RAW histograms after taking the image?
« Reply #55 on: October 09, 2017, 01:31:42 AM »
Good. Please let us know if we need to test anything again.

Test the latest code changes.  Does the latest fix actually work?  Did it brake something? 

Test as if you are doing QC for a company.

heyjoe

  • New to the forum
  • *
  • Posts: 41
Re: How to view RAW histograms after taking the image?
« Reply #56 on: October 09, 2017, 02:34:58 AM »
Test the latest code changes.  Does the latest fix actually work?
Where is it? Has it been published? What exactly is new to test?

Quote
Did it brake something? 
I definitely prefer not to test for the purpose of answering this particular question. A non booting camera is the last thing anyone needs.

Quote
Test as if you are doing QC for a company.
I have been doing it so far. Not sure what you are implying.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3406
  • Blunt and to the point
Re: How to view RAW histograms after taking the image?
« Reply #57 on: October 09, 2017, 06:07:06 AM »
Where is it? Has it been published?

Here.  Specifically here and here.

In that first link, on the right hand side next to the date, you can see that the fix was pushed to branches crop_rec_4k and iso_research, not the nightlies (unified branch).  In other words, the semi-stable stable nightly build doesn't yet have the fix applied, because...........it needs people to test it first.

So, if you really want to help the project (no implications, simply statements (perhaps I should find a way to reword this statement)), the next (easy) step is to go here.  If you have a 5D3, 100D, 700D or EOSM, yay, we have a winner, look at "4K raw video recording; lossless compression".  You will notice in the changelog for the latest builds there.

Code: [Select]
Raw backend: allow non-harsh clipping for white level
(should fix clip warnings at long exposures with high ISOs, e.g. 30" ISO 6400)

So these builds have the latest code change applied.

If you don't have one of the above mentioned cameras, and still (really, really, really*) want to help, then you'll have to learn how to compile the source code, compile the iso_research branch and test as if you were.............

*  If you really want to help, but cannot or will not follow the above, then sorry, you can't help.  Maybe next time.

What exactly is new to test?

The raw histogram.

a1ex has pushed some new code changes regarding the raw histograms, to fix an issue he noticed (while testing as if he was doing QC for a company) regarding high ISOs and long exposures. This code change may have fixed the issue, it may have only fixed the issue on his camera, it may have broken something else (maybe the histogram no longer works on a 6D).  And so, to beat that dead horse, this needs testing as if..........

I definitely prefer not to test for the purpose of answering this particular question. A non booting camera is the last thing anyone needs.

Whoa, slow down there.  I didn't mean the latest fix might shatter your camera into a million pieces (although it may), only that the histogram might not even work any longer on your camera after this code change.  The histogram might be broken.  Maybe the histogram menu item no longer displays.  Maybe some other minor little thing (that won't result in a non booting camera) might be broken.  Someone needs to test this.  You asked "Please let us know if we need to test anything again".

Not sure what you are implying.

Most of the time we focus on something simple, or expect someone else to do all of the work, or maybe we just aren't entirely sure exactly how to help efficiently.  So don't just take a single photo, observe that the histogram appeared to work, and call it a day.  Test that sucker.  Try and find a repeatable test case where the histogram doesn't work.
If you find something broken, good, tell a1ex while he's in the mood to play with this specific piece of ML code.  If you don't find anything broken, even better, make sure you tell a1ex that also, as it will build his confidence to push this fix into the nightly builds so that everyone can enjoy the fruits of your work.

Most importantly, if my post offended you, go back and read it with a sense of humor.  :D  It's not personal.  Perhaps you cannot help further, that's fine.  But someone else quietly reading this thread may be able to help, and so the steps I have outlined may be useless for you, but...........

Audionut signing off, good day Sir.

heyjoe

  • New to the forum
  • *
  • Posts: 41
Re: How to view RAW histograms after taking the image?
« Reply #58 on: October 09, 2017, 06:06:15 PM »
Thanks for the clarifications. I just needed to know when the changes are complied and ready for testing as I still don't know how this whole system of development works.

Unfortunately I cannot test "as if doing QC test for a company" because:

- companies testing all possible scenarios provide the necessary equipment for it (which I don't have)
- people at companies who test have a lot of fully dedicated time for it (which I don't have)

So it would be quite silly to pretend that I am doing something which I am not. What I can do is to repeat the tests done so far (for consistency). Perhaps I won't be able to set up a scene for testing ISOs above 6400 with exposures longer than 30" as I normally work with sufficient light (e.g. strobes or day light) and such low light scenario would be quite out of my range.

I will test the latest crop_rec_4k build and write again.

Danne

  • Hero Member
  • *****
  • Posts: 3390
Re: How to view RAW histograms after taking the image?
« Reply #59 on: October 09, 2017, 06:27:39 PM »
Checking the branches(code) is an easy way to follow what´s happening:
https://bitbucket.org/hudson/magic-lantern/branches/

The recent changes will always be on top. ML is heavily open source and changes to code are published all the time. Here is crop_rec_4k branch for instance:
https://bitbucket.org/hudson/magic-lantern/branch/crop_rec_4k

Learning how to compile and set up a source tree is a great way to start digging into code(I hardly can follow 1 percent of the stuff but it´s nice to read and compile it nevertheless :P):
Mac
http://www.magiclantern.fm/forum/index.php?topic=16012.0
Win
http://www.magiclantern.fm/forum/index.php?topic=15894.0


Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3406
  • Blunt and to the point
Re: How to view RAW histograms after taking the image?
« Reply #60 on: October 10, 2017, 01:50:38 AM »
Equipment needed.

- Camera............check
- Grey matter between ears...........check

We are not a company, we are a bunch of guys continually doing stuff on our free time.  a1ex tends to just have a bit more spare time then the rest of us.

The best part of this whole thing, is that we are not some company.  We don't have a hill for shit to roll down when things go wrong, shareholders screaming and bitching and moaning, middle managers over stepping their pay grade, or all of the other negative things associated with a company.  We understand that each person is helping, just because.  Not for monetary gain, but for reasons that supersede money.  So when we say, test as if doing QC for a company, we don't mean, do this and we will employ you as QC, we mean, spend a little extra effort with that grey matter between your ears then you might have otherwise have normally applied.  Test outside of your range and don't just consume.

Cheers.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3406
  • Blunt and to the point
Re: How to view RAW histograms after taking the image?
« Reply #61 on: October 11, 2017, 06:02:22 AM »
That last post sounds a little harsh.  Feel free to tell me I'm being a prick via PM.  I promise that's not my intention, but I would rather have you vent some steam, and continue to support the project further, then get discouraged and pack your bags.

heyjoe

  • New to the forum
  • *
  • Posts: 41
Re: How to view RAW histograms after taking the image?
« Reply #62 on: October 11, 2017, 10:31:16 AM »
That last post sounds a little harsh.  Feel free to tell me I'm being a prick via PM.  I promise that's not my intention, but I would rather have you vent some steam, and continue to support the project further, then get discouraged and pack your bags.
I am glad you are self-critical :)

I am also sure that your intentions are good but perhaps it is better to slow down with the multiple "if you really want to help" and the extra effort remarks as the whole thread is getting quite off topic. Perhaps it is a good moment to note that I highly appreciate the work a1ex did here but please also bear in mind that it may have not been done if this very thread didn't exist in the first place. So we all do what we can.

I already promised that I will test and I explained what exactly I can do (as soon as I have time for it). That is what is possible for the moment. No bags packed :)

heyjoe

  • New to the forum
  • *
  • Posts: 41
Re: How to view RAW histograms after taking the image?
« Reply #63 on: October 15, 2017, 11:59:52 PM »
Tested as promised:

CR2

160 0.8s

160 1s

160 1.3s



100 1.6s

100 2s

100 2.5s


6400 30s 8.0

6400 30s 7.1

6400 30s 9.0

6400 30s 10.0