Messages

Ah right, I thought I read somewhere it was getting backed up.

I might run a website cloner on the hudson repository, because why not!

Is this downloading all of the comments on commits, PR's etc?

Did you look in the .hg folder?

A random folder downloaded here:

I'm seeing the same, they are all ".hg/hgrc".

Another one in red every now and then is something like "stream ended early, expected xxxx bytes, but got xxx bytes".

And some 404: not found.

I initially had 5 scripts running which was keeping my 100Mbps reasonably saturated and things were singing along nicely, but after some time my storage drive couldn't keep up with all the random writes and crashed to a halt. #beware

There's probably an easier way to do this, but here's what I did (since HDD thrashing was my limiting factor).

Create another folder on another HDD, drop the script and (other) hg-repos-xx files in there. Then drop a symbolic link of that folder into the "home" directory for ubuntu.

Then in ubuntu.

Code Select Expand

cd /home/newfolder
sudo ~/myscript.sh hg-repos-xx

I had to implement a good QOS to stop my eldest son complaining about his ping times. Has the advantage of doing things like this while keeping the connection perfectly usable for normal tasks.
Went down a rabbit hole involving bufferbloat.

Frustrates me trying to work out all of the dependencies to get things like this running. Anyway, I've got the following running...

hg-repos-07
hg-repos-13
hg-repos-17
hg-repos-18
hg-repos-19
hg-repos-20
hg-repos-21



edit: See this post by kitor before proceeding with the below: https://www.magiclantern.fm/forum/index.php?topic=24420.msg228146#msg228146

For those following along with a windows 10 x64 box who want to help.

You need to install the windows subsystem for linux and get ubuntu: https://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/
*remember username and password

Install mercurial from within ubuntu:

Code Select Expand

sudo add-apt-repository ppa:mercurial-ppa/releases
sudo apt-get update
sudo apt-get install mercurial

Copy a1ex's hacky download script from above and save it as whatever.sh
This may help: https://www.howtogeek.com/261591/how-to-create-and-run-bash-shell-scripts-on-windows-10/

Download the hg-repos-xx from above.

The default home folder in ubuntu is located at:

Code Select Expand

C:\Users\xxxx\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\username

Replace (xxxx) with with your windows 10 user. Replace (username) with whatever username you used when installing ubunutu.

I would suggest to cut the username folder and paste it wherever you want to store these downloaded repos. Then drop a symbolic link back into the original ubuntu home location.
I use a link shell extension to make that task easier.

Copy the script you saved and the hg-repos-xx to the username folder.
In ubunutu make the script executable.

Code Select Expand

chmod +x ~/whatever.sh

Then run it:

Code Select Expand

sudo ~/whatever.sh hg-repos-xx



I initially had 5 scripts running which was keeping my 100Mbps reasonably saturated and things were singing along nicely, but after some time my storage drive couldn't keep up with all the random writes and crashed to a halt. #beware

I have a good connection and plenty of space (I also have significant hosting space available) but lacking the mindset to tackle (extended) debugging for this task.

If you create a relatively simple workflow I'll jump on it.

Possible to split the task up by month? 12 concurrent scripts running to download a years worth of data.

From memory in the autoexposure module settings, in the graph of settings, there is a blue horizontal line after taking an exposure, or metering a scene with half shutter.

This blue line represents the EV value of the scene you just metered.

So....the module will use the settings on the graph that match with the blue line.

If you want the module to be using different settings at that metered exposure, then you need to adjust the module and tell it to use different settings.

The module doesn't care what happens when the shutter button isn't being pressed. When the shutter is pressed, the module grabs the EV and sets the exposure settings accordinally, for that shot.

From memory that's how the module works. Every time you take a shot it will use the settings in the module.
Maybe the settings stick in M mode, I don't remember.

What's the problem with the current functionality?

If you're not going to tolerate my petty demands, then just delete my account now.

Classic NPD.



The capitals really drove home the message being conveyed. 

You're post is self-centered and useless (to everyone except you).

You've made this thread 4 minutes after registering an account because you were to lazy to find the answer yourself.

If you bothered yourself to put some effort in, you would find plenty of answers in a search for 2000D on these forums, and in the FAQ.

Negative attitude = / = Strong criticism.

we've basically got the 5d3 iso invariant from 100-400

No, you have not.

with a vast improvement in the noise of 100.  it's actually cleaner in the shadows than 200 now!

Everything else being equal, no, it is not.

CMOS sensor > downstream electronics.

CMOS sensor in 5D3 capable of 14.7 stops of dynamic range.

Sensor Full Well Capacity at lowest ISO: 68,900 electrons.
Sensor dynamic range = 68900/2.5 = 27,560 = 14.7 stops.
(note: limit read noise to ISO that give at least 8 stops dynamic range)

The 5D3 doesn't have 14.7 stops of DR at ISO 100.  Why?  Because downstream electronics are to noisy, limits DR otherwise available from the CMOS sensor.  About 3.7 stops of the sensors DR is lost in the noise of the downstream electronics.

When you adjust ISO (sensitivity ), you say to camera, clip (throw away) 1 stop of highlight data so that you may better capture 1 stop of shadow data.  The full stop of shadow data from the sensor that would otherwise be lost in the noise of the downstream electronics is boosted so that it is no longer lost in the noise of the downstream electronics. Of course, the drawback is that you MUST sacrifice one full stop of highlight data to capture that shadow data.

So......

Saying that you've suddenly made the camera ISO invariant from 100-400, is the equivalent of saying you increased the DR of ISO 100 by 3 stops.  You have not!
Saying that the noise in the shadows of ISO 100 is cleaner then the noise of the shadows at ISO 200, is the equivalent of saying you increased the DR of ISO 100 by over 1 full stop, but haven't been able to extract any improvement at ISO 200.  This doesn't make any sense what so ever, since the improvement has to come from the downstream electronics, and adjusting sensitivity (I mean ISO) gives the shadows a boost by default.

[your]

I wasn't commenting on the story but on your logic.

There's your problem.  Let's move on!

In a film camera you have film.
In a digital camera you have a processing chain (sensor, amplifiers, digital manipulation).

In a film camera, if you change from ISO 100 film to ISO 400 film you have increased the sensitivity (for the same amount of light hitting the camera = brighter output) of the camera.

If you adjust the ISO setting on a digital camera, you also adjust the sensitivity of the camera. Did you adjust the mechanical sensitivity of one specific part (sensor) of the processing chain, no, but (in Canon cameras especially) you gain an increase in output (brightness) of the camera with a corresponding increase in the quality of pixels (output).

ISO is exactly a sensitivity setting.

The sensitivity of the downstream processing chain from the sensor. It is a gain control (input sensitivity) for further downstream processing components.

In a film camera you are stuck with a preset sensitivity of the camera based on the film. To change the sensitivity of the camera you need to change the film. With a DSLR you can simply change a few knobs.

[based on the time and effort of others]

Ok, lets play semantics.

What makes the author of that software a definitive authority on this subject?

https://www.iso.org/obp/ui/#iso:std:iso:12232:ed-3:v1:en

3.7
ISO speed
numerical value calculated from the exposure provided at the focal plane of a DSC to produce specified DSC image signal characteristics
Note 1 to entry: The ISO speed is usually the highest exposure index value that still provides peak image quality for normal scenes. However, a DSC does not necessarily use the ISO speed value as the exposure index value when capturing images.

3.3
exposure index
EI
numerical value that is inversely proportional to the exposure provided to an image sensor to obtain an image
Note 1 to entry: Images obtained from a DSC using a range of exposure index values will normally provide a range of image quality levels.

3.9
photographic sensitivity
general term used for numerical values calculated based on the exposure at the focal plane of a DSC which produces a specified DSC image signal level, such as the standard output sensitivity or recommended exposure index
Note 1 to entry: In practise, the photographic sensitivity is often called the "sensitivity" or the "camera sensitivity". It is sometimes called the "ISO sensitivity", for historical reasons that date from ISO standards for photographic film cameras.

3.13
sensitivity setting
numerical value of the photographic sensitivity used by a DSC when capturing images
Note 1 to entry: In some cases the sensitivity setting is set by the user. In other cases it is set automatically by the DSC.
Note 2 to entry: In DSCs employing an automatic exposure control system, the difference between the EI value used to capture an image and the sensitivity setting is called the "exposure bias". The value of the exposure bias is typically indicated using EV (exposure value) units.
Note 3 to entry: For historical reasons, the sensitivity setting of a DSC is often labelled the "ISO".

This is totally unnecessary, lowers the tone of debate, and contributes nothing of value to the discussion. Not appreciated.

You, yourself, acknowledge that you do not understand what is going on behind the scenes, so to speak. What you are doing is throwing shit at a wall, and seeing what sticks.
You are somewhat randomly (based on the time and effort of others) creating a preset of values for registers that you do not understand, applying no scientific method at all, and presenting the shit that's left stuck to the wall as useful data.

Then when someone who has spent the time finding those registers, and trying to define what they do, comes along and questions the shit that's flung all over the walls around the place, you deflect from the topics of discussion which hold value (how is the quality of the pixels being captured improved, and how is that being scientifically verified) with semantics and hurt feelings.

http://www.catb.org/esr/faqs/smart-questions.html#not_losing

Exaggeratedly "friendly" (in that fashion) or useful: Pick one.

I absolutely and unequivocally respect you getting your hands dirty (playing with code).

But I'm describing the difference that it reports between pairs of figures

Yes, but you don't seem to understand exactly what it is reporting!

At some point in the cameras processing chain, the register values you are tweaking have increased (brightened) the pixel values in the raw output file.  That is why the fastrawviewer underexposure percentage has dropped, because less pixels in the raw file are below the defined saturation point compared to Canon settings. But this value does not define the quality of those pixels or the amount of noise in those pixels, only that they are brighter!

Whatever the tweaks are doing behind the scenes I don't pretend to know, but surely no [register] tweak can increase the light hitting the sensor, or increase its sensitivity.  So, for my money they're not increasing exposure.

Nonsense.  ISO is exactly a sensitivity setting.  Let's not get hung up on the semantics of definition of exposure.
What we want is quality of pixels. The registers directly affect the quality of pixels.

Because it's there.

I honestly think this topic (register tweaking) is well past the point of blindly throwing shit at a wall and seeing what sticks.

The underexposure percentage value in fastrawviewer only indicates how many pixels are below a defined saturation point. It DOES NOT describe pixel quality, noise in pixels, DR or any other value of interest to the discussion.

So what register is it that you are tweaking to increase exposure (reduce underexposure percentage in fastrawviewer)? And why?

Since the sensor will be at ISO100 and then an higher ISO, should I under expose a little the image? This way, the lines at ISO100 will have correctly exposed highlights and little underexposed midtones, while the lines at ISO1600 will have correctly exposed shadows and a little overexposed midtones. Am I right? Is this the right way to take photos with Dual ISO?

No, because of shot noise: The higher the illumination, the less apparent the shot noise; the lower the illumination, the more apparent it is.

So in the end Dual ISO will let me brighten the darkest regions of a picture much more than usual. Is that right?

There will be less noise in the darkest regions than usual.

In other words, with a scene where you are able to capture all the DR but where the histogram is showing data to the left, with low tonal resolution, would shifting the shadow data by 1-3 stops to the right, using Dual ISO, help processing in post.

If you're simply shifting that data (as a whole, so to speak), likely not much at all, since you aren't attempting to take data with a limited number of discreet steps in luminosity and stretch that data to cover a greater range of exposure stops.

You're not taking data captured with 1 or 2 bits of tonal resolution and stretching that data out requiring more display bits.  If you're display chain is not accurate and thus shrinks areas of luminosity and expands other areas of luminosity, that's another issue.

If the data of interest is way over to the left (ETTL) and contains tonal resolution issues in the source data (likely hidden by noise (think dithering)) and you clean the noise and then shift that data to the right......

If you want to test (and we would all appreciate it), I wouldn't bother trying to capture good looking display images.  I would focus on capturing good data just in the shadows and focus on post processing that shadow data (highlights be damned).

I agree with your assessment of temporal artifacts in traditional HDR capture and processing, although maybe I just suck at it.  And mistakes make for the best discussion when they don't descend in to pettiness. 

The tonal quality only really comes into play if you want to stretch the contrast.

The top EV has 2046 levels and thus can be stretched quite aggressively.  When there is only 32 levels, this area of raw captured data can't be stretched as aggressively for the same quality.

If that area of raw capture (32 levels) is in the shadows after PP, it's (edit: likely) a non issue.

In your example I think 100/400 would have been more then enough.

Tonal quality and image noise are semi related but unrelated, if you get my drift.

In adtg_gui.c there are no hardcoded values for registers.  adtg_gui.c is looking up the values (as currently set by Canon) and displaying them to the user.  From what I can gather, however adtg_gui does that is your answer.

iso_regs is an old attempt to bring register tweaking to a more user friendly level.  It's 5D3 only and contained hard coded values because it was useful for the specific task.  Those values were determined from looking at what adtg_gui displayed.

They've no doubt been discussed before – but possibly way back in the thread.

You haven't read the entire thread.....shame. 

Might find something useful here too: https://www.magiclantern.fm/forum/index.php?topic=6751.0

It sounds like you are saying all of the register values (in adtg) have been found and hard coded (so that we know what they are and can call upon them).  That doesn't sound like something a1ex would do.  Most of the values are different for each camera and at different shooting modes for each camera.

In adtg after adjusting a value you can press set (or some other button I don't recall) to return the register to initial value (before modification).  Does that code help?

Can you dump the console to a text file with LUA.

Implemented via LUA: https://www.magiclantern.fm/forum/index.php?topic=24723.0