Messages

Have new card now: Sandisk Extreme Pro SDXC UHS-I 64gb (170MB/s read, 90MB/s write)

Compiled latest updates. Card-spanning and SD overclocking working well now:

I can get 1min+ 3600 x 1532 / 24fps / 2.35:1, continuous at 3072p. 1080p 14bit on SD card alone – continuous!

Many thanks @ilia3101, @Danne.

@ilia3101 With your newly posted build I get:

Card spanning + SD overclocking = on, I get msg: File Create Error. Stops recording immediately.

SD overclocking = off, Card spanning = on, I get Frame order... issue (Think you have discussed this?) and stops after 8 secs or so.

Upon restart had msg twice: Camera not shutdown properly. Skipping module loading.

Cards: CF Sandisk Extreme Ultra Pro 160mb/s (Been using for 3K fine for a year) + SD Sandisk Extreme 90mb/s. (Also been fine for ML core files).

@ilia3101 Aha! Thank you.

just move both of them to the same folder/card, then you will be able to open it with MLV App.

I see this to be true now.

I will try your updated builds and report back.

Thanks!

@ilia3101 Thank you for your hard work bringing this to 5D3!

I have tested both a build compiled from Danne's branch :
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets_card_spanning

and yours:
https://github.com/ilia3101/mlv_lite_spanning/releases/download/test-builds/magiclantern-Nightly.2020Jan23.5D3113.zip

Fastest card I have is Sandisk Extreme 90Mb/s U3.

On Danne's branch: I get almost instant stops in UHD 3850px with message: "Frame order error...Slot 10(?)...e13...expecting 19..." The message disappears very quickly so I can't read it all! A file has been saved to SD I notice: M25-1206.M00 – file is 50mbs but I can't open it in MLVApp.

**EDIT** I have just tried turning Spanning = On, but Overclocking = Off. It now records one file to CF and one to the SD, both with same timestamp but the SD one with the .00 suffix. Can't open .00 file in MLVApp or MLVFS.

On ilia's I get an error printed about: Missing crop-marks and also sd_uhs will not run on this camera. (I'm on 5D3 1.1.3)

Not my understanding of exposure. From the people who bring us RawDigger – a tool I believe you have used in the past in your research into this subject. Source of quote: https://www.fastrawviewer.com/blog/mystic-exposure-triangle

Code Select Expand


Definitions and Facts
To start with, let's lay out a few basic definitions and empirically verifiable (and indeed verified) facts that have to do with exposure and ISO:

Exposure time: this is the time during which light hits the photosensitive material in your camera; when the shutter closes, exposure ends.
Photographic exposure is only about two things: the light intensity and the exposure time: the light, coming from the scene, hitting the front element of the lens, reduced by the aperture opening, and acting during the exposure time.
By definition, photographic exposure is calculated as the product of the illuminance of the photosensitive material and the exposure time, and is measured in lux-seconds; photographic exposure is the product of luminous flux per unit area by time.
The elements that have direct control over exposure in the camera are the shutter and the aperture (not counting built-in neutral or other filters).

As such, ISO cannot be an element of exposure; it's determined through the exposure (see standard ISO 12232:2019, "Photography — Digital still cameras — Determination of exposure index, ISO speed ratings, standard output sensitivity, and recommended exposure index" and the wiki), and is "applied" after exposure.

ISO 12232:2019(E) - ISO speed definition
Standard ISO 12232:2019, section 3, p. 3.7 (page.2)

At the same time, ISO is not the sensitivity of the sensor (see the datasheets on sensors): the sensitivity of the sensor doesn't change when altering ISO, the only thing that is changed is the scaling coefficient applied to the signal / data obtained as a result of exposure.



blindly throwing shit at a wall and seeing what sticks.

This is totally unnecessary, lowers the tone of debate, and contributes nothing of value to the discussion. Not appreciated.

...fastrawviewer only indicates how many pixels are below a defined saturation point.

Useful to know! But I'm describing the difference that it reports between pairs of figures; I'm not changing the saturation point in the prefs – and both images reported 16281 as a white level in the raw_diag readout. So both are at their clipping point I presume.

...So what register is it that you are tweaking to increase exposure (reduce underexposure percentage in fastrawviewer)?

Whatever the tweaks are doing behind the scenes I don't pretend to know, but surely no [register] tweak can increase the light hitting the sensor, or increase its sensitivity.  So, for my money they're not increasing exposure.

The register tweaks are no different to the discoveries made by yourself, a1ex and others way back in the original thread, and all available to check on my repo in the source code and Lua preset that defines the values. https://bitbucket.org/rivertim/magic-lantern-danneclone/downloads/

As for the increase in dynamic range, I'm getting a difference in almost 2/3 stop as reported by raw_diag.

And why?

Because it's there.

Thanks @Luther! 

how much of this better DR was cause by the ADTG tweak versus changes in exposure? If you're changing the ISO between stock 100 to 108, this will also increase exposure and could account for the changes in DR.

It's my present understanding that 'exposure' is: an amount of photons hitting the sensor. This amount can only be altered by a) Lighting. b) Aperture. c) Shutter speed. The sensor only has a sensitivity that is factory-fixed – it cannot be made more or less sensitive by settings in the camera. The micro-voltages from the sensor (photo-electrons?) are then only amplified by the analogue gains and then converted into digital.

Since I didn't change a), b), or c) between the pairs of shots, and in the light of the above – can this be described as an exposure increase? I'm sure I'll be corrected if I've got this wrong!

But irrespective of the photo-mechanics and the terms we use – and now these presets are easy to load (for 5D3 anyway...), anyone is free to try out the presets to compare results!

Cheers!

**PS** What is crucial to maximising these 'gains', is nailing your 'exposure' to maximum White Level –  absolutely accurate ETTR –which is a challenge! and requires some attention to the tools ML provides us with.

Here's part two of my videos exploring the results of register gain adjustments. Using this tool: https://www.fastrawviewer.com/ the video examines the extracted dng files in much greater detail, comparing percentage of over and under-exposed pixels in each pair of comparisons.  Surely we all want less underexposed and overexposed pixels counts...?

@Danne

From your rather alarmist tone one might think I was trying to force the community to "drink the Kool-Aid!" Let's get some perspective. This is simply my contribution to an open-source project. I'm not seeking endorsement, or support, or asking that my code be merged into the main branch.

Other users are free to test it and see if their results match mine – or like the proverbial TV, are welcome to turn it off at any time if they don't like it.

The values for the presets are all in the lua script! 5d3iso-tweaks.lua.zip https://bitbucket.org/rivertim/magic-lantern-danneclone/downloads/

The preamp values have to be calibrated for each camera to the max white level. So your camera maybe slightly different.

For sure! That's the wrong branch! I updated my op video post awhile back with the branch I'm now using. https://bitbucket.org/rivertim/magic-lantern-danneclone/src/iso-regs-Q-submenu-access/ All the reg values are there to see – or on the camera in Q-menu. Nothing hidden!

Oh right. There are annotations on the film for the important bits!

Regs and values are clearly visible in the submenu when the presets load! The stock ISOs are – stock! No reg tweaks.

Strange. Audio fine here. Does it play correctly on YouTube?

Here are some results from my most recent ISO test (Stock ISO vs Tweaked ISO) using my custom ML build for 5D3 and lua script to load the presets.

Branch: https://bitbucket.org/rivertim/magic-lantern-danneclone/commits/branch/iso-regs-Q-submenu-access

iso-dngs.zip contains the test files. All available on my repo downloads page:

https://bitbucket.org/rivertim/magic-lantern-danneclone/downloads/

By the way. You should port card spanning to mlv_lite

+1

I'm looking into this as well! I see the code in sd_uhs.c for enabling it in 5D3 (untested), but I assume additional code has to be ported into mlv_lite.

In which module does the code that needs to be ported live now?

@Dmytro_ua

The names for some of the submenus have changed. I think that's the main difference. Mine is working now as per my excerpt on the previous page if you want to compare.

The answer is (according to a Stack Overflow user):

Code Select Expand

menu.set("Presets", "CMOS[0]", 0x318)

Remove the " " from around the HEX value.

Righto.

Anyone have any helpful ideas why lua appears not to want to set HEX values?

Thanks.

@Danne Got the menu structure working. Now it sets some regs in "Presets" menu – but strangely not CMOS values in Hex. 

So here for example it sets all values – except the CMOS[0]

Code Select Expand



menu.set("Presets", "CMOS[0]", "0x333")
menu.set("Presets", "preamp_89ab", "-43")
menu.set("Presets", "gain_0xfe", "-4")
msleep(300)


and here it sets all values – except the CMOS[4]

Code Select Expand



menu.set("Presets", "CMOS[4]", "0x318")
menu.set("Presets", "gain_888x", "-258")
menu.set("Presets", "preamp_89ab", "-60")
msleep(300)



?

Running the block (excerpts below) of lua on my compile of Danne's commit #6dc0688 main gives me an error on the console:

Lua console:

Code Select Expand

no submenu, fall back to edit mode.

Lua:

Code Select Expand


-- Fast video presets for 5d3

-- Script enables video presets in MOVIE Tab ver.1.04

--console will be all over if not applied
  console.hide()
  menu.select("Movie")
  menu.close()
 
-- this script is for 5D3 only
if camera.model_short ~= "5D3" then
   display.notify_box("Script is not working on this cam", 2000)
   return
end

-- preset function begin

-- Preset 3k 2.35:1 24fps TNT-ISO100

function main1()
  console.hide()
  menu.close()
  lv.start()
  lv.zoom = 1

-- enable sound
-- menu.set("Sound recording", "Enable sound", "ON")
-- if menu.get("Sound recording", "Enable sound", "") ~= "ON" then
  -- display.notify_box("\nPlease, enable: \n\nmlv_lite.mo \nmlv_snd.mo \ncrop_rec.mo \n\n      ... and run script again", 6000)
  -- return
-- end 

-- warnings
while camera.mode ~= MODE.MOVIE do
  display.notify_box("\nSwitch to MOVIE mode")
  msleep(1000)
end

-- Presets

-- menu.set("Presets", "bitdepth", "10 bit")
--menu.set("Presets", "set 25fps", "OFF") -- Change to "OFF" to use 24fps by default
--menu.set("Presets", "reg_8xx8", "-31") -- Set iso-tuning regs
menu.set("Presets", "CMOS[0]", "0x113") -- Set iso-tuning regs
menu.set("Presets", "reg_8", "-1") -- Set iso-tuning regs
menu.set("Presets", "reg_fe", "-3") -- Set iso-tuning regs
  msleep(300)

-- enable crop_rec.mo. Checking first after trying to enable 3k preset
if menu.get("Movie", "Presets", "") ~= "3K 1:1" then
  display.notify_box("\nYou forgot to enable crop_rec.mo", 5000)
  return
end

-- movie
  menu.set("Movie", "FPS override", "OFF")
  --menu.set("Movie", "HDR video", "OFF")
  menu.set("RAW video", "Resolution", "3072x1308")
  menu.set("RAW video", "Aspect ratio", "2.35:1")
  menu.set("RAW video", "Crop rec preview", "auto mode")
  menu.set("RAW video", "Data format", "14-bit lossless")
  menu.set("RAW video", "Aspect ratio", "2.35:1")
  menu.set("RAW video", "Preview", "Framing")
  menu.select("Movie", "RAW video")
  msleep(100)
  menu.open()
  msleep(100)
  key.press(KEY.Q)
  msleep(100)
  key.press(KEY.SET)
  msleep(100)
  key.press(KEY.WHEEL_DOWN)
  key.press(KEY.WHEEL_DOWN)
  key.press(KEY.WHEEL_DOWN)
  key.press(KEY.WHEEL_DOWN)
  key.press(KEY.WHEEL_DOWN)
  key.press(KEY.WHEEL_DOWN)
  key.press(KEY.WHEEL_DOWN)
  key.press(KEY.WHEEL_DOWN)

-- Overlay
  menu.set("Overlay", "Focus Peak", "OFF")
  menu.set("Overlay", "Zebras", "OFF")
  menu.set("Overlay", "Magic Zoom", "OFF")
  menu.set("Overlay", "Cropmarks", "OFF")
  menu.set("Overlay", "Spotmeter", "OFF")
  menu.set("Overlay", "False color", "OFF")
  menu.set("Overlay", "Histogram", "RAW RGB, Log")
  menu.set("Overlay", "Waveform", "OFF")
  menu.set("Overlay", "Vectorscope", "OFF")
  menu.set("Display", "Clear overlays", "OFF")

-- done, turn on global draw
  menu.set("Overlay", "Global Draw", "LiveView")

-- go back to Presets Tab
  menu.select("Movie", "Fast video presets")
  msleep(200)
  menu.open()
  msleep(200)
  menu.close()
  msleep(600)
  key.press(KEY.MENU)
  msleep(600)
  key.press(KEY.MENU)
  msleep(600)

-- success!
display.notify_box("\nScript is done")
end

-- preset function end



Code Select Expand



mymenu = menu.new
{
    parent = "Movie",
    name = "Fast video presets",
    help = "Choose a preset with predefined settings",
    submenu =
    {
{
        name = " ISO100 3K 3072p 24fps 2:35.1",
    select = function(this) task.create(main1) end,
    help = "3K preset / 24fps / Ratio: 2.35:1 / 14-bit lossless / iso100",
--help2 = "For 24fps set 25fps to OFF in Presets menu Q",
        },

    },
}




@Danne Does the new menu structure of your build require any magic sauce?

I'm trying to target Digital Gain regs with an adjustment. I find this in iso_regs:

Code Select Expand


if (reg == 0xc0f37ae4 || reg == 0xc0f37af0 || reg == 0xc0f37afc || reg == 0xc0f37b08)
        {
            default_digital_gain = val;
            if (digital_gain) *copy_ptr = digital_gain;
        }


I've tried:

Code Select Expand


adtg_new[40] = (struct adtg_new) {6, 0xc0f37ae4, 0x200 + gain_dgain};
adtg_new[41] = (struct adtg_new) {6, 0xc0f37af0, 0x200 + gain_dgain};
adtg_new[42] = (struct adtg_new) {6, 0xc0f37afc, 0x200 + gain_dgain};
adtg_new[43] = (struct adtg_new) {6, 0xc0f37b08, 0x200 + gain_dgain};


or

Code Select Expand

adtg_new[44] = (struct adtg_new) {6, 0x8030, 0x200 + gain_dgain};

Do any of these reg addresses look correct?

Cheers!

Hi! Would it be possible to add a clip text-labelling feature? To be able to add a short text note in the clip window to identify clips would add super-shine to MLVApp!

I cloned your repo at commit #5a6f55b 2019‑12‑31 and compiled locally.

I can see that you have now enabled modules by default and that the modules tab doesn't display. But if you add other 'extra' modules in – as I did with a1ex's raw_diag.mo – the module tab then appears and can be accessed to enable these extra modules – but the rest of the modules still remain 'invisible'.

So I don't think it is an 'older build'. ?

How are you running raw_diag module?

Just by loading and enabling the module. I only really use it for the raw info readout window.

These are the error screens, but I noticed that Sound recording was enabled – my bad. However, I was running raw_diag.mo. I disabled that module and it seems to be ok again. Never seen that error before!  Good to know!