Messages

Can you render DNG - Cineform from Resolve and play it with MPV, just to make sure there is no OS issues causing this.

Hah! That worked. Thanks @Kharak MPV plays the CF file rendered YUV 10bit and RGB 16bit option from Resolve from DNGs.

Thanks @Danne @Kharak

Have installed MPV player. CF file rendered from MLVApp crashes it. The only app playing the file correctly is Resolve so far.

@Kharak Thanks. Which build should I use? (on Mac) https://mpv.io/installation/

@Danne Happy to render out to Cineform if @masc requires. Don't know how to render using ffmpeg command line though.

I had to remove my re-install of Cineform Studio from the other day as it kept crashing. Confirm that QuickTime OSX doesn't play CF files natively (even with CF Studio installed) – it does some sort of converting in that case. The player that did use to work natively with CF was the old QT7 player – long gone I believe.

The man who co-invented/engineered CF is David Newman https://cineform.blogspot.com/ He might be able to help out – and would certainly know about the active metadata side. He's very approachable.

PS Clarifying: also for me CF only plays [smoothly] in Resolve. All other players don't play at all, whether CF Studio is installed or not. QTX player converts during load if CF is installed – doesn't play at all once CF Studio is removed.

@masc

Agree with @Kharak that basic Cineform 1080p should always play like butter as it was designed for lower powered systems I believe (apart from the Film Scan flavours!) If in Mac, there used to be a system preference pane that controlled certain codec functions. Might check there if appropriate?

Re QT gamma shifts in Resolve/Mac read this thread on the Blackmagic Forum. The definitive answer:  https://forum.blackmagicdesign.com/viewtopic.php?f=21&t=114047

Found an old version of GoPro Studio on an old backup disk. Seems to have installed ok on Mojave. Can confirm that file exported from MLVApp now plays back fine so far in this app (GPS).

@masc Can confirm the result here on Mac 10.14.6

In the old days one had to download the Cineform app before the files could be played. I think there is a decoder element which is required. Owners of Premiere Pro should have this [decoder] already installed because I think Premiere ships with Cineform support. Anyone confirm?

@masc The range of corrections it carried in metadata included primary, image scaling, vignette, sharpness, LUT looks. It was a mature system. How that would work in MLVApp I have no idea! 

MLVApp has the greatest out-of-the-box colours for ML. If somehow even the very basic primaries and transform adjustments in MLVApp (which are hard to match in Resolve) could be carried 'live' into the Resolve timeline, then the two would be a ML powerhouse.

Probably off-topic, I know...

@masc It did happen! Any cineform file that carried the cc metadata would carry those corrections through to any timeline hosted by another application. It used to work in FCP7, Vegas, Premiere, and if my memory serves me right in early Resolve. You tweaked the colour in Cineform-Studio and the correction would appear in the edit timeline. I used it for years like that.

I'd show you now, but alas the Cineform-Studio is discontinued and doesn't run on anything higher than Mountain Lion I think.

Thumbs up for Cineform codec. I used it for years. It produces a tad large file sizes though.

The active metadata concept was awesome. I'd colour correct in the Cineform-Studio app, and edit in FCP7. As you do the corrections, they show up instantly (without rendering) in the FCP timeline.

Imagine if we could harness the awesome colour of MLVApp carried in metadata through cineform files – but edit in Resolve and use its render power.

*UPDATE*

Decided to start over again. Deleted qemu-eos folder.

Followed these excellent video tuts: https://www.magiclantern.fm/forum/index.php?topic=16012.msg191686#msg191686 and https://www.magiclantern.fm/forum/index.php?topic=2864.msg190851#msg190851

Installed with no error messages.

Partial success: I can now get to the qemu canon screen. But the keystrokes do not control navigation. I can see that the keystrokes are received at the terminal – but no menu navigation happens. I read an @a1ex post which suggested adding the Terminal to Security panel. This allowed me to control the menus with the arrow keys one time – but subsequently froze, and hasn't worked again.

Ideally I'd like to be able run qemu from a custom image of my physical SD card– but am having trouble making this work also.

Any suggestions?

I have read the thread, the README and searched.
Mac OSX Mojave 10.14.6

First attempt at installing QEMU in my local environment.

Install itself seems to be ok, but get error screen when attempting to run:

/path/to/qemu-eos$  ./run_canon_fw.sh 5D3,firmware="boot=1"




Why the message about 1.2.3? The code I copied to the sd.img was 1.1.3 So...

1) I mounted my camera SD card (with my current build on it) and copied the files to the sd.img as usual. Is this correct?

2) "For models that use a serial flash, you may have to dump its contents using the sf_dump module, then copy SFDATA.BIN as well." Should I be doing this for 5D3?

All I have in my QEMU/5D3 folder is:

debugmsg.gdb
ROM0.BIN
ROM1.BIN

Thanks.

If this is what the ACES converted to rec709 looks, it is wrong. While I don't know what the scene looked like, I don't believe it looked like that.

@ilia3101 Yes, I suspect that it's wrong also! Still researching...

I am asking some questions on the acescentral thread re this IDT/DCTL issue. Will report back here with any pertinent findings.

Closer examination show a bias towards purple with these DCTLs. Continuing this discussion on the DaVinci Resolve and ML Raw thread here: https://www.magiclantern.fm/forum/index.php?topic=15801.msg225208#msg225208

@baldavenger

Hi Paul

Firstly, thank you for your development and a highly informative read here.

I'm continuing this discussion from over on the MLVApp thread here: https://www.magiclantern.fm/forum/index.php?topic=20025.msg225175#msg225175

With help from @ilia3101 I've taken the CTL data from the test results in this thread *edit* https://acescentral.com/t/results-from-an-idt-evaluation/2229 and translated it into DCTL. Now I have some questions!

1) To what extent do your existing tools offer this already? (e.g. An IDT or DCTL for 5D3 ML raw to ACES in Resolve). I have your ported DCTL files installed and am not sure whether they cover the same ground, or exactly which DCTL to compare.

2) With the translations we have done today (from the data on that acescentral link above) the files look very biased towards purple. Can you spot something amiss in the translation?

Cheers!

Ok how about this: https://acescentral.com/t/aces-idt-dctl-generator/2566

Oh My God! That looks ****in awesome! Thanks for the great find @ilia3101 And it's only 4hours old. Syncronicity at work.

Playtime.

PS Hope all this will work with MLVApp also!

@ ilia3101 Yay! It works. Awesome work. Thank you!



That was a 3200K CTL

There's also a 5500K one:

Code Select Expand


// Canon_5D_Mk_III - 5500K
// Generated on July 15,2019  9:55:47 AM

import "utilities";

const float B[][] = { {0.852627, -0.013844, 0.161217},
  {0.028905, 1.168449, -0.197354},
  {0.057001, -0.365605, 1.308605} };

const float b[] = {2.293205, 1.000000, 1.453861};
const float min_b = min(b[0], min(b[1], b[2]));
const float e_max = 1.000000;
const float k = 1.000000;

void main (
input varying float rIn,
input varying float gIn,
input varying float bIn,
input varying float aIn,
output varying float rOut,
output varying float gOut,
output varying float bOut,
output varying float aOut )
{
float Rraw = clip((b[0] * rIn) / (min_b * e_max));
float Graw = clip((b[1] * gIn) / (min_b * e_max));
float Braw = clip((b[2] * bIn) / (min_b * e_max));

rOut = k * (B[0][0] * Rraw + B[0][1] * Graw + B[0][2] * Braw);
gOut = k * (B[1][0] * Rraw + B[1][1] * Graw + B[1][2] * Braw);
bOut = k * (B[2][0] * Rraw + B[2][1] * Graw + B[2][2] * Braw);
aOut = 1.0;

}



Which I translate as this in DCTL language (Is this correct @ilia3101 ?) It certainly looks correct in Davinci...

Code Select Expand



__DEVICE__ float3 transform(int p_Width, int p_Height, int p_X, int p_Y, float p_R, float p_G, float p_B)
{
const float r = (p_R * 0.852627f) + (p_G * -0.013844f) + (p_B * 0.161217f);
const float g = (p_R * 0.028905f) + (p_G * 1.168449f) + (p_B * -0.197354f);
const float b = (p_R * 0.057001f) + (p_G * -0.365605f) + (p_B * 1.308605f);

return make_float3(r, g, b);
}



Here are the resulting DCTL/IDT files if anyone wants to test: https://bitbucket.org/rivertim/magic-lantern-danneclone/downloads/

@ilia3101

Thanks! I don't know how to add that matrix to MLVApp, but I've made a .DCTL from it which Resolve now recognises. But the transform is not correct yet:



Does it maybe require some extra code from the CTL, like:

Code Select Expand

void main (
input varying float rIn,
input varying float gIn,
input varying float bIn,
        etc...


?

I believe this CTL data is based on the spectral sensitivity analysis of the cameras (5D2 and 3) carried out in testing.

I thought it would be great to see how it performs in Resolve, but it needs to be in DCTL form, so it needs translating from CTL.

Apparently this is a straightforward job! Just not for me!

Anyone fancy a go?

Maybe MLVApp can use these transforms also?

**Update**

As I understand it, the .CTL data needs to be converted to .DCTL for Davinci Resolve to recognise it and treat it as an ACES IDT function.

Can anyone assist with translating the language of the .CTL file from the ACES links here:

Code Select Expand


// Canon_5D_Mk_III - 3200K
// Generated on August 05,2019 10:40:40 AM

import "utilities";

const float B[][] = { {0.883020, -0.083165, 0.200145},
  {-0.008164, 1.114861, -0.106697},
  {0.048320, -0.441363, 1.393044} };

const float b[] = {1.589617, 1.000000, 2.151074};
const float min_b = min(b[0], min(b[1], b[2]));
const float e_max = 1.000000;
const float k = 1.000000;

void main (
input varying float rIn,
input varying float gIn,
input varying float bIn,
input varying float aIn,
output varying float rOut,
output varying float gOut,
output varying float bOut,
output varying float aOut )
{
float Rraw = clip((b[0] * rIn) / (min_b * e_max));
float Graw = clip((b[1] * gIn) / (min_b * e_max));
float Braw = clip((b[2] * bIn) / (min_b * e_max));

rOut = k * (B[0][0] * Rraw + B[0][1] * Graw + B[0][2] * Braw);
gOut = k * (B[1][0] * Rraw + B[1][1] * Graw + B[1][2] * Braw);
bOut = k * (B[2][0] * Rraw + B[2][1] * Graw + B[2][2] * Braw);
aOut = 1.0;

}



to .DCTL as described in this guide: https://drive.google.com/open?id=15AB3eZ9m78pT03nJNY8SO3t1IqnF23lt

It's waaaaay over my head! 

PS This may be off-topic – or not, as it might be useful in MLVApp also?

@ilia3101

Thanks for sharing that info. Very interesting. Has anyone tried loading the 5D .ctl files (acting as IDTs) from the linked post into Resolve?

I am putting them into the LUT folder as described here http://colorizer.net/index.php?op=aces but Resolve isn't 'seeing' them at all.

PS This is the dropbox link that has all the sensor data files in: https://www.dropbox.com/sh/xepdrlu8qtubhhl/AADR_QuBf5Sn2WO7lp5MDNGOa?dl=0

[Mac-only weirdness.]

Windows users can safely ignore this post! This is to help out with Mac-only weirdness.

If you are using the new card spanning and sd overclocking features from ilia and Danne you may find that formatting your cards to exFAT with larger allocation block sizes could improve the write speeds. Thanks to @70MM13 for alerting me to this option!

When formatted in exFAT from Mac's own Disk Utility, my SD Sandisk Extreme Pro 64G is given a default block size of 32768 bytes (0.3Mb).

Write speed was then tested on the Black Magic Speed tester, = 51

After formatting to exFAT with a block size of 16Mb, the write speed = 72

Your mileage may vary!

You can format the card to this spec by either doing it in Windows disk management, or use the Terminal in Mac:

Code Select Expand

sudo newfs_exfat -b 16777216 -v NameThePartition /dev/diskNumber

That's all great, but if you now try to mount an exFAT card on the Mac that has been formatted with anything larger than 1048576 bytes (1Mb), the partition will just not mount. No way. So you either need Windows running to empty the cards – which I started doing, but it's a pain. Then I found this project on GitHub: https://github.com/relan/exfat

I think it's based around FUSE and it works a treat! Anyone who has used MLVFS will be familiar with the concept. When commanded it will mount the custom exFAT cards on the Mac with ease!

I do:

Code Select Expand

cd exfat

sudo ./fuse/mount.exfat-fuse /dev/diskNumber /Users/Me/Desktop/mount

Note for when the installation and directions call for being a 'root' user. In Terminal you can start a root session by:

Code Select Expand

sudo -s

Be careful though – and type with care in this mode. This gives you super powers to destroy planet earth – and your Mac entirely!

[For Mac users!!]

A postscript to @70MM13's post: For Mac users!!

I tried to format a custom 4Mb block size in exFAT using the terminal like this:

Code Select Expand

sudo newfs_exfat -b 4194304 -v NameThePartition /dev/disk3s3

It appeared to have done so, but I couldn't get it to mount under any circumstances. The card will work in the camera – but you will be unable to mount it to retrieve your files.

The closest I could get to a larger custom block size was 1Mb (1048576) (These numbers have to be powers of 2 apparently.) This will mount, but it caused problems in the camera and I started to get only about 15sec @ 3072p with or without card spanning/sd overclock on or off.

Not sure if it was related but I then kept getting card errors: 'Card1 cannot be accessed' after formatting custom block sizes from the Terminal. Reflashed the Firmware, finally erased formatted to exFAT from disk utility with the default [tiny] block size of 32768 (0.03Mb).

All appears to be ok again now. But bottom line may be that Mac really doesn't play well with exFAT/exFAT custom block sizes. I wish I'd left well alone! 

PS If you format the cards in-camera I noticed (both my SD and CF cards being 64Gb) that the SD card gets exFAT, while the CF gets FAT32. Just noting...

@Stousen Yes!

In 1080p = Real Time.
In Crop modes = Framing.