Magic Lantern Forum

Using Magic Lantern => Raw Video => Topic started by: kfprod on May 25, 2014, 03:11:18 AM

Title: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on May 25, 2014, 03:11:18 AM
Hi,

The combination of

1. Mirrored external monitoring
2. Playback on set (without having to remove the card)
3. ProRes backups on an SSD (with the RAW files on the card in the camera)


is very close. This is thanks to the possibility for the Atomos recorders to be triggered to record by a HDMI trigger from the camera called REC COMMAND.

This works fine with the camera set to record h264 video but unfortunately it does not work yet when you're recording RAW.

I will make a nice donation to the ML development team if this was to happen, I have a shoot next week that would benefit greatly from this.

Here's how the normal setup works https://www.youtube.com/watch?v=k62B8T5Na3I

Thanks,
Karl
(Long time RAW shooter, here's a video I shot 2 weeks after the raw hack was first released https://vimeo.com/82213325 )
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: mixmastermike on May 25, 2014, 05:38:16 AM
I also have an Atomos device, a ninja blade and the HDMI trigger works great with the camera on H264, but mlv_rec not so much.

it used to work on Raw_rec, but i haven't tried it on recent builds.

Would love to have this feature!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on May 25, 2014, 02:07:56 PM
Thanks good to hear that others have the same need for this feature. If there are more of us please post here and maybe we could get some momentum going!

I just tried it with the RAW_REC module and it doesn't work. Do you mean there was a build with the newest Canon firmware 1.2.3 that worked with the HDMI Rec Command or was it on the previous firmware?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Steven on May 25, 2014, 06:50:01 PM
I need it too!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: mixmastermike on May 25, 2014, 07:09:47 PM
I will have a look through old builds and try to find the exact build that worked with raw_rec. maybe that way we can help isolate the code that changed and maybe one of the programmers can take a look.

If anyone keeps old builds I would suggest you do the same.

This really would be a great feature, you even get proxies recorded with sound, that should mean pluraleyes can easily and quickly sync a high quality external audio recording from a Zoom or the like...
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on May 26, 2014, 03:37:52 AM
Thank you, much appreciated!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: jasondhines on May 29, 2014, 12:24:30 AM
Would be awesome to record Prores files to an external recorder while recording raw video to the CF card when needed. Say when you're filming a wedding ceremony, you could record the entire event in Prores and record the key moments (vows, rings, kiss) in raw. Do you think this will eventually be possible? Can you start and stop the raw recording while still recording a clean HDMI feed?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 07, 2014, 11:13:51 AM
Any progress with the rec command? :)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: pobox on June 07, 2014, 12:22:49 PM
Don't you have full HD output through the HDMI while recording raw with 5d3 1.2.3?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 07, 2014, 03:42:17 PM
Quote from: pobox on June 07, 2014, 12:22:49 PM
Don't you have full HD output through the HDMI while recording raw with 5d3 1.2.3?
Yes but no rec command, whitch would come in very handy with many clips.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Stedda on June 07, 2014, 03:50:13 PM
Quote from: Susteren on June 07, 2014, 11:13:51 AM
Any progress with the rec command? :)

No Dev committed to doing this for you. Generally that means they have no interest/use in it for themselves and if you want it the burden is on the interested parties.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 07, 2014, 09:45:53 PM
Quote from: Stedda on June 07, 2014, 03:50:13 PM
and if you want it the burden is on the interested parties.
I'm sorry but I don't understand what you mean by this. :)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Midphase on June 08, 2014, 01:22:45 AM
Quote from: Susteren on June 07, 2014, 09:45:53 PM
I'm sorry but I don't understand what you mean by this. :)

What that means is that if you want it, you better roll up your sleeves and learn how to code.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 08, 2014, 05:58:09 PM
Ok that's too bad thanks for the replies.

Again if there are more users ending up here please post in the thread and maybe we could still get some momentum going!

Not giving up... I have a background in coding and I would try to write the code myself for this specific thing but the notes about possibly breaking the camera while testing turned me off... Maybe I'll buy a 50d to start learning on that, can't really afford to break a 5d mk3.

There are no guns for hire amongst the incredible wonderful intelligent programmers?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: a1ex on June 08, 2014, 06:14:25 PM
What you need here is to reverse engineer how Canon sends the command to the recorder (or, if the recorder figures it out by itself, to know what it's looking for - display mode change, red dot or whatever).

Without reverse engineering, the easiest way would be to record RAW and H.264 simultaneously. This one fits into easy coding tasks.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 08, 2014, 06:31:49 PM
Thank you for the reply and thank you for the amazing work you're doing.

Not sure I understand you correctly but if there's a way to write the RAW video file to the CF card and the H.264 file to the SD card (and also trigger the Atomos this way) that would be amazing.

That's a feature many people would have turned on anyway even without the Atomos connected as that H.264 file would playback in realtime on set, it would be a backup, it would have the sound in sync and it would be a perfect offline editing file that you'd just reconnect to the DNG files in DaVinci Resolve once you're done editing. This way you wouldn't have to transcode anything else than the shots you're actually using. That would save a lot of time in post.

From what I understand the RAW video began when you piped the raw feed that went to the internal H.264 encoder straight to the CF card. If there's a way to forward it to the H.264 encoder and then that resulting file to the SD card (after it's written the raw/mlv file to the CF card) it would be happy days for all users!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: a1ex on June 08, 2014, 06:39:33 PM
Didn't actually try it, but I still believe it fits on the easy coding task list.

The easiest way should be to select the SD card from Canon menu (since H.264 will use that), and force RAW/MLV recorder to use the CF drive regardless of the current card choice.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 08, 2014, 06:56:36 PM
Sounds perfect.

I understand the basics of this but setting up the coding environment and learning from scratch would be very hard for me. I'm on a Hackintosh that doesn't run Mavericks. And I'm scared of messing up and breaking the camera. So I'm at your mercy and as I said in the beginning of this post if anyone could get this working I'll make a nice donation to the ML development team!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Midphase on June 08, 2014, 07:07:58 PM
Quote from: kfprod on June 08, 2014, 06:56:36 PM
as I said in the beginning of this post if anyone could get this working I'll make a nice donation to the ML development team!

I'm always puzzled by this 'implement-something-I-want-and-I'll-give-you-a-donation' tactic. I think it's kinda backwards, in the sense that the developers aren't doing this for money, but the money does go toward purchasing gear that they can test on. In this case perhaps an Atomos Ninja. By withdrawing your money until after the fact, you're kinda negating progress to be made in the first place by not allowing them to go out and purchase the very thing they need to realize your request.

If your thought about a "nice donation" is meant as a monetary reward for the team of developers, so that they can take their kids to Disneyland, I seriously doubt that the amount you'd be willing to contribute would go very far toward improving their financial welfare.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Midphase on June 08, 2014, 07:10:28 PM
Quote from: a1ex on June 08, 2014, 06:39:33 PM
Didn't actually try it, but I still believe it fits on the easy coding task list.

The easiest way should be to select the SD card from Canon menu (since H.264 will use that), and force RAW/MLV recorder to use the CF drive regardless of the current card choice.

Hello Alex,

Would this task the CPU too much? I actually never thought that it'd be possible for the camera to handle recording simultaneously h.264 and raw, but I will say that if such a thing could be implemented, it would be a fairly major game changer and confidence booster for many videographers. Cool, I didn't know it could possibly be done.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 08, 2014, 07:13:40 PM
Midphase: I agree, you're right regarding donation. I'm sorry. Couldn't come up with anything better to put this on someones agenda.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: a1ex on June 08, 2014, 07:17:25 PM
Since the H.264 encoder is a hardware chip, I don't expect a significant performance drop. Here's a hint from somebody who tried: http://www.magiclantern.fm/forum/index.php?topic=5860.msg42534#msg42534

It even looks like this was a bug in some versions: http://magiclantern.fm/forum/index.php?topic=10259

So, definitely a task for anybody who wants to jump in and code something easy.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 09, 2014, 02:27:37 PM
Wow, recording h264 and raw IN CAMERA would be totally awesome! If that's possible an external recorder isn't even needed anymore for video playback! :D
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 09, 2014, 03:27:25 PM
It really would be great and you're right the external recorder is not needed if it works... Should have bought the SmallHD dp7 instead for monitoring.

Trying to get going with the dev environment and I have it all set up but it's been 7 years since I did any programming... So it'll take a LONG time if it's left up to me (but hopefully not 7 years). I know how to set stuff up with static CONFIG_INT and menu_entry raw_video_menu[]
as a starting point but then the main idea of forcing the raw to the CF card without touching the h264 going to the SD card will take a while for me to figure out... Hoping for someone more experienced to chime in!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 09, 2014, 04:47:22 PM
I would be so glad if I could help with programming, but I only know HTML and PHP. :(
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 09, 2014, 05:59:47 PM
While looking for how to do this I found a thread where this was discussed previously (2014-01-19) "With this, one can handle easily the case where you want to record both raw and h264, for example." Can't really tell if anything was implemented though

https://bitbucket.org/hudson/magic-lantern/pull-request/336/1100d-fix-recording-macros/diff

Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: dmilligan on June 09, 2014, 06:39:39 PM
that's simply about handling the status of "recording"/"not recording", not the actual recording itself
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 11, 2014, 10:48:36 AM
Any updates? :D
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: a1ex on June 11, 2014, 10:57:09 AM
Yeah, here: http://www.magiclantern.fm/forum/index.php?topic=934
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 11, 2014, 12:01:34 PM
I have a fairly good idea about how to handle the separation of raw/H.264 files to different cards and to listen for the trigger but I can't for the life of me figure out where the H.264 gets turned off and taken over by the RAW recorder.

RECORDING_RAW takes over RECORDING_H264 somewhere, is it recording / recording custom that can't be active at the same time? (edit)

Any pointers appreciated, I'm such a noob.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: a1ex on June 11, 2014, 01:45:38 PM
In the key handler from the raw recorder.

Key events get sent by Canon firmware (or maybe faked by ML), and they are intercepted in the GUI task (gui.c) by ML key handlers. These handlers will decide whether the event is interesting for them or not (so they might do some action when they receive a certain event), and they also decide whether the event should be sent to the next handlers (the last one being Canon's own button handler). To allow an event to be handled by other routines (including Canon's), a key handler should return 1; to block it, should return 0. The order in which ML key handlers are called is hardcoded in gui-common.c.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 14, 2014, 08:57:37 PM
Thank you,

So I actually managed to get the camera to write both RAW and H264 to different cards. Unfortunately the camera freezes after about 12 seconds when using these settings and I don't understand why. It's not about the screen size that gets put through the raw module: I tried both 640x360 and 1920x1080 but no matter the setting the camera freezes.

What I've tried is:

    if (rec_key_pressed)
    {
        switch(raw_recording_state)
        {
            case RAW_IDLE:
            case RAW_RECORDING:
    raw_start_stop(0,0);
    return 1; //Lets the camera carry on with what it was doing.


What could be the problem, not enough computing power?

I used ( filename[0] = 'A'; ) to make sure the RAW always goes to the CF Card and I'm setting the H264 to go to the SD through the normal Canon menu.

Still looking for either the solution to write the H264 to the SD card and the RAW to the CF or to find the HDMI REC trigger so the Atomos can auto start recording (and forget about the H264).

Also added seconds to the file name for my own module, that gives a smaller chance of same file names when shooting 2 cameras. Would be better to have a hex file name structure as RED has. When using MLV Mystic and RAW Magic the programs crash if the file names are the same. I'll write a separate post about that.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 16, 2014, 04:14:26 PM
Possible breakthrough:

I found this string
ff1936c8 PROP_TIMECODE_HDMI_REC_COMMAND (%d)

(Also:)
ff1f18b4 PROP_TIMECODE_HDMI_REC_COMMAND:%ld
ff75bd50 DlgMnTimeCodeHdmi.c IDC_DPM_REC_COMMAND
ff75bf48 DlgMnTimeCode.c PROP_TIMECODE_HDMI_REC_COMMAND

In ROM1.BIN.strings converted from the ROM1.BIN of the 5d3.123 release.

Can someone please help me turn this into a function that I can trigger?

Thank you for the tutorial a1ex you should make it a sticky: http://www.magiclantern.fm/forum/index.php?topic=12177.0
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 16, 2014, 05:13:55 PM
Something like:
prop_request_change(PROP_TIMECODE_HDMI_REC_COMMAND, &v, 1);

I'd love to give it a go... Only this comment scares me off.
/** Properties are persistent (saved in NVRAM) => a mistake can cause permanent damage. Undefine this for new ports. */
/** The 5D3 port is young, but... let's give it a try! **/
#define CONFIG_PROP_REQUEST_CHANGE

Please help!!

Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: a1ex on June 16, 2014, 07:45:02 PM
Nice find. These seem to be from some GUI code which I don't quite understand. However, if you look at ff75be7c, you can extract some property numbers (easier if you have a decompiler):


0x8003002C PROP_HDMI_PHYSICAL_CONNECT
0x80040056 PROP_TIMECODE_HDMI_OUTPUT
0x80040057 PROP_TIMECODE_HDMI_REC_COMMAND


This code seems to react to these properties, but it doesn't look like a prop handler; instead, it's in the middle of Set***ToWinSystem (which I believe it's used to print GUI symbols).

Let's see how Canon calls it (look for 80040057 => ff19d8ac). The code is roughly like this:

uint32_t aux = arg0;
prop_request_change(0x80040057, &aux, 4);


The above code is called from CtrlSrv task I think (where most Canon GUI stuff happens) when some GUICMD event code is 0xB IDC_DPM_REC_COMMAND (these are not our usual button codes, but one layer above that). The REC button is defined as 0x1E in gui.h, which gives a GUICMD code = 0x1000000f (at ff1a7ddc+8+0x1e*4). So, I'm not exactly sure what triggers it. Anyway, the argument (arg0) seems to be *0xFF9E898C, which is 0.

It's unlikely that our property would be a permanent setting in NVRAM (it looks more like something that triggers an action), so I'd say it's worth trying (but that's not a guarantee). If g3gg0 confirms my analysis, it's probably safe.

I've tried the following code on my 5D3 (don't click me), without a recorder attached, and the property handler did not trigger.


PROP_HANDLER(0x80040057)
{
    NotifyBox(1000, "%d ", buf[0]);
}

static void run_test()
{
    msleep(1000);
    uint32_t aux = 0;
    prop_request_change(0x80040057, &aux, 4);
    beep();
}
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: mixmastermike on June 16, 2014, 09:21:53 PM
Really encouraging, thanks to @ kfprod and @ a1ex for the efforts being made
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Chucho on June 17, 2014, 05:43:31 AM
A couple of weeks ago I was playing with the HDMI functions on the 70d. The 70d has PROP_TIMECODE_HDMI_REC_COMMAND (0xff19f6d8) PROP_TIMECODE_HDMI_OUTPUT (0xff19f5b4) PROP_DUAL_OUTPUT (0xff19b6c8) (mirroring) and PROR_HDMI_FLAME_RATE (0xff19b6a8) (HDMI frame rate). The 70d is missing the GUI function that display the HDMI menu but with some simple code you can achieve the desire outcome. Maybe debuging the TIMECODE HDMI rec command will give you some usefull information. Debug(GUI, 3, "PROP_TIMECODE_HDMI_REC_COMMAND (%d)", r4 + 0x4)); in cause of the 70d r4 is 0x98E30.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 17, 2014, 09:40:05 AM
I love that the firmware coming out of Japan has FLAME_RATE instead of FRAME_RATE  8) Lock and LoLL!!!

It would be good to hear from g3gg0 if he confirms a1ex's analysis that it should be safe to play around with and I'll do that.

A1ex is that going through and checking all the different states of PROP_TIMECODE_HDMI_REC_COMMAND? In prop_request_change(0x80040057, &aux, 4); would it make sense to step through &aux and 0-4 or does the test function do that?

Could also be that because you don't have anything connected to the hdmi port it just won't trigger, maybe just putting a cable in to a TV (or at least just putting a cable in) would do the job and give you that triggered NotifyBox?


Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 17, 2014, 07:37:46 PM
Great guys, keep on goin'!  :)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: actingnurse on June 18, 2014, 08:49:47 AM
This is awesome! I'm putting the purchase of the Atomos Ninja Blade on hold for a few until I see what develops here... If we could get realtime playback with audio by shooting the H264 file to the SD slot then COUNT ME IN!!! Although, the blade still has the benefit of recording 10bit footage (not as good as ML's 14bit, but better than the H264's 8bit) as a backup in case of card or file corruption through ML or card issues....   Keep up the good work! Ya'll rock!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 18, 2014, 09:38:54 AM
It's not 10 bit but 8 bit, that is what the Canon camera HDMI outputs. It is however 422 instead of 420 and it has never been through an h264 codec so that's good.

But if someone does get the h264 simultaneous recording to work without problem I'd suggest buying the SmallHD dp7 instead of the Atomos though, it's a better monitor and you'll only use the h264 files as proxies anyway. Unless you do a lot of interviews where you can't shoot raw, then the benefits of the recorder will be worth it.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 19, 2014, 01:26:34 AM
So I've tried this with the recorder attached but as you say it doesn't trigger.

PROP_HANDLER(0x80040057)
{
    NotifyBox(1000, "%d ", buf[0]);
}

static void run_test()
{
    msleep(1000);
    uint32_t aux = 0;
    prop_request_change(0x80040057, &aux, 4);
    beep();
}


What about the others

ff1f18b4 PROP_TIMECODE_HDMI_REC_COMMAND:%ld
ff75bd50 DlgMnTimeCodeHdmi.c IDC_DPM_REC_COMMAND
ff75bf48 DlgMnTimeCode.c PROP_TIMECODE_HDMI_REC_COMMAND

Could any of them be used? I don't have a decompiler set up unfortunately. Most of this is really over my head  :(

I tried a couple of other ways to do the raw to the CF card and h264 to the SD but it always crashes after a while.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 29, 2014, 04:15:48 PM
Back from a job and doing some tests again. When I do

prop_request_change(0x80040057, &aux, 0);

I get a red box with PROP_LEN(80040057) and

ML ASSERT:
PROP_LEN(80040057) = 0
at ../../src/property.c:325 (prop_request_change), task GuiMainTask
lv:1 mode:3

This is not the same as earlier. Perhaps it's only the error handling code that has changed?

I'm still wondering if it would be worth it to try and do the same test with these values (converted to 0x800...) but I can't decompile ff1f18b4 -> 0x800...

ff1f18b4 PROP_TIMECODE_HDMI_REC_COMMAND:%ld
ff75bd50 DlgMnTimeCodeHdmi.c IDC_DPM_REC_COMMAND
ff75bf48 DlgMnTimeCode.c PROP_TIMECODE_HDMI_REC_COMMAND
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Susteren on June 29, 2014, 11:44:04 PM
So you got an error, the h264 didn't start recording?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: a1ex on June 30, 2014, 08:42:23 AM
To avoid this error, place a PROP_HANDLER (even if it's empty). It's a safeguard.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on June 30, 2014, 05:00:21 PM
I did put in the PROP_HANDLER with a Notify Box, it was there all the time..
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on July 03, 2014, 05:59:08 PM
Great that this is looked at, it is the missing link for a good RAW / Playback workflow, if no code can be found, perhaps atomos could give a clue about the command.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on August 07, 2014, 11:22:39 AM
This may help: Atomos have released their HDMI START/STOP TRIGGER & TIMECODE standard as open source. The Atomos protocol for HDMI is available for free from www.atomos.com/hdmi-protocol

I received the PDF by just filling in the form. Unfortunately "Neither the whole nor any part of the information contained in, nor the product described in, this document may be adapted or reproduced in any material form except with the written permission..."

So sign up there if you want to have a look and you'll get it within 24hrs.

On another note the Atomos can trigger by just listening to if the camera outputs a REC RUN hdmi time code (not using the REC COMMAND) so if we could just make the camera output that instead that would also work...

Regarding the other part of what I've been testing, writing h264 to the SD card and RAW to the CF card, there is something that happens after 12 seconds and the camera crashes. Is that the processor that's overused? Perhaps someone could look into this. If you do just set

            case RAW_RECORDING:
                raw_start_stop(0,0);
                break;

to

            case RAW_RECORDING:
                raw_start_stop(0,0);
                return 1;


and change the normal camera setting to write to the SD card but the RAW file to the CF card. This is the quickest way


        //if(card_spanning)
        //{
            filename[0] = 'A';
        //}


If we could get this to work this would also solve the Atomos recording as return 1 will let the normal camera h264 recorder get on with what it was doing with all the HDMI stuff as well.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on October 17, 2014, 01:59:06 PM
Is anyone looking into this already?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: beasone on December 09, 2014, 10:27:23 AM
Any updates?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on December 09, 2014, 12:18:12 PM
Looking for this thread for a long time.


the h264 and raw recording would make a world of difference.

Or just a Rec command signal.

In the first months of ML raw I am very sure I read some thread where some guy was asking why the Rec command didn't work anymore, he said he had been using Atomos and Raw rec together for quite some time until some nighly build made that option unavailable.

Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on December 11, 2014, 08:00:54 PM
Do you mean this post at the beginning of this thread?
http://www.magiclantern.fm/forum/index.php?topic=12022.msg116549#msg116549 (http://www.magiclantern.fm/forum/index.php?topic=12022.msg116549#msg116549)

Here is the protocol , but the strange thing is the protocol to send the REC command over HDMI is already in the 5D, it's just not triggered for raw, only h264.
This would mean something part of the signal is stripped by using the RAW recording function right?

http://www.atomos.com/hdmi-protocol/ (http://www.atomos.com/hdmi-protocol/)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on December 15, 2014, 01:24:14 PM
Good to hear that more people are requesting this. Unfortunately I'm just a hacker and not a proper programmer so I can do some things but I haven't been able to get this to work- although I have tried all ways I could think of. If this could get some attention from the "real" programmers here that would be fantastic!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on December 15, 2014, 10:04:08 PM
Oh I suppose that post must be the one I meant! Not sure if he is mistaken or it did really work in the early builds of raw 1.0

When doing a one man show shooting and recording sound, I still struggle a lot to get sound and video to work together as one. It is the biggest downfall of the ml raw workflow.

Sometimes I wish I had a trained monkey to clap a slate on every shot I do.

Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on December 16, 2014, 12:13:17 AM
If the record trigger would work, then we have a perfect workflow!   8)
Record in camera raw and record external prores.

Would be easy to sync since they have the same length. 
For a fast turn around edit with proxy files and later use the raw for grading.


There has been quite a bit of work done on the HDMI code as can be seen in the highly builds,
The thing is that Alex is very busy at the moment and has not so much time to look into it, I'm hoping that early next year when Alex has more time this HDMI trigger code will be made to work.

d16f504 | a1ex <[email protected]>
Merged in external-monitor-backport (pull request #576)
9d16556 | Alex
Fix conflict with qemu branch
6ebe96d | Alex
Undo 73a7158 (not needed)
73a7158 | Alex
Cropmarks: fix right border disappearing
caee312 | Alex
Raw backend: fine-tuned raw buffer alignment (pixel-perfect match on 5D3)
fa3d4cb | Alex
mlv_play: fix issue 1771 (borders in HDMI mode)
ab16f73 | Alex
mlv_play: fix issue 1771 (borders in HDMI mode)
8e9dbb7 | Alex
Merged external-monitor into 5D3-123
b3b4b10 | Alex
Fix merge conflict
a2a3182 | Alex
5D3-123: changes from cleanup branch
3528f67 | Alex
Merged unified into 5D3-123
6805a0e | Alex
Raw backend: fix raw2lv translation component (tested on 5D2)
1003f8a | Alex
Raw backend: fix raw2lv test
7ccc830 | Alex
vram: after updating display parameters, force a refresh of RAW backend parameters
cf3b1a6 | Alex
update_vram_params should be private
6b9f9f2 | Alex
Cropmarks: fix redraw on external monitors
1d5994c | Alex
External monitors: check the size of PROP_HDMI_CHANGE_CODE
eb1f5ad | Alex
Revert debug code
3f20640 | Alex
External monitors: fix LV buffer display address for 5D3 1.2.3 in HDMI 1080i
495f855 | Alex
External monitors: recognize hdmi_code == 20 as HDMI 1080i (for 5D3 123)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: dubzeebass on December 16, 2014, 01:06:13 AM
Because of the extra latitude shooting raw video, wouldn't the prores proxies recorded into the Atomos be over exposed because they don't have the same latitude to overexposure as the raw footage? In addition wouldn't the ninja Blade scopes be unreliable because they were in a different color space than raw?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on December 16, 2014, 02:32:53 AM
Prores will struggle if you ETTR. But ofcourse they don't contain the same information. HDMI outputs a 8 bit 422 signal in to a 10 bit codec.

But the proxies are great and Mic jacked to atomos triggered by rec command, sync issues over or so I hope :)

Or else I better start training that monkey..
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on December 27, 2014, 06:48:19 PM
An extra h264 on the SD card next to the raw on the compact flash with a record trigger to the atomos via time code would be ultimate :)


and off course audio over hdmi, don't understand why canon did not implement this.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: dubzeebass on December 28, 2014, 03:24:36 PM
Today I use the Blade in conjunction with MLV for the first time on a commercial. Hoping for the best!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on December 28, 2014, 09:33:43 PM
Dont forget to press record on both devices :)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: DeafEyeJedi on December 29, 2014, 01:38:18 AM
How'd it go for you @debzeebass?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: dubzeebass on December 30, 2014, 04:52:25 AM
I ended up shooting flat on the NB and using that footage over the raw. It looks as good because I lit properly, shot with Zeiss glass in the sweet spot, and the spokeswoman was a knockout.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: dubzeebass on December 30, 2014, 07:28:14 PM
(http://s30.postimg.org/618500gwh/Untitled.png)

A bit compressed by the website I posted it to, but with proper light, lenses, and focus, raw becomes secondary if you have broadcast 422 and know your way around shooting flat and postproduction.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 01, 2015, 05:45:11 PM
didnt notice this thread.

ive reverse engineered parts of the HDMI CEC stuff that the 5D3 has implemented for fun.
are those atomos commands CEC style?
does the atomos accept commands to start recording initiatied from canon side?

as i do not want to register on this site, can one send me the atomos spec?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 06, 2015, 02:59:22 AM
okay seems the atomos doesnt use the CEC commands for that task.
maybe it still accepts CEC commands?

if there is a 5D3 v1.1.3 user with an atomos (or any other HDMI device) out there, can you test that (http://ml.g3gg0.de/modules/cec_test.mo/cec_test.mo) module?

try to send source 0x3, dest 0xF, command 0x0F without payload.
if your atomos records, than ML can automatically do that on recording start.

you can browse http://www.cec-o-matic.com/ for the commands that are specified.
enter payload in this format: "00 11 AF" etc (IME modules: ime_base (http://ml.g3gg0.de/modules/ime_base.mo/ime_base.mo) ime_rot (http://ml.g3gg0.de/modules/ime_rot.mo/ime_rot.mo)) but only provide EXACT the amount of bytes that the standard tells you.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 06, 2015, 05:00:06 PM
a quick test with a samsung smart tv allowed me to power off the TV using that module. (command 0x36)
this means basically it is working. so i am wondering which devices also support HDMI CEC.

in theory we could use the camera as HDMI based remote control :)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on January 06, 2015, 05:13:40 PM
Great progress g3gg0!

I have a ninja to test, I will have a look at it tonight.
Here's a setup how it normally works with h264

https://vimeo.com/76582803 (https://vimeo.com/76582803)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: tonybeccar on January 07, 2015, 01:58:27 AM
Wow! I am wondering which kind of commands are available to send via HDMI.. ! Imagine selecting which overlays are sent to the HDMI feed... that simply makes me drool all over hehe.

Awesome work!!
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 07, 2015, 01:37:45 PM
see this: http://www.cec-o-matic.com/

there you can forge the commands.

*if* there is some *serious* interest in automating things etc. i can make a plugin that allows you
to specify some commands in e.g. text file format that can be executed from menu. (or on startup or other events)


Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on January 07, 2015, 02:17:01 PM
For me the most serious application would be having the Record command send to the Atomos ninja, when using the raw video on the 5D mark 3.

If there is anything I can do to assist in that let me know. I'm not completely sure how to proceed.

The first thing I would test if the timecode can be activated from the 5d during raw recording. 
I think the ninja can possibly get triggered to record from timecode.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: kfprod on January 07, 2015, 02:35:57 PM
I think that as Lars says the record function is the most important. If there's any way of playing back the latest clip on the Atomos that would be amazing too (but that's probably not part of this spec)...

Sorry but I don't have the 5d camera at the moment, it's used in production. Lars do you have yours available to test this? I'll get mine back later this month.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on January 07, 2015, 04:48:06 PM
Yes I have a Ninja blade and 5d mk3 to test.

I'm not a programmer however I can try to run a few commands from the camera and report back what the results are.
I would need a simple instruction how to do it
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 07, 2015, 05:58:00 PM
install that (http://ml.g3gg0.de/modules/cec_test.mo/cec_test.mo) module
install IME modules: ime_base (http://ml.g3gg0.de/modules/ime_base.mo/ime_base.mo) and ime_rot (http://ml.g3gg0.de/modules/ime_rot.mo/ime_rot.mo)

open debug menu, "HDMI CEC Test"
Source 0x3,
Destination 0xF
Command 0x0F
and press "Send"

Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on January 07, 2015, 10:35:21 PM
I have downgraded the 5d to 1.1.3 and ran the tests, no response from the atomos ninja blade.

(EDIT )The command 0x0F is not in the list or I can't find it, It seems like maybe I ran the wrong command?

I also did 0xF Record TV Screen, but with no effect ( not in the video )

Would it not be better to test it on 1.2.3?

Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 07, 2015, 11:05:09 PM
thanks!
when you select the command id, you can also choose 0xF: Record TV Screen
can you try this?

if that doesnt work, the atomos doesnt support CEC :(
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on January 07, 2015, 11:20:14 PM
Yes I tried "0xF Record TV Screen" after I saw my mistake, still no record on the ninja.

I hope there is still some way to make it work ...   8)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on January 07, 2015, 11:29:20 PM
Awesome work G3gg0!

Thank you for looking in to this!

I don't have a Atomos now and most likely wont for quite some time now, but I do have a Samsung TV, so I will try turning it off and what ever else is possibly possible.. Smart Remote 2015

I am on 113 btw
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 07, 2015, 11:48:40 PM
yeah from the atomos specs, the use a totally different signalling scheme than CEC.
they use the packets that get embedded into the video signal, just like audio etc.

CEC is a separate signalling channel and has nothing to do with that.
unfortunately the atomos doesnt parse that CEC commands :(
so thats a dead end.

still, smart tv's can be controlled using the camera now ;)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on January 08, 2015, 12:00:55 AM
How about if you would enable h264 on the sd card and raw on the compact flash simultaneously.
The h264 should trigger the Atomos record right? even when the h264 would have a very low bitrate.

This would only work on firmware 1.2.3
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on January 08, 2015, 12:03:57 AM
Too bad about CEC not being the way..

So its audio signal or beep that starts the Atomos' recording?

I know this doesn't further the cause of Rec command, but could you point me in the right direction for TV remote MK III.

I set Source as Tuner 1

and destination as TV

But I couldn't get the TV to turn off or do anything.. Tried every command that had a description and some random unknowns.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 08, 2015, 12:12:27 AM
its not a beep that triggers recording.
the HDMI signal has various packets that can be sent along with the video.
audio frames, info frames, encryption stuff etc.

and the timecode and record flags are embedded into info frames.

i tried it with my samsung smart tv and i could power off it using command 0x36.
please when sending data, use the exact number of payload bytes that is being shown on cec-o-matic i linked above.
if you forget to send payload, the TV will throw away the packet.

(command 0x36 does not need any payload. command 0x0f also doesnt need one)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on January 08, 2015, 12:33:39 AM
ok, seemed too simple to be a beep..

Though I think the Tascam DR 60 listens for a beep signal to start recording, but ofcourse that is Audio related aswell..

Anyways.. I tried 0x36 and tried switching destination to Broadcast and every other, but I cant get it to work, Maybe its the TV model .. I got Samsung tv too.

0x3
0x0 and 0xF and everyother
0x36

Where exactly do I see the required payload? in the numbers that briefly are shown when sending ?

PS. Took me awhile to figure out how to delete the payload, for anyone else trying this, use Joystick to go back and then delete the numbers.

Thanks anyways
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on January 08, 2015, 07:38:53 PM
Thought I found the culprit a moment ago..

Was going through Canon Menu and noticed under the Play tab that Ctrl over HDMI was disabled, enabled it but alas no difference in sending commands to TV, I couldn't turn it off.

But as I don't recall ever disabling this option, I think maybe its standard and Lars could look in to it on his camera and try it with his Atomos again.

@G3gg0, do you have it enabled?

Though honestly I am not sure what "Ctrl over HDMI" does, but I can only imagine it means Control.

Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 08, 2015, 09:38:43 PM
yeah, i have the control over hdmi enabled.

if you connect your camera over HDMI, does your TV show its name when choosing source?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Kharak on January 08, 2015, 10:12:35 PM
Eureka !

Well not really.. but I managed to turn off the TV via HDMI.

Under Applications I could see 5D MK III Menu and 5D MK III Info, under setup Anynet+HDMI (CEC) Auto Turn off was off. Has to be ON in order to let devices request shut down.

But no, I could not see the Samsung name in TV source.

and under CEC-o-matic I cant seem to find where it says which payload to send?
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: g3gg0 on January 08, 2015, 10:19:24 PM
haha no i mean the TV should show the camera's name :)
the module i made will not show any name.

in cec-o-matic you enter some stuff and if you press the arrow up on the right side, the data bytes in the top line get updated.
the first byte is the source+destination pair. the second byte is the command ID.
if there are 3 or more bytes, thats the payload. just skip the first two bytes and enter it into the payload dialog.

the short popup telling you what data has been sent, only shows you one payload byte (or zero if there was no payload)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: beasone on January 30, 2015, 05:28:00 AM
I'm preparing for a feature film shoot, and this feature would really help with our work flow.  It's time consuming to load the raw footage into the computer and watch MlRawViewer.  This way we'd have immediate playback.  8)
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on February 16, 2015, 12:36:46 PM
Yes It's a great thing to have, I read in the atomos site that the recording can also be triggered with timecode, if we can have MLV recoding with timecode it might work already.
Title: Re: REC COMMAND via the HDMI trigger - Atomos
Post by: Lars Steenhoff on August 27, 2017, 05:55:46 PM
Update

With the new h264 proxy recording
https://bitbucket.org/hudson/magic-lantern/commits/71a6676c01590cb83cccde9d39d5ff6ffbd24215 (https://bitbucket.org/hudson/magic-lantern/commits/71a6676c01590cb83cccde9d39d5ff6ffbd24215)

Its possible to record Raw to the compact flash, h264 to the SD and trigger the external recorder from the h264 signal.
I just tested it with a canon 5d mk3 and Atoms ninja blade.