Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)

Started by g3gg0, July 15, 2013, 10:58:23 PM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

g3gg0

Quote from: AnotherDave on July 31, 2013, 04:19:27 AM
Tonight's build is far slower for me...  I'm only getting 81.5MB/s 54ms idle, 4000-7000 frames expected,  takes a lot longer to works it way up to the speed too.

5D3, KB 64 1000x, 24fps, 1080p
thanks. what is "far slower" in numbers?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

g3gg0

Quote from: Redrocks on July 31, 2013, 02:22:23 AM
Can you recheck the files for me or have I messed up? I put all your new .mo files in the modules folder, left the .bin in the root and deleted the .sym. Then the camera refused to fire up. Tried the build on three cards and ended up having to insert a clean card whilst the camera was switched on to roll back to the latest regular nightly.

this is for 5D3 :)
you just need the module (i assume you have a recent core build)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Redrocks

tcc: error: undefined symbol 'msg_queue_count' [E] failed to link modules

This is with only the .mo files placed in the module folder of a.d's 728e571a1276. Can't see how I'm doing things any different than usual, but I'll see if there is something I've overlooked.

g3gg0

no, this symbol was introduced a few days ago.
thats the reason why i said you should use a recent ml core ;)

edit: ah forget it, i just looked up that symbol for 5D3. would have to add 5D2 first.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

AnotherDave

Quote from: g3gg0 on July 31, 2013, 01:10:41 PM
thanks. what is "far slower" in numbers?

Latest ML RAW 1 - 94.5MB/s always continuous ok

Latest ML RAW 2 - 83.3MB/s, 573ms idle

I recorded 2 mins of footage, and videotaped the display.
Here is a link to the display screen:
https://www.dropbox.com/sh/wvimd6cdqel7m3b/2Dp3_iXYyR

Hope that helps...

g3gg0

is this one any better regarding performance?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

gnarr

I got pretty much the same performance on my 5D2 yesterday as with the old format. Maxed at 76MBps.


g3gg0

here is some special build for the 5D3 with spanning enabled.
can you please try with your fastest SD and fastest CF cards, if you see any speedup over the version in my post before?

-> the output files will for sure be buggy, its just about writing speed

thanks!
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

AnotherDave

It did something!

it always says continuous ok now...

and it says the write speed is 94.1MB (~80.4mb)

it doesn't display the name of the file it is recording though...

ilguercio

Speed seemed the same on my 6D, as far as i could tell.
Haven't tested it properly yet, though.
Canon EOS 6D, 60D, 50D.
Sigma 70-200 EX OS HSM, Sigma 70-200 Apo EX HSM, Samyang 14 2.8, Samyang 35 1.4, Samyang 85 1.4.
Proud supporter of Magic Lantern.

g3gg0

the last build has these changes:
- files are stored in card root folder
- two writer threads are started
- one thread writes to SD, the other to CF card
- one file is named .MLV, the other .M00
- frames are saved on either the one or the other card
- the order of the video frames in the .MLV / .M00 is random (well, tbh it is not random but depends on card speed)

in technical words:
- i have set up a file spanning mechanism over SD and CF

in non-technical words:
- increasing data rate by using SD and CF in parallel



the good thing:
- write rate is obviously faster than before
the bad thing:
- write rate on CF only is not as fast as alex' optimized version, but i am trying to improve that.


ah, one more (obvious, but worth mentioning) conclusion:
- this is only possible on 5D3 ;)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

pravdomil

nice, soon I'm going to port it on 5D2. Maybe if you will reach some milestone?
I want to solve the CF speed issue.
What is the write speed CF+SD? 100+?
Do you want some help with coding?

g3gg0

i cant get beyond 70MB/s with my crappy card.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

Danne

Will try to find a faster sd card. Awesome work by the way!

g3gg0

Quote from: pravdomil on August 01, 2013, 10:55:44 AM
What is the write speed CF+SD? 100+?
Do you want some help with coding?

maximum possible: 120 + 20 as alex said once.
but when you have computers, 1+1 is never 2 but 1.8.
so its less than 140MiB/s and more than 120 MiB/s under ideal circumstances.

we will see if theory matches results :)

help is always appreciated in this project - why ask? ;)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

dadinio13

Quote from: g3gg0 on July 31, 2013, 01:07:04 PM
why did you specify dng ? thats a raw file.

hm will check later, maybe a copy paste error and my compiler didnt warn?
sorry I was tired I did not realize that I used dng2raw instead of raw2dng   :-\
anyway, i still can't open the .raw file from the mlv dumper :

MacBook-Pro-de-david:~ david$ /Volumes/FREECOM\ HDD/Wine/drive_c/RAW/WORKFLOW/raw2dng /Volumes/FREECOM\ HDD/Wine/drive_c/RAW/WORKFLOW/out.RAW
Error: This ain't a lv_rec RAW file


what version of arm none eabi are you using please?

AnotherDave

Does it have to write to both the SD & CF?  I know it will speed things up, but that also makes things even more complicated.

I love the work that everyone is doing with the hack, and to be honest... I think the quality of the images being captured is higher than most professional cameras.

Because the quality is so high, I'm hoping for a simplification of the process of recording in and syncing to raw with the ML 5D3 with version 2.  I want to be able to shoot with it to get amazing image quality... but I also want to be able to be hired out with my camera to shoot in raw.

The more complicated the process gets, the less likely it is that someone will hire a DP with a hacked 5D3.  The more technical knowledge you need to operate a hacked 5D3, the more it is going to be looked at as a "work around" or something "not professional".

The results of this are VERY professional, and I think the format is in the right place at the right time.

But for it to be successful, it really needs to record 1 clip to the CF card that contains ALL the possible information that is gathered while shooting.  When it is opened on a computer, it needs to look as similar to what was shot in the field as possible.  And, syncing the audio needs to be accomplished in 1 click via timecode.

But all of those things together... and you've got a cheap Red camera, and everyone wants to use it.  Continue to make it hard to work with... people will buy or request a BMCP for shooting.


Danne

I for one like the idea with filespanning between cf and sd. Better ratios for slowmo is one benefit. Really cool work

g3gg0

Quote from: AnotherDave on August 01, 2013, 01:39:49 PM
Does it have to ...I'm hoping for .... I want to... I also want to... it really needs to... audio needs to...

uhm, what?
did you forget that this feature is still EXPERIMENTAL?
we still try to figure out how far can we go and what is possible.

i am happy about any recommendation about usability, but please leave us enough room for experiments.
and as the first post says, it IS an experiment.
what will be in the ML release is another story.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

pravdomil

Quote from: AnotherDave on August 01, 2013, 01:39:49 PM
Does it have to ...I'm hoping for .... I want to... I also want to... it really needs to... audio needs to...

g3gg0 please implement 500fps to raw_rec I really need that feature to capture my breakfast also I need sound recording if something horrible will happend
but seriously - one more reason why dont share unfinished features

Quote from: g3gg0 on August 01, 2013, 12:43:17 PM
help is always appreciated in this project - why ask? ;)

well I have some ideas to raw_rec, but now I have to do some other part of ML, I will message you when I will be ready

AnotherDave

I just think it is important, when experimenting, to keep the end user in mind.

I say something as simple as "Rec 709" to a producer with 20 year of experience... and they look at me like I have 2 heads.

Not everyone is as technologically inclined as the folks on this board, and I want to be able to use these amazing images to improve my work.  The harder it gets on the backend... the less likely it is that will happen.

Dave

Stedda

Quote from: AnotherDave on August 01, 2013, 03:49:20 PM
I just think it is important, when experimenting, to keep the end user in mind.

They are nice enough to share their work in progress with us and clearly mark it as so.

Keep hounding them for features they said they'd work on for the future or for your own personal use and ruin it for the rest of us.

For something that is free and they work hard on it's never enough for some of you....
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

AnotherDave

Quote from: Stedda on August 01, 2013, 04:30:55 PM
They are nice enough to share their work in progress with us and clearly mark it as so.

Keep hounding them for features they said they'd work on for the future or for your own personal use and ruin it for the rest of us.

For something that is free and they work hard on it's never enough for some of you....

I'm not hounding, I'm trying to help.  I'm a professional, working filmmaker who loves the quality of the image I am getting through my "hacked" 5D3 with their awesome code.  I'm only asking to have changes implemented that will help with the perception that his custom firmware isn't a "hack" or something unprofessional, and can be used artistically in a production.

If they're not interested in my input, ask me to stop offering my experience and I'll just as easily stop posting.

Thanks.

Stedda

Look at g3gg0 quote from you, seems at least one main DEV has the same impression.

Don't ruin it for us patient ones that are happy to get whatever they can come up with because it's 1000 times better than what we had.

5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

AnotherDave

Quote from: Stedda on August 01, 2013, 05:20:06 PM
Look at g3gg0 quote from you, seems at least one main DEV has the same impression.

Don't ruin it for us patient ones that are happy to get whatever they can come up with because it's 1000 times better than what we had.

I'm not a developer.  I'm a DP, Director, Editor... when I say "I was hoping" what I should say is "a professional would expect" but I'm also not a jerk who thinks his ideas are the most important.

Bottom line is... this produces an image that I think is better than a Red.  It allows me to work on lower budget projects that can't afford a Red, but still get amazing images. 

It has its flaws though.  I know it is experimental.  But since we can already, rather comfortably, shoot 1080p on the 5D3 with no real issues.  *I say this based on the fact that I used it for a short where we filmed 1,400 clips and 2.68TB of data with it and had no problems.

As a professional, it seems more important to me to focus on what it is missing.  IE: WB information, external TC sync, Picture Profile information, easy backend processing...

If you'all don't think I should be using this thing and asking for improvement cause I am "annoying" them... fine.  The BMPC 4K is only $600 more than a non-hacked 5D3... so instead of getting what professionals want to use, we can just bow to marketing.

I thought the idea was to make it professional.  Was I wrong?