Author Topic: uncompressed 14-bit RAW video recording  (Read 747044 times)

Nspa32

  • New to the forum
  • *
  • Posts: 6
Re: uncompressed 14-bit RAW video recording
« Reply #900 on: May 16, 2013, 12:58:53 AM »
It's been said it probably wont work. Read a few pages back.

I don't understand why. If it's ported to the 600D and maybe the 7D, what makes it not usable on the T2i? I mean, doesn't the 600D have almost the exact same internals as the T2i? I don't get it. Someone please chime in here before I decide if I need to buy a new camera.

Also, good job to the people that made this happen. Amazing work.

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1857
  • Have you tried turning it off and on again?
Re: uncompressed 14-bit RAW video recording
« Reply #901 on: May 16, 2013, 01:14:53 AM »
@A1ex - Just trying out your new raw_rec.mo (from the 5d3). It's helping a lot on the 600d. Longer record times with hardly any corrupt frames and I'm still using a poorly performing card (10MB/s max).
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

mattmvpmedia

  • New to the forum
  • *
  • Posts: 24
Re: uncompressed 14-bit RAW video recording
« Reply #902 on: May 16, 2013, 01:17:58 AM »
Andy600 - forgive my newb question, but did you have to compile a build using the new raw_rec.mo for use with the 600D, or is it simple to just add it into the modules folder?

Rush

  • Member
  • ***
  • Posts: 159
  • 5D3, EOS M
Re: uncompressed 14-bit RAW video recording
« Reply #903 on: May 16, 2013, 01:20:42 AM »
I agree, last changes are very good! Thanks, a1ex!

Can I ask to add 434px height?
1280x434 will work for 10-20 sec max I think, but it will result in nice 16:9 1280x720 after restoring correct aspect ratio (when recording in 720p mode of 600D). I think that it will be my favorite resolution for 600D.
Greetings from Russia!

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1857
  • Have you tried turning it off and on again?
Re: uncompressed 14-bit RAW video recording
« Reply #904 on: May 16, 2013, 01:27:31 AM »
Andy600 - forgive my newb question, but did you have to compile a build using the new raw_rec.mo for use with the 600D, or is it simple to just add it into the modules folder?

http://www.magiclantern.fm/forum/index.php?topic=5413.25 go to reply #30 I assumed the raw_rec.mo was a unified module so I tried it. Works. Works well :) Just replace the one you are using with the one from the .zip

I forgot to thank the devs for it in my previous post so thanks guys!!  8)
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

mattmvpmedia

  • New to the forum
  • *
  • Posts: 24
Re: uncompressed 14-bit RAW video recording
« Reply #905 on: May 16, 2013, 01:29:06 AM »
Thanks Andy600!

squig

  • Hero Member
  • *****
  • Posts: 515
  • Crash test dummy MK3
Re: uncompressed 14-bit RAW video recording
« Reply #906 on: May 16, 2013, 01:34:18 AM »
There's no color profile assigned because its raw, color primaries and therefore gamut are not even defined, when raw is 'developed' in whatever app used you define the color space, dcraw for example lets you develop as XYZ, Prophoto, Adobe RGB or sRGB, lets you choose any gamma to be applied, so use a color managed app, define your working color space, choose different working spaces, import dng's, compare histograms and channel clipping, go from there.

I'm working on a Canon standard replicant sLUT for adobe camera raw.

squig

  • Hero Member
  • *****
  • Posts: 515
  • Crash test dummy MK3
Re: uncompressed 14-bit RAW video recording
« Reply #907 on: May 16, 2013, 01:36:55 AM »
Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...

Could be a load of bullshit marketing hype.... but I bought it. Should be here in a week or so.

squig

  • Hero Member
  • *****
  • Posts: 515
  • Crash test dummy MK3
Re: uncompressed 14-bit RAW video recording
« Reply #908 on: May 16, 2013, 01:43:23 AM »
Does anyone know why I continue getting this error when I boot the camera(5D3)
camera was not shut down cleanly. skipping module loading

I get that message after I boot the camera. but it only happens after I've recorded a raw video, then shut the camera down.
Only way I know to get rid of the message is to format the card and reinstall ML

Yeah same here, started happening on day 2 with the May 13 build.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: uncompressed 14-bit RAW video recording
« Reply #909 on: May 16, 2013, 02:00:58 AM »
Looks to be fixed from latest commit. I have to try it.. its just a lock file in the dir.. no need to format anything.

tonybeccar

  • Senior
  • ****
  • Posts: 319
  • 7D
Re: uncompressed 14-bit RAW video recording
« Reply #910 on: May 16, 2013, 02:11:30 AM »
Could be a load of bullshit marketing hype.... but I bought it. Should be here in a week or so.

Great!! Let us know what happens! :)

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: uncompressed 14-bit RAW video recording
« Reply #911 on: May 16, 2013, 02:18:08 AM »
The new module is the shit.

Beep should be on/off for now I guess turn off beeps.

My write appears to be lower, dunno if its calculated better but despite the numbers it appears way more stable.

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1857
  • Have you tried turning it off and on again?
Re: uncompressed 14-bit RAW video recording
« Reply #912 on: May 16, 2013, 02:20:01 AM »
Just did a quick and dirty 10 second test (on a slow SD card) to test ISO and scaling. It's not very pretty but...

http://www55.zippyshare.com/v/57521426/file.html (40mb file)

600d using the new raw_rec module - 960x360 scaled to 1080p / 2.35:1 (almost) shot at ISO2500 (which I would never use). I just made a few tweaks to levels and color in AE. I'm gonna try some proper shots tomorrow and maybe a green screen test.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

squig

  • Hero Member
  • *****
  • Posts: 515
  • Crash test dummy MK3
Re: uncompressed 14-bit RAW video recording
« Reply #913 on: May 16, 2013, 02:20:22 AM »
Great!! Let us know what happens! :)

Costs about the same as the Lexar so once I post the benchmarks we'll know which is the one to get.

mattmvpmedia

  • New to the forum
  • *
  • Posts: 24
New raw_rec module does help the 600D
« Reply #914 on: May 16, 2013, 02:35:37 AM »
Fo shizzle - the new raw_rec module helps a bunch. I'm still dropping a few frames at 960x540, still getting a little bit of tearing and magenta blocks, but much improved. Thanks!

hirethestache

  • Freshman
  • **
  • Posts: 78
Re: uncompressed 14-bit RAW video recording
« Reply #915 on: May 16, 2013, 02:37:34 AM »


I was about to ask these exact same questions...beat me to it!


Since the last I inquired about recording to the HDMI-out, the developers implied that you did not even thing continuous recording would be possible...but your magical ingenuity has broken through! Is the idea of running through the HDMI's [through an external field monitor such as the Atomos Ninja2] limit of 10gpbs (realistically 3.2) even a feasible approach, or is it completely unrealistic? This would reduce the strain on the camera, and heat would be much less of an issue...

Im going to catch flack for asking this again, but I did not see a response in the pages since posting.
@HireTheStache
www.HireTheStache.com
C100, 5D3, 5D2, 6D

ease

  • New to the forum
  • *
  • Posts: 7
Re: uncompressed 14-bit RAW video recording
« Reply #916 on: May 16, 2013, 02:39:30 AM »
i now just want to say thanks a lot for your amazing work here.



Kabuto1138

  • Freshman
  • **
  • Posts: 87
Re: uncompressed 14-bit RAW video recording
« Reply #917 on: May 16, 2013, 02:42:18 AM »
The new module is the shit.

Beep should be on/off for now I guess turn off beeps.

My write appears to be lower, dunno if its calculated better but despite the numbers it appears way more stable.

Hey guys, where is the new module?

thanks!

mattmvpmedia

  • New to the forum
  • *
  • Posts: 24
reply #904 has info on new raw_rec module
« Reply #918 on: May 16, 2013, 02:48:04 AM »
Go to reply #904 from Andy600 for details on where to get the new raw_rec module

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: uncompressed 14-bit RAW video recording
« Reply #919 on: May 16, 2013, 03:00:05 AM »
New stuff should be up for 6D/600D. New module and all.. .can hit rec to start stop movies.

It beeps twice because there is an unpress/press event and something is up with how its defined on 600D.

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1857
  • Have you tried turning it off and on again?
Re: uncompressed 14-bit RAW video recording
« Reply #920 on: May 16, 2013, 03:05:35 AM »
@1% - thanks man. I was just reading the new commits. Looks interesting :)

Just a thought. If performance takes a hit from Global Draw being on could the raw_rec module switch off things before recording is started and back on when recording is stopped?  :)
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

mattmvpmedia

  • New to the forum
  • *
  • Posts: 24
Updated build for 6d 600D
« Reply #921 on: May 16, 2013, 03:12:52 AM »
Thanks 1%!

FYI, if you are new like me you can find the new [600D] Beeps Twice.zip file here:
https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads

jasonp

  • New to the forum
  • *
  • Posts: 11
Re: uncompressed 14-bit RAW video recording
« Reply #922 on: May 16, 2013, 03:54:33 AM »
Komputer Bay 1000x 64Gb
The latest raw module posted previously by Andy in this thread works perfectly in 1920x1080 with a 5D3, drops 1 frame in x1152 and several in x1180. Let me know if you need more tests with different resolutions.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: uncompressed 14-bit RAW video recording
« Reply #923 on: May 16, 2013, 04:02:20 AM »
Global draw can be on for indicators. Mainly no peaking/zebras.

KMA_WWC

  • Freshman
  • **
  • Posts: 70
Re: uncompressed 14-bit RAW video recording
« Reply #924 on: May 16, 2013, 04:20:23 AM »

I just ordered and received the same CF card as you... the KomputerBay 128gb 1000x CF card and have been running tests with each of the various Raw builds over the last 3 days.  The latest build for 5D3 someone just posted from today seems to be the best in terms of speed... see http://www.magiclantern.fm/forum/index.php?topic=5413.25 (Reply #30).  This includes some small memory tweaks that definitely helped me get a little bit longer runtimes as well as moved me up to the next resolution without dropped frames.

Here has been my experience with the KomputerBay 128 GB 1000x CF card...

For my tests, I tried Global draw turned ON and OFF and there was no significant difference except maybe a second or two extra of recording time but the extra time was not consistent.  So I decided to leave Global Draw = ON (with just peaking and crop marks enabled).


My tests using the ML benchmark utility shows read speeds that range anywhere from 52MB/s to 72MB/s.  Using CrystalDiskMark and ATTO Disk Benchmark, I get similar write speeds so my particular card seems to have a write speed that varies a bit.  I would say the average write speed that I get most of the time is about 65MB/s which is completely in line with the ML benchmarks and the listed speed needed by each resolution listed under the RAW video section of ML.  So I would trust the ML benchmarks and recommended write speeds for each resolution.  I have also confirmed that at least one other person is getting similar write speed using this card.

That said, the buffer/memory tweak Alex made today did have a significant impact when using my card. 

Here are my tests Before today's build...
1920 x 720 or less resolution (no skipped frames for complete 4gb file)
1920 x 840 (no skipped frames for complete 4 gb file)... before today's build, this was the best I could do.
1920 x 900 (no skipped frames for complete 4 gb file)... tweaks added to today's build allowed me to use resolution for first time
1920 x 960 (maybe 10 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (maybe 1-2 seconds before frame skipping begins... unusabe beyond that)

After using today's build...
1920 x 900 (no skipped frames for complete 4 gb file) before today's build, I could not use this resolution
1920 x 960 (went from 10 seconds to 20 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (went from 1-2 seconds to 5 seconds before frame skipping begins... unusabe beyond that)

So the tweaks made by Alex today have definitely helped so I encourage further memory/buffer optimizations... they are making a difference.

I don't know how EOSHD, Nuemann Films, and Cinema 5D are getting 1080P and greater without dropped frames. They say they are using the same card.  Maybe some of these cards are getting faster write times.  Or maybe they are just getting that resolution for a limited time but a bit longer than me before frame skipping appears.  If any of them can chime in here to clarify, it would helpful for those considering the Komputerbay cards.  This card is by far the most realistic in terms of pricing for the vast majority of people who want to take advantage of this RAW update and plan on using it beyond test videos and personal work.  None of the other cards comes close in terms of price/gb.  The other solutions by Lexar and Toshiba (soon) are more like $650 for 128gb vs $178 or KomputerBay. I'm sure this will eventually change, but that could take a year or longer before cheaper alternatives are available.  But there is no getting around the fact that the less than ideal write speed of these cards may prevent continuous 1080p for some (depending on their particular card?).  But continued memory optimization and other techniques by the ML team may be able to extend 1080p recording from 5 seconds to 30 seconds or possibly more when using this card.  If that can be attained, the vast majority of folks will be able use this resolution in many shooting scenarios, short of documentary or event work.  I do realize that smaller resolutions can be scaled up in post nicely, but I tend to avoid that as do many others I'm sure.

Lastly, on the topic of workflow mentioned above... it's completely worth the time as far as I'm concerned.  The difference is night and day.  There are a lot of test videos coming out now, some of which don't fully capture the quality improvement gained here.  EOSHD, Nuemann Films, and Cinema 5D did great job of showing what's possible.  Just wait until some shorts and features by other skillful DOPs start to appear online.  While I don't love the added work required in post, I find it impossible to go back to H.264 after getting used to the look of my footage now...it's improved that much.

Thank you Alex and team for this... also to g3gg0 who apparently had a huge role in making this particular breakthrough happen.

I feel like you were siting next to me and seeing what I saw :) I have/had same issues like you did. I got my Komputerbay card and the speed vary between 60-72mb/s. I haven't tried the new build yet, but that's what I'm going to do. Hopefully its as good as other people say or at least better.


My personal observation about this whole thing:

I think right now not just everyday, but almost every hour or so there are new improvements in the firmware. I like the fact that most of the bugs can be fixed or worked around. I personally think installation and whole post workflow isn't as hard as some people describe it. It only took me 10min to downgrade from 1.2.1 to 1.1.3 and then install everything needed. Yes, post workflow is time consuming, but overall i think within a certain time that would be worked out too. I really like where everything goes and hope to contribute and help.

Also, I just tested the SanDisk extreme pro 32gb CF card and I was actually able to shoot at 1080p (1920-1080 exact) with that card until it reached 4gb. I had only 9 skipped frames (to be honest not 100% if it skipped any frames at all). Test shows the write speed around 74-88mb/s. Its much faster than my brand new komputerbay 128gb 1000x even though on paper komputerbay supposed to be better. All those test were done with an old build and I'm pretty sure after I install new build it would be much better.

QUESTION: Where do I send/attach the list of bugs I encountered also log infos?
Canon 5D Mark III