Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - maxotics

#1
I try to explain the Magic Lantern RAW video look that can't be replicated with in-camera H.264.  Recently, I've been playing with the SL1. Amazing!  The ML devs got it to do something I thought impossible.  I bought a BMPCC4K (and will have to upgrade my PC to deal with it!).  Did you know you can't write CinemaDNG?   I also don't believe any of the current HDMI to RAW is real raw down to the pixel. For certain types of grainy looks, ML is becoming the only way to achieve it.

#2
Quote from: togg on January 11, 2018, 02:42:17 AM
very nice videos! I've listen to them while cooking, thanks.
I'm sending this to my sister.  She will crack the heck up :)
#3
I feel there's a tremendous amount of misinformation about LOG gammas, how they supposedly give RAW like dynamic range.  So I've been working on stuff to clarify it.  Anyway, this is another proof of just how great ML!  How else could I create RAW DNGs at 640x360 resolution!  Like all my videos, I wish it was better, that I had more time, but I hope it will help someone out there get beyond the hype of S-LOG, V-LOG, etc.

BTW, took a long time to figure out how to pull out integer values from a DNG file, so, in case there are other slow people here like me:

import rawpy
import numpy

raw = rawpy.imread('D:\\M08-2120_000520.DNG')
rawdata = raw.raw_image
numpy.savetxt("D:\\dng520.csv", rawdata, delimiter=",")


VIDEO:
https://youtu.be/ONfdarINvmw
#4
Raw Video / Magic Lantern RAW on 7D vlogging
October 23, 2017, 01:36:01 PM
Was curious how easy it would be to "vlog" on a 7D running raw.  My findings
1. Reliable, never had a corrupt file, running at 1728x972 24fps
2. With an external powered mic and camera set to +32db, and some audio post, sound is reasonable
3. Half-press shutter button before recording would get a good focus
4. Using a tape measure to set manual focus if far away worked fine
5. Shooting a high f-stop at 3200 expected ISO gave a much better image (noise organic) than 8bit in my experience.
6. The number one benefit compared to A6300 4K was that the 4K still blew out highlights whereas the RAW always had nice dynamic range roll-off. 
7. MLVProducer for the PC makes working with MLV files a pure joy (and I don't say that lightly about software).
8. Premiere's new audio presets make it much easier to improve any audio

I built a gadget that allows one to see the back screen, but I now see it's overly complicated.  Will create a newer/cheaper one soon.  You can see my gadgets here: http://maxotics.com/

And the video I shot is here

https://youtu.be/28gq9zVgSb4
#5
Share Your Videos / ML RAW 7D with Sigma 17-50/2.8
March 04, 2017, 11:24:05 PM
Just something I did quickly at a shoot, with Magic Lantern and my 7D.  Processed in MLVProducer. Formatted for Instagram.

https://www.youtube.com/watch?v=r7pY4BlCXzk

At article about the super high resolution photos I did at this shoot, more here at http://maxotics.com/2017/03/04/fiona-in-a-half-gig/.
#6
Camera-specific Development / Re: Canon 7D Mark I
January 30, 2017, 01:36:35 PM
I don't know if tearing is fixed.  When I use an external monitor and turn off focus peaking and such in ML, letting the monitor do it, it records fine.
#7
Hi Walter, yes, of course you're right.  There are  many errors in what I wrote (but my aim was to explain away some misconceptions greater than the difference between color spaces and bit depth), feel free to explain it better! I found that the more precise I tried to become, the more difficult to get my point across. I'd love to hear a better explanation of how RAW differs from whatever you want to call it ;)

#8
I haven't read this thread very closely, so apologies if I miss the point.

1. sRGB generally uses (edit thanks Walter) a 24-bit color space that represents around 16 millions colors, which is the MAXIMUM of what we can see and which our equipment can display.  EVERYTHING ends up in a similar bit-depth, adobe RGB, rec709 etc.  All viewable/practical dynamic range is represented by 8-bit channel/24-bit full color data range.     

2. RAW is a NON-VISUAL per pixel measurement of light intensity, usually in 14-bits, or 16,343 values that falls on a camera sensor.  Each pixel has a color filter placed above it that is used to measure color.  Each pixel can only see one color.  Neighboring pixel color values are necessary to create a full-color value.  This creates a huge barrier to understanding.  (8-bit color in sRGB video is a misnomer because it really means 8 bits per channel, when it is really a 24-bit full color value). 

When you see the washed-out footage of RAW video you are seeing A LIE

RAW data can create 4 trillion colors, or dynamic range BEYOND which we can see on our displays, or with our pupils fixed at one size.  Therefore, RAW DATA CANNOT BE SEEN.  What you see is a visual proxy.  RAW data could be shown as hyper saturated and then all grading would show going from hyper-saturated to normal saturation.  The way RAW video is displayed is completely arbitrary.  The videos one sees where the person shows the washed-out RAW footage and their grade is amateur hour (sorry, I know it's fun).  A "flat-profile" in 8bit video removes saturation (color) for more brightness detail.  You can add color back in, but not as well as if you had it in the first place.  "Grading" RAW, again, RAW doesn't have any look, gray, washed out, or otherwise.

RAW data must be understood scientifically as the readings of silicon wafers which we must put in visual space, just like x-rays, ultra-violet and other near infrared frequencies.  Yes, you can no more "see" RAW video than you can see x-rays.

I wish I had the time to write a better post here. 

Anyway, exporting DNGs into Premiere doesn't make sense to me because MLVProducer will create a nice sRGB data set so you don't need to process the RAW files in Premiere. If you are going to use DNGs, Photoshop has better tools for RAW to RGB.  Keep in mind a benefit of processing RAW on the computer, vs camera, is you have access to more powerful de-bayering algorithms (the programs that borrow color values from neighboring pixels).  The camera might use bicubic for example to save power, where you can run aMaze on your PC, which does a better job.

As for noise at the floor.  You have to pick a range of data in RAW to create a real-life sRGB color space.  "Noise" is subjective.  Converting RAW to something usable is about subjectively representing 42-bits of data (which can't be fully seen) into 24 bits of visual data within our biological dynamic range. 

Another way to think of it is that in sRGB you could record someone talking in the park and they look great, but the sky is washed out, as it would be in real life (unless we looks up at the sky and our pupils contracted).  Later, you discover there were UFOs in the sky.  You can't recover them in the sRGB footage because the sky is washed out (again, just as it would be in the sky if you tried to look at them in real life and you had just gone to the eye doctor, who gave you something so your pupils couldn't contract).  If you had RAW data, you could re-translate the data, "cast" the sky data into an image where you could see the UFO.  Of course, the same algorithm would darken out your original subject.

As for s-logs, they are 8-bit KLUDGES that move the sensitivity of light around in a 24-bit space, AT THE EXPENSE OF COLOR in the place they borrowed data from.  Most of the stuff about s-logs and color profiles I see on the Internet is complete cow-dung when it comes to improving dynamic-range.  It looks like magic.  You try it yourself and you'll see it's just snake oil.  S-log has a place, but a VERY LIMITED one, like shooting in a bright sunny day.  Again, it does NOT increase absolute dynamic range. 

So to answer the question here.  If you shoot RAW, don't overthink it.  Use MLVProducer (or MLRawViewer, etc.) to get a good 24-bit translation and get on with your life.  If you want higher DR in your 8-bit camera s-log is only a LIMITED solution for LIMITED situations which you better understand.  If you've seen great 8-bit video from a Canon C100 or Sony FS5 say, then you'll just have to get one ;)  They do in-camera DR translation better than consumer cameras but it has more to do with having s-log.  That is, they don't need log profiles to give more natural looking video than consumer cameras.



#9
If anyone has any doubts about ML Raw as a professional solution then all they need to do is watch your behind the scenes productions.  It looks like you have a lot of professionals trusting their day to ML RAW.  Like you, once I saw what RAW could do I couldn't watch Panasonic/Sony video without feeling I was watching people as weirdly-colored plastic moving mannequins.

I'm posting here for you poor filmmakers out there!  Although the 5DIII is dropping in price, the 7D is falling faster.  I bought one for $400.  Although it doesn't shoot 1080p easily like the 5DIII, it's close enough and it seems rock solid.  In other words, if you like what Francis has done get a 7D and either use his post solution, or MLVProducer for PC, or MLRawViewer (cross platform).  Yes, you'll have to spend an extra few minutes with each clip to get them into an editable 8bit space, but even doing a half-ass grade will give you a natural look one simply cannot get any other way.



#10
Did you tap the screen with two fingers simultaneously so you see the ML splash screen?  Then after that, press the menu key?
#11
I've been looking for a variation for this answer for 5 years and still I remain baffled.  My 2-cents of where I'm at.

90% of the benefit of shooting RAW happens in the first transcoding.  That is, when I set my exposure, whites, blacks, contrast, basic color and export to H.264 4:2:2 say, I can see little different in ProRes444 if I exported to that later in image processing.  Full disclosure, I'm not big into color grading, and it shows  :'(  Therefore, my goal switches from image quality to NLE efficiency.  If I use lossless, the computer hardware seems to struggle, if I use highly compressed (like MTS), the software seems to struggle.  So I'm always looking for the best compromise between the two.

For now, it seems H.264 is a good compromise (seems all the manufacturers focus on it).  Though I'm going to do more with DNxHD next.

Again, my approach, don't worry about CODEC for image quality, just worry about getting the conversion as close to the look I want.  Do worry about the CODEC in how nicely it will play in the NLE.  ProRes 444 seems high-maintenance in that regard.
#12
the detail and DR of RAW is so intoxicating (to me) that I often want to protect the shadows and highlights, as you have done.  However, I think this almost impossible in day-light.  Unless you're going for a specific low-contrast look, best to let your highlights go and let the model's face go bright.  In other words, the video was a bit dark to me.  Another idea is you keep the image muted in the street scenes, but once she goes into the light, let the image explode in color and brightness.  Then end it there.  Why go back to the dark street?  That's my 2-cents.  VERY NICE STUFF!
#13
I tried using hot pixel fixes with dcraw and never got anywhere.  As dfort pointed out, the pixels aren't predictable (at least I couldn't) and the ones that can fire take up almost the entire frame. if I remember correctly.  For me, the problem is solved because our sensitivity (or lack of it) for color allows a significant amount of color-information removal to take place before we notice it--the whole idea behind 4:2:0 compression.  At first, that answer I'm giving you was not good enough for me.  I hated the idea of losing, or doubling up on data points like video compression.   So even when Alex or G3ggo came out with chroma smoothing I resisted it.

Now it doesn't bother because my primary love of RAW video is the fact that it allows me to set each pixels 8-bit value (even if smoothed) from 14-bit sources.  One of my biggest curiosities is why I can never get 8-bit video to have the same DR feel as RAW.  Even with the EOS-M at sub 720p, the RAW has a more nuanced looked than, say, 4K from a GH4 or A7 (I've tried just about every consumer camera you can think of).  How can that be?  The only cameras that seem to approach the RAW "nuance" starts with the Canon C100 or Sony FS5, both close to $5,000.  My guess, until someone with more knowledge can explain it to me,  is that in-camera exposure is always a tiny bit off and once it writes to an 8-bit (or 24bit full color) space, even if 4K, there is no way to re-adjust the exposure without seriously degrading the DR feel. 

If the above is right, everyone's effort in 10bit is game-changing to me.  Initial tests I've done, and looking at other videos' shows that even 10bit RAW is vastly superior to any in camera video out there, even 4K.  10bit not only allows higher resolutions, it can also reduce file size.  A focus pixel removed through crude chroma smoothing will probably not be visible to anyone who put a priority on DR.  Video shot in 10bit RAW, I wager, will provide a feel that even 8K will not provide.  Thoughts? 
#14
Thanks Danne. Yes, I knew that but you're SO RIGHT.  I need to rave about dfort too! :) DFORT, consider yourself an ML hero in my book up there with Alex and g3ggo!  (Whatever happened to 1%?).  Wait, getting off track.  I've been using and loving my EOS-M and I know I have DFORT to thank!
#15
ImageJ is pretty cool.  I've used it for photo stitching.  I'm curious why EOS-M Focus Pixels is still an issue when MLVProducer seems to deal with them very well, with additional filters to soften aliasing (moire).  It even does 10bit.  Is it because MLVProduder is PC, or is there something else I'm missing?  Also, MLRawViewer removes focus pixels through "stripe" removal.  Finally, I want to take very opportunity to rave about MLVProducer  :D
#16
Great stuff!  I love the natural "grain" of RAW which comes though nicely in your video. 
#17
I'm looking forward to the 7D.  FYI, it doesn't seem you can use normal MLV with the EOS-M 10/12-bit build.  It will say "thread not starting" or something like that.   So seems like that is a 10-bit build only.
#18
The question you're asking is based on a faulty premise, along the lines of, "all firefighters are men; all men are firefighters."  You can make MagicLantern RAW on the 5D3, say, look like h.264 from the same 5D3, but you CANNOT make h.264 from the 5D3 look like all "grades" of RAW from the 5D3. 

Certainly, all cameras have "looks" straight out of the camera that differ.  Canon is redish, Sony is yellowish, and I don't know what exact shade of green Panasonic is ;)  (only joking!).  Determining what kind of camera was used in such low-light, highly post-processing stylization, is difficult, maybe impossible.

Like everyone else on this forum, all you can do is experiment, look at other people's experiments, and figure out how to get to the look you want.  Sometimes you can use just about any camera, sometimes you need a specific camera, but even that's more a function of image output (RAW or H.264), (1080p or 4k) than it is anything else.

I didn't watch a lot of those videos, but my 2-cents is most common APS-C (because of background blur) cameras can get you that look.  But before I get flamed, yes a speedbooster on a MFT can do the same.
#19
Thanks dfort, I'll do that!  BTW, I don't even see ANY youtube videos on the site because Chrome doesn't like the "unsafe scripts".  I have to click on a warning icon in my address bar to see them.  So there's something funky with the forum which I don't feel like bothing g3gg0 about, since it doesn't seem anyone else is experiencing it?
#20
Raw Video Postprocessing / Re: MLVProducer
December 31, 2016, 11:12:52 PM
Is there a place for MLV Producer requests?  I'd like an option to keep the "settings" or "options" for one clip to carry through to the next.  That is, if I "grade" some footage to see how it looks I'd like those same setting when I open the next clip, which is usually shot under the same lighting conditions.  Right now, each time I open a clip all the settings go back to default/neutral.  THANKS!

Also, while I'm here.  What is the best format, in your opinion, to save the MLV file, in that it retains the most information for the compression in a way that I could delete the large MLV files?  I don't want to save DNGs, so I'm looking for an 80% solution.  THANKS!
#21
More footage taken with 7D shooting ML RAW at 1172x972 with Sigma 17-25/2.8, which is a really nice lens.  Internal audio.  I bought body used, and lens used (though new) for $700.  Without MLV Producer I wouldn't even bother.; that is, it makes shooting RAW video a livable post-processing chore ;)  It even displayed EOS-M 10bit clips the other night with focus pixels removed and moire reduced.  What an amazing Magic Lantern tool.  Great work Magic Lantern engineers!  Don't have enough money for a Canon C-100, or 5KmkIII?  Get a 7D!  It is super sweet, IMHO.  I haven't had one error yet.

AND...of course...YouTube doesn't do the original footage justice.  It is beautiful and completely film-like.

#22
I was able to do various  10-bit videos on the EOS-M.  1600x900 is amazing!  Files play fine in latest MLVProducer.   Amazing!  dfort you are DA MAN :)   I see builds for the 7D and it sounds like others have got it to work.  Should I stick with the EOS-M or should I, can I, try on 7D? 
#23
Thanks Teamsleepkid, was wondering about that.  Will try it on the EOSM.  It always warms my heart when someone says, "Hey Max, you turned me onto Crack, thanks!" :)  Is there an instruction page anywhere, hard for me to piece together from threads.
#24
Hi D_Odel.  I thought he only missed one captioning of footage.  The most obvious difference between H.264 and RAW is either in the dark or bright parts of the image.  H.264 usually "crushes" dark parts of the image to black, whereas you can see more detail in RAW.  Same for bright parts of the image.  The sky will usually band, or get blotchy in H.264 but more light grays and detail in RAW. 

I thought the test very conservative in that he could have went for a lower contrast image possible in RAW. 

Anyway, if you look at the video again, but then look at the dark and bright parts of the image between each clip you'll get that AH HA moment :)
#25
I'm happy to test the 12/10 bit on my 7D.  But would need instructions.  I can't make complete sense of it reading threads.  Again, only if you need someone to test.  I can wait ;)