12-bit (and 10-bit) RAW video development discussion

Started by d, May 22, 2013, 10:58:34 PM

Previous topic - Next topic

0 Members and 4 Guests are viewing this topic.

reddeercity

No joy with exiftool , tried with mlv_dump , results below 
Will there is problems , Can you upload the corrupted MLV file if not too big,
you skinny can it down with mlvdiag.1.0.1.zip if on pc
C:\New\test>mlv_dump -o updated.dng --black-fix=valve M03-1839_000008.dng
MLV Dumper v1.0
-----------------
Mode of operation:
   - Input MLV file: 'M03-1839_000008.dng'
   - Setting black level to 1
   - Rewrite MLV
   - Output into 'updated.dng'
File M03-1839_000008.dng opened
Processing...
[ERROR] Invalid block size at position 0x00000000
Processed 0 video frames
[ERROR] Failed to rewrite header in .MLV file
Done


You have some odd white & black levels
Unique Camera Model             : Canon EOS 5D Mark III
Black Level                     : 418
White Level                     : 1018

Light valve seems off too
Light Value                     : 3.1


MitchLally

It is 8.5GB file. I'm on Mac.

I found a similarly corrupted file that is much smaller.

https://www.dropbox.com/s/rmkyyg6q3nn47yj/M03-1356.MLV?dl=0

a1ex

Setting black level to 128 fixes it. There's a bug in MLVFS that doesn't let you change black level with exiftool, but didn't manage to look into it yet.

MitchLally

I used mlv_dump instead to convert to DNG then used exiftool black level to 2040 – seems to have fixed it.

Although now there is an issue with the top section of the frame. It looks smeared.

https://www.dropbox.com/s/ucathd53p1megkt/M03-1839_000001.jpg?dl=0

Danne

Just a note. mlv_dump supports specifying both black and white level manually.
Also had issues with mlvfs dng files and exiftool. DNG files turning black or not letting exiftool changing tags.

a1ex

Quote from: MitchLally on March 06, 2017, 09:17:48 AM
Although now there is an issue with the top section of the frame. It looks smeared.

With mlv_dump from http://builds.magiclantern.fm/utilities.html ?

If yes, try disabling cold pixel fix (--no-fixcp) and upload a small MLV so we can look into it (even cutting the first few MB from a file will do the trick). M03-1356.MLV appears to be OK.

reddeercity

Just a little update on the digic4 cams downloads


10-12bit downloads_DigicIV
by RedDeerCityTV, on Flickr

Seams 5D2 users are very hungry for Bit reduction  148 downloads and counting  8)
I don't think even 5d3 has that many bit reduction downloads
Long Live the mighty Digic IV Cams 7 years old+ and still in the game  :P

MitchLally

Quote from: a1ex on March 06, 2017, 01:08:18 PM
With mlv_dump from http://builds.magiclantern.fm/utilities.html ?

If yes, try disabling cold pixel fix (--no-fixcp) and upload a small MLV so we can look into it (even cutting the first few MB from a file will do the trick). M03-1356.MLV appears to be OK.

I'm on mac – is there an updated mlv_dump for mac?


Surin Dmitriy

Whats going on with in the raw_twk, mlv_play and 10 12 bit??? A lot of people confirm the issue whith playback - you can only review 2-3 clips before you get black screen?
5D3


Anstech

I'm a programmer in a very different field, and I'm just starting to try to get up to speed with understanding the ML / Digic context, so please excuse any naivety in this question:

One way that ArriRaw, amongst others, reduces its data requirements is to write the 16bit linear data from the sensor into an uncompressed raw 12bit log data stream, which - given the familiar inefficiency of linear encoding of light values - drastically reduces frame sizes.

The current 14-to- 12- and 10-bit strategy is an amazingly impressive piece of work, and I can see took years to achieve, and a lot of specialist knowledge. If I understand correctly, it's a bit-chop lossy reduction in representation precision. Obviously the raw-log-encoded method would be less of a quality drop from 14- to 10-bit, if it existed.

So before I 'go off on one' (as my old AI programming tutor used to put it when he saw me disappearing down a research rabbit hole / dead end again):

Is e.g. a 14-bit to 10-bit log-encoding of the raw uncompressed (linear) sensor data - rather than a lossy bit-chop - computationally plausible in this context? Or is it waaay too ambitious for the resources available on-camera during recording?

It's caught my imagination, but I realise it might be a silly idea if you know enough - so if that's the case, I'll be sensible and take on a far more manageable task from the 'to-do' list as a 'teach myself' project instead...

Thank you!

a1ex

On CPU, even a simple memcpy is too slow. So, the only remaining possibility is finding out how to configure Canon's image processor to do that.

Relevant links here, here, here and here.

TKez

Chiming in to add/confirm reports on 550D every 2nd frame is a copy of the last frame, except the top 3rd which looks like current frame, only shifted on X axis.

If I can contribute to testing in anyway, gimme a job! 550D is 'B' cam, but love to get more than a couple of seconds of Raw at full res.

ricbi1

Anstech... if you achieved that, it would be great! Just trying to encourage coz I can't do code for my life. Absolute ZERO. I tried over the years but no, can't do sir! Thanks everyone who is coding for sharing their efforts and developments!  :)

Eugenia

Hello, first post here. :)
I have two questions, that I couldn't find answers for:

1. What is the max res of the EOS M using 10 bit, no crop, at 24p continuous?

2. What is the max res of the 5D MkII using 10 bit, no crop, at 24p continuous? Why is there no 10-12bit ZIP file to download for the 5D-MkII? I couldn't find any on the exp. download page.

thx!

far.in.out

Hey guys. Sorry for asking such a question but I really have no time to read all the thread, been away for too long...
What is the status on 12/10 bit raw? Is it working? Are there any downsides compared to 14bit?
I'm mainly interested in 50D as it's the camera that I have.
If there's a compiled module for my cam please give me the link. Thanks.
EOS M (was 600D > 50D)

ilia3101

Red deer city has some builds uploaded a few pages back, including 50D, I know for a fact that the 50d can't film 10/12 bit without corruption in full sensor, but maybe it works well in crop mode like the 5D2, give it a try.
edit: it's on page 46

beauchampy

Just came back from Malta filming a client piece entirely using 12bit and 3x3 50p. Flying by the seat of my pants, but the footage is lovely and not a single corrupt frame (5d3). I'll post some results next week. As far as I am concerned, this is more than ready for commercial use IMO.

andy kh

atleast 4 footage gets corrupted out of 10 with wrong black level.  im using 70D
5D Mark III - 70D


beauchampy

Quote from: andy kh on March 27, 2017, 12:27:24 PM
atleast 4 footage gets corrupted out of 10 with wrong black level.  im using 70D

I'm using cr2hdr which enables you to specify a black level. Not sure what it is on the 70D but on my 5D3 I'm using 2048.

I do have the odd clip here and there with strange pink highlight clipping. I had also specified a white level of 1163.

Danne

Could you post an example MLV which shows the corruption? You could shorten it before uploading. Use mlv_dump and open up terminal and do following:
mlv_dump -f 5 -o Output.MLV Input.MLV

mlv_dump
http://www.magiclantern.fm/modules/modules/mlv_dump.zip/mlv_dump.zip

Input.MLV being the corrupted file and Output.MLV the name of the shortened file.
Beware to do things right since a wrong command could erase the file in question.

By the way. I think black and white levels should be around this for the 70D
White level 12277
Black level   2049

andy kh

@Danne  please check this out. i use mlvp and set the black level to 125 for all 10bit mlv raw.
https://expirebox.com/download/fcfa483143425529e11d9cd2f4c43776.html
5D Mark III - 70D

Danne

Yes, classic black level issue. From your file after extracting with mlv_dump
Black Level                     : 7312
White Level                     : 14992

When reprocessing and specifying numbers below picture is perfect.
Black Level                     : 2049
White Level                     : 12277

In mlvp you specify 10bit levels so they differ. Above numbers are for upressed to 14bit. Visually the picture should look the same for both 10bit and 14bit after fixing.