Messages

@chris_overseas
the key (if its static for all bodies, and it should) then will be important for encoding created files. nonetheless, i'm watching now your decoded files.

edit: do i see that right? the encoding key is 256kb long. (for pf3) holy sh*t.

@chris_overseas !big! kudos!
Do you have a encoding key extracted? Pieces? maybe its worth looking inside a canon-firmware for this key. a binary search. the longer the better.

jepp. the technicolor.pf2 is only about 5kb. if we can decode pf2, it would be a big milestone afor understanding the plain format - after that pf3 should be not that difficult.

@agentirons
a "nearly empty file" has no information for us to get the knowledge, we have decoded accordingly. i assume its still binary not readable (like xml).

• the copyright/title fields can be max 32/20 chars long. have to check if it s maybe as long as the encoding key. (dont believe it )
  
• if the known fields are long enough and xor-decoded, the key should start repeating. win.win.win.
  
• furthermore, if this decoded binary file is comparable to tiff, there should be unique tag-numbers (fixed length) and maybe a hex00 for data-end.
  
• maybe there are no tagnumbers but only hex00 (or other static bytevalues) to describe beginning and end of datafields.
  
• in terms of statistics this separation-value should be the val with the most occurence and its only one or two times successive.
  
  maybe [tag][datavalue][separator][tag][datavalue][separator][tag][datavalue][separator][tag][datavalue][separator]..
  or
  maybe [datavalue][separator][datavalue][separator][datavalue][separator]..
  or
  maybe [separator][datavalue][separator][separator][datavalue][separator][separator][datavalue][separator][separator][datavalue][separator]...
  
  regexlike: /[separator]{1:2}[^separator]+/
  
  ah, and datetime could be a value we can work with.
  
  regards chmee

@snep
i dont see a problem. sorry. i coded a simple previewer. demosaicing and color rendering is not that easy with raw-files. so finally my output does its job not that bad

uuh. i was (theoretically) sure, the styles are used after demosaicing and before/while rescaling to 8bit/jpg.

here two screenshots about the plaintext-riddle i assume, if we take this hurdle, we can read the whole file.


pf3_plaintext_decode_01

the difference is hex20 as in ASCII, but sometimes -20, sometimes +20


pf3_plaintext_decode_03

the same characters are encoded differently. maybe some of you see a pattern But: Do not forget, after decoding we ve got still a binary file, we have to find offsets for the entries.

point taken, @Danne you re right. But, if so.. we could try to extract the cinestyle-Preset from a cr2 and then compare with the pf3.

iccXML is able to convert the icc files to XML-human readable data. just for analyzing purposes. As i did that, i ve seen, icc files are not simple luts as i thought, its not a converting list 0->0, 1->1.2 and so on. its much more complex. (maybe there are 32bit-floatinpoints saved instead of 16bit-integer?)

later i will post some more findings on decoding pf-files. it seems something like a simple bitshift over the whole file or a list of bitshifts/adds/subs. its kind of a puzzle we can solve together

the main icc's are not in the cr2-files - theres only a integer/string to name the needed picture-style. Thats the reason, Adobe isnt able to use the real canon-styles - only dpp. for us, its only important for mp4/mov-recordings.

regards chmee

I did my thoughts on canon-profiles (aka styles) as well. i couldnt believe, these files arent hacked yet. there naturally is more than just contrast, saturation and so on. my thoughts by now ( i started yesterday a little bit analyzing files)

* a "simple and fast processing" inside the body is needed, therefore it must be a simple encoding.
* the pf3-files are ~double the size of the icc files found in the preset-style icc-path

example neutral with no adjustments
* icc file size - NS.ICC - 212kb
* pf3 file size - 425kb

* maybe for safety purposes the lut is written two times.
* maybe there s a inverted lut inside as well

* i saved a pf3-file with no changes three times with ~5secs difference. it seems, theres a timestamp in it, but the timestamp does not affect the encoding. nearly the end of the file a byte changes as well. a checksum? integrity check?
* the pf3-file must be rockstable to avoid crashing or deadlock the processing inside the body.
* the title and copyright arent saved plain, this leads me to the idea of a simple encoding. rle? difference? xor? not?

example links:
Canon Neutral SRGB 6091 ICC as XML (parts of it)
pf3 neutral saved three times

Because i dont know much about ICCs, i have to read more about ICC-LUTS to understand and recognize lut-data inside the hexview maybe i find a byte/word (simply the clue) to decode the pf3-file. after this it should be much more simple to create own luts and pf3-files. Working only with the luma-curve is unsatisfactory.

by the way. did you tried combining your lumacurve with low-contrast-setting? is this affecting the max-luma from 235 to maybe 255?

simple toDos:
* decode title and copyright (we know the plaintext)
* decode the simple things (contrast, saturation hue aso)
* decode lumacurve
* decode 6axes-data
- i assume there are all in the beginning, then the lut comes.

regards chmee

@zerocool22
lately.. generally it should export into seperate folders. did you changed the folder/filenaming-pattern?

@GutterPump
i'm very sorry. i hadnt had the time since half a year to patch the code - and i believe, i wont anymore. sorry sorry.

regards chmee

thx @reddeercity.

easy

So, lets put some small footage into the samples thread.

http://www.magiclantern.fm/forum/index.php?topic=11899.0

@Kharak
Please give me some more footage. Generally its the Fix coded by a1ex, but sometimes my code inverts the analyse-decision.

@Ake A.
Please use MLV or MLVLite. i know, raw is not working well in this version. 

look on start if theres a new version and ask. automatic updates are contraproductive. and finally you re not able to revert, because the older version will update itself.. loop of death.

(is there a coder out there, who guarantees, new version will have less bugs? nope?
&
today on twitter: automatic updates are the golden keys for backdoors. just pack your virus/malware into a "new version" (or infect the new version with a virus) and let spread it by the "automatic update"

So - NOPE, sounds not like a good idea.

@ibrahim
rename the last one to M00.. because all VIDF-Chunks (aka frames) have timestamps, its not necessary to hold the files (and frames) in the right order. but no one can recover files, you dont know, if they existed

@reddeercity
will check that when i'm back home. net is awfully slow here in geneva.

@ottoga
thats good news.

@Kharak
will look into it. (because i revert after converting to "convert" but it doesnt..)

thanks for testing. regards chmee

v1.7.5 is out
* please! treat it as a beta! i have no splitted files here (on my laptop).
* main patch - framebug solved (i think)
* toDo: Audiofix for Resolve (at home, then, in ~10days) | MLVlite-Support(need Samples) | tolerance on missing Files (see above M00-Example)

http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

regards chmee

1. what are the last lines inside the debug-file?
2. starting as administrator?

The tool will get an update next days. until now 1.6.5 is recommended.

@walterschulz
a pitty thats not fair kind of "dontlikethat" /thx

Audio Glitch (Resolve Problem) - could someone tell me if its working with raw2cdng 1.6.3?

1.6.1 - working audio under resolve
1.6.3 - audiolength fixed
1.6.5 - adjusted timeRef

(i dont know what to revert, audiolength or timeRef)

regards chmee

Hi there

@all
good news, missing-frame bug resolved, needs to be tested. but before yelling "new version out" i need to optimize/fix some other things.

(a) audio-glitch - its said, 1.6.1 did good audio-files for resolve, newer ones not, right?
(b) vertical banding. sometimes it was ok sometimes not. give me some more short video-examples to fix it.

@bobberesh
maybe a problem with folder names? try to dragdrop from other place..

regards chmee

@mothaibaphoto
thanks. just downloaded. will test now.

@ibrahim
this bug shouldnt occur with 1.6.1 or 1.6.5.

Code Select Expand

(a) [F]_C0000
will create very long digits as framenumbers, and fi premiere wont work with them.

Code Select Expand

(b) [F]_C0000(_)
underline on the end in parentheses will do a much more pleasing result..

Code Select Expand

Example:
(a) M29-1212_C0000000123.dng
(b) M29-1212_C0000_000123.dng

regards chmee

thanks for clarification. in this case i think i just left the last frame (generally) and its not under this "lostframe"-symptom. If you say, this bigger MLV has problems in my converter - could you leave me a bigger part, that loses frames somewhere in the middle? maybe about 250 frames? zip'ped?