Messages

didn´t say it works with 12 bit.  premiere just can´t  import anything greater than 12 bit.
i couldn´t import 14 bit tiffs and i couldn´t import the 14 bit cdng files. neither wrapped nor straight.

premiere has 8, 16 and 32 bit effects.

and there´s no native cdng importer.

as far as i know share after effects and premiere the same media core and it is capable of 32 bit rendering.

tired from the long and unsatisfying render and data wars i faced the last days, i packed the cam on the shelf and went to my note board....

i hope, i didn´t miss anything in the forum and can finally contribute to the community:

first, it´s not free!!
and maybe some adobe people take this as a hint...
you should have some experience using premiere. and a little discipline according to file and folder organisation.
no renaming or dislocation during the process of any folder or file!
this is crucial!

you have to manually install software and copy plugins in specific folders.
there´s a premiere plug in (necessary!!)
and a after effects plug in (not necessary)

you´ll need adobe premiere, after effects and adobe camera raw.
plus the ginger hdr merger. you can get a free limited license, but the dng merger is freeware.

so, let´s begin:

1. grab your memory card, stick it in your reader and copy your raw container to a harddrive of your choice.

2. get raw2cdng. the newest version with batch support.

3. drag and drop your cr2 files to raw2cdng. set bit depth to linear 12 bit. (this is vital, because premiere can´t handle bit depths greater than 12 bit.... not true!! with the ginger plug in i was able to load 16bit cdngs)

4. hit the green convert button. raw2dng now extracts the dng files from the container and writes them into a newly generated folder named after the .cr2 file  ( use the prefix CNDG to avoid confusion and helps to keep your file structure clean. verry important! )

5. now comes the funny part. grab your first cdng (your dngs are now officially cinema conform....  ) from your first folder.
open it in adobe camera raw and set the white balance right. no need to do more. just to avoid pink surprises.
save the dng file. meaning, just click "DONE".

6. now comes the fascinating part. open the merger from your fresh installed ginger HDR software. select wrapper (batch)
a new window opens
from top to bottom:
- you set the in step 5 generated reference frame. the merger takes the image settings for all the wrapped cdngs.
- select the first folder. let it be FOLDER1
- select the last folder. lets say its FOLDER46
    the merger will process all inbetween folders. FOLDER1 to FOLDER46
- set your correct framerate
- leave everything else as it is and watch this video if there are any questions left http://vimeo.com/54754773
- generate the gnr files
- enjoy the incredible speed!!

6.5  you can leave this step. with the ginger plug in installed you can import the cdngs directly!

7. check if you have the gnr plug in in the correct premiere folder (there is a readme/install file!) if not, premiere won´t recognize the gnr  files.

8. import your new and small gnr files.
what´s so special about this files?
till now your camera raw files have only been slightly compressed by 2 bit. that´s just a very simple mathematical process. there´s no real loss in that. despite 12 bit is the highest real regular "cinema cameras" record.
the gnr file is just a pointer to the original cdng image sequences.  it tells the premiere plug in where to find the cdngs and how to treat them. in terms of framerate and basic quality settings.

the gnr plug in for premiere is a highly optimized and very fast small raw converter. it leaves a lot of information alone touching nearly nothing than really basic image developing parameters. no noise reduction, no sharpening, no CA-correction.... and so on.

thus it is very fast and not really ugly. and way enough for editing!

9. sync your sound. or take your favourite music and start editing.

10. you can even apply effects and i think transitions (not tested yet)

11. once finished with your editing duplicate your master sequence.

short explanation:
what we do here is a so called off-line or proxy cut. but our proxies are no separate files. they´re the original. saving two crucial things:
time and space, but not in an astronomical sense...... work/render-time and harddisk space.
just remember, we´re still working with the original cdng files!

12.
- rename the new sequence "Online Master" (or something else that sounds important and mark it red. always!)
- hit ctrl+a in the sequence. all the clips in the sequence should be selected.
- right click the selected clips
- select "replace with after effects composition"
- after effects will start automatically.
- there will be a new (master)composition representing the premiere timeline. and other compositions containing your clips/footage
- if you didn´t install the AE plug in all of your footage will be offline.
- that´s good news. it reassures you, you are on the right way

13. check your master composition settings (size, framerate)

14. right click your first offline footage. "replace" the reference gnr file by the original cdng sequence.

15. adobe camera raw opens during the import process. take it as it comes and tweak it later or correct it to a flat image. or crunch and compress right away....

16. replace every offline gnr file by the corresponding original.

17. set bit depth in AE to 16 bit.

17. do whatever you want to do in after effects.

18. render the final master in after effects.

19. ideally your sound is mixed and mastered and you join it with the image right in AE

20. you are done


noticed something?
the only stage in this workflow when real compression (not necessary, you can also output a uncompressed 12 bit image sequence, prores 444....) is in the final output of the finished piece.
through the whole process you are completely independent of color spaces and keep all options of delivering your material.

no need to render till the final step.

no need to find a special intermediate codec. why the hell adobe has no own lossless codec???

you could even switch back to premiere. via dynamic link it is suddenly possible to have cdngs in your timeline.
but i wouldn´t recommend rendering in premiere!

AE is much more stable and has a greater variety of supported formats.

i also would always suggest to render your final movie as a image sequence. 12 hours of rendering and one line in your .mov header ruins everything!

i tested this workflow for a short sequence. the wrapped cdngs perform pretty good with 1/2 resolution in premiere.
as said: just edit in premiere. do the other things in AE.

why not resolve you might ask?
i tried the cdngs in resolve. resolve is fast, using the gpu for all the raw things. but the images look ugly....maybe i played to less with it.
but finally, when it comes to deliver your material from resolve to your editing application one has to decide which way to go.
good for the prores users. but.... you double your storage amount.
avid dnxhd.... there´s an 444 option, but not in the free codecs (or am i wrong?) premiere couldn´t handle the dnxhd RGB files.
also here........the storage!

i hope i didn´t forget anything. if there´s a better solution. tell me!

of course you can use proxies for the editing. then generate a edl, import it into resolve replace the proxies and then grade......i know the usual workflow.
but i don´t see such a powerful tool in terms of dng development in resolve compared to acr or raw therapee (it has six pretty good different debayering methods....but crashes a lot and is very slow)

1920x872 scales pretty nice to 4k  (4096x1920)....... this is so awesome!! 

thank you a1ex for that wonderful work!

has anybody found a reason why there´s always that mysterious 2 times so called  warm up recordings?
it can´t really be a thermal warm up.... then we would be way out of regular technical specifications.

what´s your strategy in optimizing the (probably) card specific writing strategy?
firmware check on the cards (brand, size, manufacturing date)?

or a new benchmarking tool in ML to determine the best settings?
is there a way to format the card from inside ML with optimized cluster settings and/or aligning?

i think the problem here will be the cards and their constantly changing inner structure. let there be a new controler IN the card and the writing strategy might be a nightmare.... a komputerbay x1000 might work for one user and for the next it might be completely "useless"....because everybody is looking for the highest resolutions possible. sick pixel x pixel marketing....

or am i wrong and everything you do is card independent?

hey jagnje,

i recorded it in 50 fps. interpreted as 25 fps or slowed down to 50% gives you a flawless 1/2 speed real slow motion.
if you slow it down by another 50% using sth like twixtor or the foundrys kronos or the retime component of saphire plug ins or any other software using optical flow you can get a real good 1/4 speed interpolated slomo.

but it´s important to shoot with the highest shutter speed possible. the less motion blur you have in your images the better the motion flow based image interpolation works. you can even lower the software slow motion to 25% and still get medium to good results.
but the algorithms really start to struggle with those speeds and warping becomes more and more visible (see the close of the fountain).
it also depends on the image content.

it´s more something to play with than with real value

of course it´s guaranteed to work.

but the ideal solution would of course be a standard v-mount to  the canon specific voltage converter.
just search for it on ebay. india seems to be full of those things!   

i have here a bunch of NP1 batteries (anybody remembers them?) used a car 12 volt to usb 5 volt adapter to power a portable beamer.
worked fine.
maybe there´s a car to canon adapter. would be much cheaper and lighter. the inverters i know a mostly pretty heavy.

good equipment to verify the voltages and much more important the currents is imperative!
and of course you really should know what you are doing.



if you don´t know how to measure currents or voltages.... of course, always go the safe way and buy already tested and working configurations......i tell people that execute not really tested code on their pricey cams 

try to get a straight 12 V to 8.... (whatever voltage it needs) converter. your method will destroy a lot of electrical energy into direct heat...
you convert DC to AC and then back to DC. you can go DC to DC. much better!!

theoretically a resistor should work. but a stabilized power supply would be the best.

there´s another idea i got to overcome the lacking sound ability of the 50D:

is it possible, to fire the flash when raw recording starts?

one could use a regular video camera/dslr as a 2nd unit (e.g. in interviews) to record the sound and then sync both cams using the flash.

it would also be thinkable to record the flash sound via a cheap lavalier microphone and mixing it into the sound signal using a production mixer. this could save the 2nd camera, replacing it by a regular external sound recorder.
fine tuning should be possible with the ttl-settings....

could be some kind of a hardware hack.

now got the komputerbay 32 GB x1000

and i can verify all of your speeds and resolutions. even 2000x836 @23,9 was right from the start no problem. i really wonder, where this will end now...... great work!!!

hey guys!

good to hear from the writing strategy.
i thought about something like this by myself. but more in a way of interlacing the image and writing it in blocks to the card.
maybe storing it randomly on the card, so the card can be written on with a constant speed continuously?

i don´t know how the dng files are actually contained as one file. if they are in sequence or already randomly stored.
is it possible to generate a mft type list in that container to restore the correct sequence in the unpacking process on the computer?

or maybe a kind of pre record strategy. you could arm the raw recording process=>buffer gets filled and rewritten for an "optimal" amount and circles for 10 seconds. then kills itself to prevent heat and to safe battery.
then you hit the fire button and the card can take the optimal data stream.

what´s the h264 coder doing? in the 50D it has to be activated to get raw recording.

this is just theory. and of course i don´t know if anything of this is possible or even already implemented.

would love to help with coding, but my coding days started and stopped with the C64...



and i did some more practice tests. wondered, what the also increased motion resolution (compared to interframe based h264. means, h264 dumps not just color and detail information by macro blocking but also motion information which is stored in vectors)

so here you have it: horrible resolution and a nightmare of aliasing but a synthesized 200 fps camera (shot 50fps)

I'm not saying this is true, I really don't know, but couldn't one argue that the BMPC's 1080p ProRes is still better than 1536x864 RAW?

this is sth we will see, when the cam is finally out!
but despite picture quality:
this camera still offers the ability to take the movs straight from the card and start editing immediately. from a professional point of view this is a killer feature!

and 10 bit 422 pro res can provide really good resolution and enough color information to correct the usual accidents while shooting ( white balance, underexposure.... )

and you can be pretty shure  there´s less moire and aliasing straight out of cam.

the workflow is just faster and smoother.

there will be two different kinds of users:

- eos ml raw for the advanced amateur/semi-pro, experimentalists and video artists. just people that don´t have to meet dead lines.     maybe the 5DIII will realy find its market as a production raw shooter

- bmpcc   for industrial/image film shooters, maybe journalists who will like the sleek and slim design and format. the people that pre-ordered it.....   and of course as a landmark in affordable picture quality. just to set pressure on the market and make maybe 422 10 bit the new market standard, replacing 420/411 8 bit

just my thoughts!   

The biggest limitation on the 5DMKII and 50D is not file system related, but the memory bandwidth of the internal RAM. If you benchmark the write speed of the 50D without LiveView activated, it easily surpasses 90 MB/s on my 1000x KomputerBay 32GB Card. Once you activate LiveView, sensor data gets written into SDRAM, and the Compact Flash DMA has to share the memory bandwidth with the sensor image DMA stream, dropping to about 67 - 70MB/s.

The 5DMKIII has a revised DIGIC with faster SDRAM memory access, which is the reason why it can still write >90MB/s with Live View on.
[/quote]

okay, very good hint!
so, for me.....the writing speed does not alter in any way. neither live view enabled/disabled nor turning the screen completely off makes any difference.
writing speed does not exceed 40 MB/s. while reading is always about 100 MB/s.

think it is truly a limitation of the card.

tomorrow i should have the new komputerbay x1000 card. i just want around 50-60 MB/s.

but the latest build with the fast_raw recording module gives me 1536x656 @ 23.9 fps. global draw off.

and it seems that temperature really influences the recording performance. my cam gets really hot after a while. i don´t have the issues with the 550D. it showed rock solid performance even in bright sun light and 35 degrees centigrade. i would be worried taking pictures in those conditions with the 50D.

maybe it´s just the metal case of the 50D, conducting more heat to the outside??

Hey Crash-Film... Got a response to your card write-speed issue.

The math doesn't seem to add up when it comes time to how a card performs... Put a Komputerbay 64GB 1000x in a Canon 5DIII and experience 125MB/s write speeds. Place the same card in the 50D and experience approximately 70MB/s. The difference can be in the EXFAT format and the FAT32 format limitations. In addition to the card's read/write limitations, the camera also has it's read/write limitations. Thus, it's important to find the correct mate, if you will.

The reason why the math isn't really panning out is because the camera is writing data at the very edge of it's capabilities. And since it appears that your card is pushing the edge of it's capabilities, in addition to your camera, you may be dealing with what may be described as a double-negative recording process.

Try searching the ML forum and you might discover that the Lexar Professional 1000X and the Komputerbay 32GB 1000X, and the 64GB 1000X are the cards of choice. I can't write that for sure, of course. Myself, I've primarily been glued to the 50D Raw Recording forum and not so much the 5D II and 5D III. Anyway, the best value is probably 64GB 1000X from Komputerbay.... By the way Komputerbay has been quoted as stating, "We receive/sell/brand Lexar Professional's cards that didn't meet the highest, most stringent quality-control measures."

Aside from that bit of re-assurance in the quality of the Komputerbay, having originated from a high-end compact-flash card manufacturer, I guess we're discovering that the Lexar 800 compactflash card is more of an entry level UDMA 7 rather than a high-end UDMA 7.

You might consider turning off every feature not related to raw_record, and completely clear the screen of excess overlays, in addition, make sure you have all of your Canon menu settings for image & exposure set to disable. These are definite disrupters for maximizing raw_record.

Hang in there and keep up with the posts. See you around.

- Levi

thanks levi!

okay, i got 720p raw video.....with the x800 card its absolutely no problem.
that´s all i wanted since i could not get it with my 550D.
i can even get the 14.. x 5..  1:2.20 (it´s to hot, to grab the camera just for that...) resolution@23,9 fps and  the buffer is like <*.....> and idling.   just one step greater in vertical resolution and the buffer fills continuously up. so it´s the cards performance for shure.

i think card manufacturers gain their incredible x1000 speeds by creating a card intern raid. ( as in panasonics P2 cards are technically SD-Cards in a raid) also the used memory chips influence the memory bandwith with their inner structure.
so in a 32 GB card may be the doubled infrastructure of a 16 GB card. leading to a theoretically doubled performance.
and so on....
as far as i can remember CF cards where always the fastest cards because of their intern UDMA controller. maybe in some cards (komputerbay) there´s a controller with two "lanes" each with a memory interface that matches the 16GB modules. so in the 16 GB card there´s just one "lane" used. in the 32 GB card both of them are used. and in the 64 GB card a controler optimized for four 16 GB modules (or two 32 GB modules) is used. and this controler is also used in the 128 GB but organized in a way that´s good enough for marketing but technically not perfect. this explains the different performances of differently sized cards.

i know there´re no 16 GB modules and so on and memory is organized and transferred in bits.
but i hope i could explain my thoughts about this as plain as possible 

so as long as there´s no card manufacturer, that gives exakt specs of it´s cards there will be always some gambling.

you must always hope that e.g. komputerbay understands the needs of it´s customers and doesn´t just follow the law of economy and raises the prices and lowers the production costs.... by changing to a cheaper controller or memory chips.

maybe a magic lantern raw performance guarantee and a nice sticker on the box would help. of course making the card 10% more expensive than the standard cards but encouraging the manufacturer to hold up performance.

and there´s a lot of marketing in those memory cards.
sony itself offers a SxS to SD card adapter. no problem recording on a cheap 32 GB class 10 card in a EX1. or F3.... if you want to

not the fastest cards on the market will do the trick, but to find cards which match the memory interface ( or call it bandwidth limit) of the specific cam as close as possible should safe buffer and gain bandwith.

and i am pretty shure i am telling no news to the most of you...

i ordered a komputerbay 32 GB x1000. the lexar 16 GB goes back to the retailer. final step will be the lexar 32 GB x1000

would love to buy them all and cross check in my own camera. but, could get a little bit expensive.

we will see...

hi,

i got my 50D now and i am a little bit disappointed.
first of all: it´s dark outside, so there´s nothing to shoot.

and i got horrible writing speeds!!
i have a komputerbay 32 GB x600 card and can barely top the data rate, that´s possible in the 550d.

to check if there´s something wrong with the komputerbay card i got me a lexar 16 GB x800.... and now:
average write speed   41 MB/s
average read speed  100 MB/s

with komputerbay:
average write speed   26 MB/s
average read speed    63 MB/s

using the W/R benchmark

i use todays build. tryed different aligning settings. btw.    if you set the sector sizes the same as the allocation sizes the cam dies.
i had to remove the battery, start the cam with another NON ML  card, restart it with an ML card.... and it worked again.
used all the "tricks" ( raw off, noise reduction, global draw off/on...)

but back to my problem. i expected specially with the lexar card a data rate of at least 50-60 MB/s

or is this impossible with such a card?

or am i the problem?

to me it seems, as if write speed/read speed = 1/2
i expected a 2/3 ratio. at least from the x800 card.
i have just a usb 2 card reader


someone any suggestions?

thanks!!

That was absolutely stunning!!! It's nice to finally see something that isn't just cats, dogs, flowers, cars, or telephone poles! This really illustrates how much RAW can help. Care to share what settings you used on the cam?

thx!!!

the settings where:
tragic lantern raw module, mem hack on, 1152x432 @ 23,9 fps, transcend class 10 32 GB,  1920x1080 24 fps video mode, no 5x zoom (it always stops after 70 frames in zoom mode)
tested several cards. even a lexxar 400x can´t do more resolution....

and the best settings of all..... in a few days the nearly new 50d arrives. so i gave up extensive testings on the settings and concentrated on the images 

and a message to the developers:
thanks again. just a few days before the raw "hack" got public i explained to a friend of mine why it´s just a merketing thing, that you can´t buy a camera for € 3000 that records 2k raw. now nobody can say anymore, it´s impossible for that money.
and THIS is, what truly matters!!

hi everyone,

here´s some low and shallow light testing. half of the material was deleted by a mistake and i had just a photo ball head tripod...
yeah.... and it´s a little bit trippy 

workflow this time@work:
ACR=>AE (resizer to 1080p)=>dnxhd 10bit=>Media Composer

Very nice! What resolution and fps did you shoot at, and what pp workflow did you use? You've got a lot more detail in those shots than I have been able to get.

Shot entirely in 1152 x 432 @ 23,976 fps 14 bit  (RGB) RAW
Transcend 32 GB Class 10 (mem hack enabled)
shown bitrate whilst recording: up to 19,5 MB/s after a few seconds "warm up"

Flow:
- rawanizer=> dng
- canon raw software => tiff ( "grading" done completely in raw then exported as 12 bit tiffs)
- Quicktime => dnxhd 10bit
- editing/scaling in lightworks

this is a very slow workflow. i chose it because it gave me the most speed in developing the RAWs (since i´m used to photo raw since years) and maintained the image quality through the rest of the process. no furthermore color adjustments happened in the editing process.

i´m still waiting for a workflow to generate DNGs, that can be imported straight into resolve and treat the files as native camera raw.  can´t resolve do 14 bit DNGs? it just opens the preview jpegs....


in the future a special ML RAW camera specific LUT (picture profile to address CA and maybe aliasing) for recording and a resolve specific RAW setting (debayering and optimized color matrix) would be the absolute killer-killer icing on the cake feature.

lenses used:
- canon 28 mm  f 2.8
- canon 18-55 mm f 3.5-5.6
- canon 50 mm f 1.8
+ ND 4  Filter (Cokin)

i always used an aperture above f 5 an below f 14 to achieve the best possible sharpness and mostly exposed for the highlights. exposing the highlights till they´re not overexposed and then adding one f-stop more turned out to be always recoverable while developing. sometimes even more.
shadows and blacks are pushed till a nearly flat image is achieved. then fine tune with curves to get contrast and adjust gamma. here and there some color tweaks, minor sharpening, noise reduction.
activate CA reduction and edge defringing. no further lens corrections. there should be no lens distortion or vignetting since the huge crop factor.

but there are some limitations:
you can´t go to the maximums when pushing or squeezing luminance or contrast color channel wise. ( in ACR this is the place where you can read such things as "aquas" or "magentas" )
when using extreme settings you´ll get ugly noise. not visible in a still, but very noticeable in moving image sequences.

in my opinion color resolution always beats pixel resolution. sharpening and scaling algorithms benefit so much from the added image information. also the much better motion resolution is incredible (the reason why twixtor or nukes kronos can achieve much better interpolated images)

i hope, this information is sufficient for now 

and i hope some skilled people are able to address the memory controllers limitations physically. maybe a way to replace it with a faster one or to convert the data lines into IDE or even SATA.... well.....i´m just dreaming   

hello!

i´m completely new to this forum. first of all: thank you all for that incredible software!
i did some testshots with my beloved 550D.

setup is in the video specs.