Author Topic: 7D Raw Thread  (Read 327907 times)

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 7D Raw Thread
« Reply #2050 on: December 01, 2013, 11:08:23 PM »
Quote
I use a VAF7D (which I highly recommend for the 1278x972 or standard formats

Its not designed to shoot crop mode only the 1x, the sensor readout is different. For the 1728x1152 it would rock I guess but for crop make things blurry no matter the focus.

IDX is the index of the chunks, you can delete it if you want. I believe in MLV they are not sequential so you can have CF/SD writing at the same time to 2 different files on 5DIII... yea does jack for us.

BushanBengaluru

  • New to the forum
  • *
  • Posts: 25
Re: 7D Raw Thread
« Reply #2051 on: December 02, 2013, 07:51:25 AM »

Started testing the RAW, Very excited.

Thank you all.. i'm loving it..


mrnv45

  • Member
  • ***
  • Posts: 140
Re: 7D Raw Thread
« Reply #2053 on: December 02, 2013, 10:56:17 AM »

tonybeccar

  • Senior
  • ****
  • Posts: 319
  • 7D
Re: 7D Raw Thread
« Reply #2054 on: December 02, 2013, 11:47:35 AM »
Hey all!! Very pleased with the last versions of ML.. really loving the digic peaking, thanks 1%, it appears to work flawlessly!! :)

Although I'd like to report a bug, a bit dangerous one.. very weird!!:
Randomly while using the camera, when I hit record I will get the canon buffer bars, which will fill in 2 seconds. Now, here comes the confusing part: I get 2 seconds of freezed live view, after that the rec icon on the top-right is still present, although nothing is recording! I check because then I have only a 2 second file.

I cant really tell when or how to reproduce the bug, it happened twice. Very random I guess.. I wasnt using anything fancy, I did however used the digic peaking.. but cant really tell if that's the problem. I think I had modules loaded but none activated via menus. I use a pretty basic ML while working. I did not get any crash report logs with this.

Does anyone know something about this?

BTW, I'm using Transcend 166x 32gb cards. I never got any buffer warnings ever.

Cheers!!!

DFM

  • Freshman
  • **
  • Posts: 86
Re: 7D Raw Thread
« Reply #2055 on: December 02, 2013, 01:49:00 PM »
Hello!!!!I would like to know, wich XRITE do you use to create color profile???thank's!!!

X-Rite's ColorChecker products come with their own profiling software. Normally people use it as a Lr plugin but it runs in standalone mode, you feed it any untouched DNG that shows the card and it spits out an ICC profile; which you can then apply to the ML DNG sequence through the Camera Calibration panel in ACR.

hdclip

  • Freshman
  • **
  • Posts: 72
  • Canon 7D Lexar 32Gb X1000
Re: 7D Raw Thread
« Reply #2056 on: December 02, 2013, 03:27:52 PM »
X-Rite's ColorChecker products come with their own profiling software. Normally people use it as a Lr plugin but it runs in standalone mode, you feed it any untouched DNG that shows the card and it spits out an ICC profile; which you can then apply to the ML DNG sequence through the Camera Calibration panel in ACR.
Ok, Thank´s!!!!!!! ;)
Canon 7D Sandisk Extreme 60MB/seg, Lexar 32Gb X1000

m1ke

  • New to the forum
  • *
  • Posts: 6
Re: 7D Raw Thread
« Reply #2057 on: December 02, 2013, 03:38:36 PM »
Devs - An Observation on Cut-Up/Pink-Frames

I've been using ML Raw Video (raw rec) on a 7D and find it is hard to totally eliminate the occasional cut-up / pink frame:



If recording with Global Draw on, I get approximately 1 bad frame per 1000 good frames.  Not a lot, but enough to be annoying to edit out.  Turning off Global Draw reduces but does not totally eliminate it.  Looking over a number of the videos, I have realized the cut-up/pink-frames are ALWAYS ON ODD FRAMES (with the first frame in the sequence being frame zero). 

Diving into the code (raw_rec.c), this corresponds to transfers from the ML allocated memory buffer used in the liveview double-buffering scheme (i.e. fullsize_buffers[0]) as opposed to Canons' buffer (full-size_buffer[1]). 

I don't know why the ML allocated memory would be more more prone to corrupt frames than the default Canon buffer - but it seems like it is. 

As an experiment, I simply changed the code to eliminate the double buffering.  The Canon buffer is used by default and I removed the call to raw_lv_redirect_edmac().  I have looked at over 30,000 frames taken with this change while running with Global Draw on, Histogram on and Spotmeter on - mostly non-cropped (but some cropped) at 24FPS, 1728*972.  Not a single cut-up/pink-or-corrupted-frame yet.

Given this, I am wondering whether double-buffering is strictly needed.  At the end of the day, the liveview buffer on average needs to get offloaded before the next vsync cycle.  As the transfers are executed off-CPU (by EDMAC) and the transfers are set up within callbacks (are these done within the context of interrupts?) - even with a busy CPU, they should be able to complete reliably in time.

So I am looking for any feedback from the devs on this.  Was double-buffering added to solve a specific problem (that might now have been addressed by more sophisticated FIO buffering)?  Is this likely to be a 7D-only fix with other models requiring the double-buffering? For sure I need to do some more testing with larger resolutions, 30 FPS, etc.

Thanks



.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 7D Raw Thread
« Reply #2058 on: December 02, 2013, 04:12:44 PM »
Great find.

Double buffering is used to avoid problems like duplicate/skipped frames or tearing (that is, while a buffer is being filled, the other one is being cropped and queued for writing). The method of switching the buffer is not perfect, and I think that one is more likely to cause this problem.

arrinkiiii

  • Hero Member
  • *****
  • Posts: 1101
  • ML ADDICTED 7D
Re: 7D Raw Thread
« Reply #2059 on: December 02, 2013, 06:48:36 PM »

Can this be the end of pink/corrupted frames ??   :D

mucher

  • Member
  • ***
  • Posts: 121
Re: 7D Raw Thread
« Reply #2060 on: December 02, 2013, 08:18:36 PM »
I personally, without any supporting evidence, think that this pink frame issue is caused by hardware. If you remember the news that 7D will sometimes mess up two pictures into one when 7D comes out, I believe that is the reason. It might be caused by CPU or DMA chanel is not coping fast enough. So, if it is hardware, the issue will be innate, and there is no way to be fixed by software. One feasible solution might be to reduce to write speed below the threshold (if there is any) for pink frames. Based on these above speculations, I have a hard time to understand 7D's buffer technologies, it might be good for those titan cameras to squeeze out some extra MBs, but for a slow camera like 7D, most people, for example me :D, might rather stay with 1728 x XXX resolution without expecting any miracles to happen soon, just to choose the safe side. So the buffer technology that is employing on 7D recent build might be way way way too aggressive for that poor camera. :D

It has been long time in my chess, but it is only speculation so it is just to speak out my chess.

chris bouk

  • New to the forum
  • *
  • Posts: 34
Re: 7D Raw Thread
« Reply #2061 on: December 02, 2013, 10:20:30 PM »


IDX is the index of the chunks, you can delete it if you want. I believe in MLV they are not sequential so you can have CF/SD writing at the same time to 2 different files on 5DIII... yea does jack for us.
what do you mean exactly...did I wrote two different files...at the same time?
...sounds good....I have to test it again...!!!which is better, MLV or RAW???
For the moment I'm searching for RAW 2K video and I've got some good results, even with a 233X CF memory card, but only for a few seconds...
To tell you the truth I want to test for longer periods of time and see what happen...
but I have problems when watching the results, if a RAW video is in the card, I can't see my pictures taken in the same card...I have to disable RAW (off) and then reboot or play with Canon's menu. Is it normal, or I'm doing something wrong?

jayzed

  • New to the forum
  • *
  • Posts: 37
Re: 7D Raw Thread
« Reply #2062 on: December 02, 2013, 11:02:59 PM »
Thanks for the comment on the VAF-7D, 1%. What I was thinking -
(note, hypothesis on my part)
Mosaic filter is designed to work specifically at the 1728x972 pixel (not photosite) resolution, which is achieved by a particular formula of line skipping. From what I have read when the 7D records standard 1080p it is upscaled from this res anyway. Frankly, I'd rather do any upscaling myself, save a few CPU cycles during editing/post and choose exactly which algorithm and sharpening to use if I want 1080p. I'm unlikely to be producing video for anything other than the web or computer screen for the forseeable future anyway so I'm actually more likely to downscale to 720p. Doing this, with the raw files, I'm avoiding the upscale/downscale and going straight to the downscale from 972p!
Crop modes are closer to/(exactly?) a 1:1 pixel correspondence (probably not strictly true as we're talking about photosites but I'm simplifying by concentrating on the output, not the pre de-bayer) so the VAF-7D, which is effectively a clever form of blurring, is fighting me. Reducing image quality.
Does this make sense?
I'm risking being one of those people making ignorant statements about topics they don't know that much about but all the above certainly seems plausible to me.

Pelican

  • Contributor
  • Senior
  • *****
  • Posts: 406
Re: 7D Raw Thread
« Reply #2063 on: December 03, 2013, 01:34:56 AM »
I personally, without any supporting evidence, think that this pink frame issue is caused by hardware. If you remember the news that 7D will sometimes mess up two pictures into one when 7D comes out, I believe that is the reason. It might be caused by CPU or DMA chanel is not coping fast enough. So, if it is hardware, the issue will be innate, and there is no way to be fixed by software.
I bought my 7D on the first day and never had noticed that messed up pictures...
After you read m1ke's post even strange what you think... :P
EOS 7D Mark II, EOS 7D, EOS 5, EOS 100 + lenses (10mm to 300mm), 600EX, 550EX, YN600EX x 3
EOScard, EOS DSLR firmwares, ARMu, NiControl, etc.: http://pel.hu/down

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 7D Raw Thread
« Reply #2064 on: December 03, 2013, 01:47:06 AM »
Quote
Does anyone know something about this?

Maybe double buffering as a setting and trying it on different cameras... wonder what will happen...

CPU usage and canon processes will definitely do it. Killing GD on record and turning off canon dialog refresh gets rid of 99% of pink frames for me even at the highest 1x res.

I added stuff like unlock expo while recording and full time color preview so maybe those would work with only single buffered video?

Quote
which is effectively a clever form of blurring, is fighting me. Reducing image quality.

Yep, pretty much. It gets rid of aliasing via a blur deal so when you read 1:1/3 you can see the blur and your video is never sharp.


Ok, its done... its faster but dunno how tearing/etc will go. You might get some more 2k out of it :)

arrinkiiii

  • Hero Member
  • *****
  • Posts: 1101
  • ML ADDICTED 7D
Re: 7D Raw Thread
« Reply #2065 on: December 03, 2013, 10:24:52 AM »
http://tl.bot-fly.com/   it's again down??

hdclip

  • Freshman
  • **
  • Posts: 72
  • Canon 7D Lexar 32Gb X1000
Re: 7D Raw Thread
« Reply #2066 on: December 03, 2013, 11:04:25 AM »
http://tl.bot-fly.com/   it's again down??
Yes is Down...if you need the last 28Nov, I can give you....
Canon 7D Sandisk Extreme 60MB/seg, Lexar 32Gb X1000

hdclip

  • Freshman
  • **
  • Posts: 72
  • Canon 7D Lexar 32Gb X1000
Re: 7D Raw Thread
« Reply #2067 on: December 03, 2013, 11:07:25 AM »
Anyone know how to set the recording to 50fps, if I change the settings to canon 1280X720 50fps and 50 fps magic all right but as for the aspect ratio becomes crazy
Canon 7D Sandisk Extreme 60MB/seg, Lexar 32Gb X1000

arrinkiiii

  • Hero Member
  • *****
  • Posts: 1101
  • ML ADDICTED 7D
Re: 7D Raw Thread
« Reply #2068 on: December 03, 2013, 12:09:30 PM »
Yes is Down...if you need the last 28Nov, I can give you....

Thanks hdclip  :D   ...but i want to try the new build, from yesterday  :D


For what i remember, you change in the canon menu and then change in ML menu the fps and lock the fps... Wend i try it i did this and everything went alright.

jayzed

  • New to the forum
  • *
  • Posts: 37
Re: 7D Raw Thread
« Reply #2069 on: December 03, 2013, 03:03:42 PM »
Yep, 1%. I've tested it, too - removal of VAF brings back nice sharp video using crop mode. I can even get a second or so of 2.5K - sharp as a tack and after quick reviewing in AE, haven't noticed any hideous moiré. Once again, I am gobsmacked at the work you guys are doing.

arrinkiiii

  • Hero Member
  • *****
  • Posts: 1101
  • ML ADDICTED 7D
Re: 7D Raw Thread
« Reply #2070 on: December 03, 2013, 03:29:44 PM »
In crop mode (all lines read out) no moire. The moire only show up wend recorded in normal mode, in that process the reading of the sensor is in "skip line" mode and with this the moire is there. 

-Meanwhile i receive right now my first 1000x  card  (KB 32gb) by mail, nice present from girlfriend  ;D   Going to benchmark this card... after doing that do i need to make any math calculations with the numbers that the benchmark give's me, for see how much this car-d can go in speed??   AHUUUUUUUUUUU   :D

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: 7D Raw Thread
« Reply #2071 on: December 03, 2013, 04:53:36 PM »
heh, it was up like yesterday... sucks because I wonder how the single buffering is going to work out in the wild.

arrinkiiii

  • Hero Member
  • *****
  • Posts: 1101
  • ML ADDICTED 7D
Re: 7D Raw Thread
« Reply #2072 on: December 03, 2013, 05:10:00 PM »

FUCK...!!!!   My KB3 2gb 1000x   is F*#&ing DEAD  :-[     Can't format in camera or in computer !!! Even i use the kingston fcr-hs3 3.0 usb card reader (that i have already for 2 or 3 months).  Not even 15 minutes with the damn card !!!

Komputerbay - This is a love/hate relationship. I love u guys but for the first time i get a KB card and this is what happen???


chris bouk

  • New to the forum
  • *
  • Posts: 34
Re: 7D Raw Thread
« Reply #2073 on: December 03, 2013, 05:10:39 PM »
...can someone explain to me please, how it is possible to view images when you have RAW video also in the CF card into the camera?
I always have to finish watch the videos and then when is the pictures turn, I can't see theme in the LCD screen of my camera....! the frames of the RAW video are loading, normally but what may I have to do to watch the pictures...? it's the third time that I'm asking this question.... :-[

arrinkiiii

  • Hero Member
  • *****
  • Posts: 1101
  • ML ADDICTED 7D
Re: 7D Raw Thread
« Reply #2074 on: December 03, 2013, 05:15:54 PM »
...can someone explain to me please, how it is possible to view images when you have RAW video also in the CF card into the camera?
I always have to finish watch the videos and then when is the pictures turn, I can't see theme in the LCD screen of my camera....! the frames of the RAW video are loading, normally but what may I have to do to watch the pictures...? it's the third time that I'm asking this question.... :-[

Good question. Maybe you must un-load the module that makes the raw player (play_mo module?)