Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: miyake on August 21, 2012, 03:21:10 PM

Title: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 21, 2012, 03:21:10 PM
I don't wanna use SD-reader and , open the SD-card door.

https://bitbucket.org/miyake_t/magic-lanternptpdev/
Current repo's code and ptpcam combination can upload 95% of files correctly.
But it's will stop last 6k with error.(only 600D's autoexec.bin)
transferd 240k is correct values .(confirmed "od -x" and diff)


History and knowledge
-based on g3gg0's patch
-added libptp-chdk-win32(It's a GPL so we can redistribute with source)
-src/ptpcam.txt is described how ptpcam is working(how extending ptpcam by chdk guys).
-(solved)enum value is different ML and ptpcam
-(solved)current ptpcam code is not supported over 1000byte files to transfer
-(solved)current ML codes are not used callback function for receiving from USB.(If we dont use it, got hangup receive process)
-(solved)current ML codes not handle USB ack packet(Simply discarding now.)
*it's 12byte data adding each after part of USB_BULK_WRITE data.
-USB_BULK_WRITE is sending 500byte data .
-console_printf added for debugging


I'm coding for this code 1week . But I can't solved lask 6k issue. So please help me to find bugs in this code, if you interested in this.
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: ilguercio on August 21, 2012, 03:26:21 PM
Why don't you compile a bin for each camera?
Since it's not good to use the same card for different cameras it won't be a big issue if we had a single file for each DSLR.
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: nanomad on August 21, 2012, 03:50:08 PM
The point of the unfified ML tree is not to have a tons of autoexec.bin around :-\
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 21, 2012, 03:58:35 PM
Quote from: ilguercio on August 21, 2012, 03:26:21 PM
Why don't you compile a bin for each camera?
Since it's not good to use the same card for different cameras it won't be a big issue if we had a single file for each DSLR.

I can't understand what you are advice to me .
What relation is existing about "solving PTP codes" and "each camera autoexec.bin" ?

Sorry for my poor English reading.
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: SDX on August 22, 2012, 11:07:08 PM
This actually is quite cool. I have thought of exact the same thing for a few days now.

What I think ilguercio means is, that he would like to have a autoexec.bin for each camera model. Maybe that would help the 6k issue on 600D?
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 23, 2012, 01:20:55 AM
@SDX
Let's solve this bug together.

BTW, I'm using 600D autoexec.bin(make 600D) , then I got 6k bug. In addition, I was transport  platform/all/autoexec.bin. It's 1.7M. But I could transport over 250k. I forgot detailed value. But I can transport about 1.5M.
Ratio is almost same , so I think the code has bug.(loop condition had bug, I guess).
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: ilguercio on August 23, 2012, 01:39:15 AM
Quote from: nanomad on August 21, 2012, 03:50:08 PM
The point of the unfified ML tree is not to have a tons of autoexec.bin around :-\
Not only, it means that the code is shared among the cameras. It doesn't mean it has to be in one single file at any cost.
FW files are specific for each camera.
Anyway, a single bin file is good until it is too large that it can burden the startup sequence. I don't know if it made any difference if it was done for each camera.
SDX has understood me well, it was just a thought so it could help avoiding this bug but it isn't a solution for sure.
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: 1% on August 23, 2012, 06:39:09 PM
Try to send file with a different name/ext? Or just after certain size it cuts off? The loader you made can then load the bin or whatever extension. I think its better to make it autoexec.ae vs AE_autoexec.bin, what do you think? Then function renames it properly.

Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 24, 2012, 05:35:43 AM
Quote from: 1% on August 23, 2012, 06:39:09 PM
Try to send file with a different name/ext? Or just after certain size it cuts off? The loader you made can then load the bin or whatever extension. I think its better to make it autoexec.ae vs AE_autoexec.bin, what do you think? Then function renames it properly.

I'm always use different file name to store transferd files. and cuts off size is different of each files(different size).So I think my code's loop has a bug , but I can't find it.

Filename has 8.3 limitation, and my code is already have function which is copy .bin and .cfg together.
So we can't use extention. and "AE_" prefix is easy to identify by human. also we can remained 5 character to use identify the file.
Now AE_XXXXX.BIN and AE_XXXXX.CFG will overwrite working autoexec.bin and magic.cfg file.
We can more consideration on prefix naming.
Therefor I'm using AE_ prefix for naming.
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: SDX on August 24, 2012, 09:04:43 PM
I must admit that I don't know much about PTP. But here are my ideas, what could go wrong.


Again, I just barely know that PTP is a protocol, but have no clue on how that stuff internally works.

Edit: I just noticed:
Quote-(solved)current ML codes not handle USB ack packet(Simply discarding now.)
*it's 12byte data adding each after part of USB_BULK_WRITE data.
-USB_BULK_WRITE is sending 500byte data
- Maybe these 12bytes mess up your calculation of the size? I know, 12bytes on 500bytes just makes approx 2%, not 9%. Maybe not as useful as you properly hoped - I do what I can ;)

Another Edit: (I just noticed, that the crap I previously wrote was absolutely useless)
I just found enough motivation to take a closer look at the code. It looks totally okay to me. I assume that fninfo_len is the length of some sort of header which is being skipped, right?

- I would write the received data into a buffer instead of writing it directly into the file. Doing all the writing once is faster.
- Have you tried to simply save everything you receive? Then you could send a small file, 2k or something and take a closer look at, what it ends up with.
- And finally: is context->get_data_size() with or without headers, signatures and that kind of stuff.

Btw: how are you sending files?
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 25, 2012, 12:42:17 PM
QuoteI assume that fninfo_len is the length of some sort of header which is being skipped, right?
yes. The transferd byte stream has  4byte filename length and next variable length filename, then actual transfer file data.

Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: SDX on August 25, 2012, 01:55:30 PM
Okay

I would like to take a closer look at what you have done and what doesn't work. I just need to know one little thing: how to you send the files? Are you using ptpcam upload or your own application?
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 25, 2012, 02:30:58 PM
my repo has a customized  libptp.  So you can use ptpcam .
And it's a extended for win32 . But I think it can use on linux/Mac, maybe. Try it if you don't use win32.

Then,
$ pcpcam --chdk
<conn>  upload ./autoexec.bin B:/AE_PTP1.BIN


Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: SDX on August 25, 2012, 04:40:12 PM
Strange issue here:
I'm not able to establish a stable connection. Most of the times I get an error like "Failed to connect". Only the first few seconds after connecting to the camera it is possible to execute something that works. Also, Windows comes up with a "overview-of-the-camera" window, which closes and opens every few seconds. I also managed to crash ptpcam.exe *proud*.

Btw: what value does get_data_size() return? Have you compared it with the size of the file? (I should be slightly bigger than your file, of course).
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 25, 2012, 04:43:15 PM
@SDX

I got hint from your comment.
Yey.Thankyou.

This is transfer file
$ ls -la autoexec.bin
-rwxr-xr-x 1 miyake None 299008 8月  25 18:49 autoexec.bin


299008byte - 7026byte = 291982byte (un-transferd 7026k)
291982byte / 500byte = 583 (500byte is USB_BULK_TRANSFER_SIZE)
583 * 12byte = 6996byte (12byte over head of transfer)


So I think
1: USB bulk header descrived it's size. We can get from "context->get_data_size (context->handle);"
      (This is actually 299008byte(miss taken: 299008+fn_len + fn) )
2: Our receiving code is uging it data size.
3: But windows?libusb? is sending 12 byte data when sending after each BULK packet.
Then
6k is remained to transfer  on our while routine. But Canon's ptp protocol implementation is not understanding additional 12 byte data. and It's will be finish the transfer. Because we already transferd 299008byte including additional 12byte data.
4: Then we got error from recv_ptp_data()

That's my guess.

I'm already read all of code which  sending all data on libptp and libusb. It's not added 12 byte.
And I confirmed transferd raw data  by hex on libptp side . But it's correct with original hex.
So Next, we need to find how to disable 12 byte data. The libusb is using windows native function to transport by USB.
I guess, another header option will disable 12 byte data.... It's just my guess.

Anyway, If we can disable 12 byte data, we can get success to transport it.

Also, current CHDK's ptp.c has no 12byte data handring.
So I think , PC side program need some trick to disable 12 bytedata.

Will find  BULK header option

MSG added
hmm, I have no unstable connection. It's really good and stable . I'm compiled cygwin libusb. and Win7 64 bit.
The libusb driver changing by downloadable libusb-win32-bin-1.2.6.0/bin/inf-wizerd.exe.

and size is correct value on my side.
size is = filename len(4byte) + actual filename + actual file size
confirmed.
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: miyake on August 25, 2012, 05:56:41 PM
I found libusb/src/windows.c

LIBUSB_WIN32_DLL_LARGE_TRANSFER_SUPPORT
entry.

So we don't need to limit transport file size on our userland applications.
So we don't use BULK_MAX_SIZE(500Byte).
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: Pelican on September 12, 2013, 01:50:28 PM
Is this working now?
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: SDX on September 14, 2013, 11:48:16 PM
It was never finished.
Title: Re: PTP autoexec.bin uploading(save SD-card door project)
Post by: Pelican on September 15, 2013, 02:22:21 AM
Thanks  :(