Python not returning all data when reading file chunk by chunk

I'm encrypting some files using python, but I'm having problems with reading the file chunk by chunk.

It's sometimes not returning all data of the last chunk.

When the file is 307200 bytes long, I have no problems. When it's 279363 bytes long, I do.

I ran this piece of code over 2 files (respectively 307200 & 279363 bytes big)

chunksize = 65536
w = open(filename + '.' + str(cs) + '.split', 'wb')
tdata = f.开发者_JAVA技巧read(307200)
w.write(tdata)
w.close

infile.open(filename + '.' + str(cs) + '.split', 'rb')

while True:
    chunk = infile.read(chunksize)
    print "Chunk length: " + str(len(chunk))
    if len(chunk) == 0:
        break
    elif len(chunk) % 16 != 0:
        chunk += ' ' * (16 - len(chunk) % 16)

And I got this:

File 1 (307200):
Chunk length: 65536
Chunk length: 65536
Chunk length: 65536
Chunk length: 65536
Chunk length: 45056
Chunk length: 0

File 2 (279363):
Chunk length: 65536
Chunk length: 65536
Chunk length: 65536
Chunk length: 65536
Chunk length: 16384
Chunk length: 0

As you can see, the sum of all the chunk sizes of the first file are correct, the sum of the second file are NOT. And I have NO idea why.

My Python version is 2.6.5

(Running on Ubuntu 10.04.2 LTS)

*Edit: My own stupid fault. I used w.close instead of w.close() before reading the file i just wrote, and that caused the problem.

Are you running on Windows? If so you may need to open your files in binary mode:

infile = open('somefile','rb')

I feel very stupid now, but I'm kind of glad nobody else noticed the same problem.

I was calling w.close, not w.close()

That's odd.

Did you create the second file as the first one, from the file of name 'filename' and replacing 307200 with 279363 ?

By the way, it's a strange extension for a name of file: '.split'

.

I propose you to run this code:

from os.path import getsize

chunksize = 65536

for x in xrange(279363,307201):
    w = open(filename + '.' + str(cs) + '.split', 'wb')
    tdata = f.read(x)
    w.write(tdata)
    w.close

    siz = getsize(filename + '.' + str(cs) + '.split')
    if siz!=x:
        print 'file has not the right size'
        print 'x=='+str(x)+'   size of created file : '+str(siz)

    infile.open(filename + '.' + str(cs) + '.split', 'rb')

    li = []
    while True:
        chunk = infile.read(chunksize)
        li.append(str(len(chunk)))
        if len(chunk) == 0:
            break

    if sum(li)==x:
        print 'good at '+str(x)
        break

Try also

from os import fsync

chunksize = 65536 
w = open(filename + '.' + str(cs) + '.split', 'wb') 
tdata = f.read(307200) 
w.write(tdata)
w.flush()
fsync(w.fileno())
w.close