modified a given list inside a function

i would like to assign new values to the list i pass to my functions. At the case of get_p_values it works fine, but at `read_occupation' not (I get as an output: [[0], [0], [0], [0]] instead of [[1,2,3], [1,2,3], [1,2,3], [1,2,3]]).

Remark, the reason i pass to function read_occupation the list matrix[ index ] is开发者_C百科 that other wise i would need to use acquire() and release() of Lock.

#!/usr/bin/python
# Filename: untitled.py
# encoding: utf-8

from __future__ import division
from multiprocessing import Process

def read_occupation( matrix ):
    matrix = [ [1,2,3] ]

def get_p_values( p):
    for index,value in enumerate( xrange(0,3,1)):
        p[index] = value

def main():
    p         = [ [0] ] * 3 
    matrix    = [ [0] ] * 4

    get_p_values( p )
    print p

    for index in xrange(0,3,2):
            p1 = Process( target=read_occupation, args=( matrix[ index     ] ) )
            p2 = Process( target=read_occupation, args=( matrix[ index + 1 ] ) )

            p1.start()
            p2.start()

            p1.join()
            p2.join()

    print matrix


if __name__ == '__main__':
    main()

Except for what Sylvain said, of course it still didn't work because a process can't modify the variables of its parent process. The variables are copied when a new process is created.

You can try the following example.

from multiprocessing import Process

def modify(idx, li):
    li[idx] = 'hello'

a = [0] * 3 

p = Process(target=modify, args=(1, a))
p.start()
p.join()
print a

modify(1, a)
print a

The result will be:

[0, 0, 0]
[0, 'hello', 0]

There are many reason why it is not working.

First, in the function read_occupation, you are changing the binding of a local variable. The original object is left untouched. On the contrary, in get_p_values, you are modifying the object (the [] operator call the __setitem__ function of the object that change the internal representation). A better idea in this case would be to use a proper object.

If you want to change the whole content of the list in read_occupation, you can use the list splicing operator to assign to the whole list:

def read_occupation(matrix):
    matrix[:] = [ [1, 2, 3] ]

BTW, if you call you function read_occupation the caller will probably expect it not to change its parameter. You should probably rename it update_occupation or something like that.

Second, when you create your variables, via multiplication, you get a list where every index contains a reference to the same item. The code p = [ [0] ] * 3 is equivalent to:

>>> l = [0]          # Naming the list for more clarity
>>> p = [ l, l, l ]  # Each index points to the same list `l`

Third, the Process class expect a tuple (or in fact an object following the iterable protocol) for its args parameter. You are not passing a tuple, but a single item that happens to be a list of one int (which is why you get a TypeError exception). You need to use the following syntax to pass a tuple:

# Please note the comma after the closing bracket, it is important
p1 = Process( target=read_occupation, args=( matrix[ index     ], ) )

# In python, the syntax for a tuple is weird for tuple of size < 2
#  . tuple of 0 item:  ()
#  . tuple of 1 item:  (1,)
#  . tuple of 2 items: (1, 2)

Fourth, you use multiprocessing.Process that spawn a new process in which your code is executed. There is no communication back in your code (I don't know if it is possible to communicate back from this process to the original one). So the variable in the original code is not modified.

mp.Manager provides a way to share a Python list across multiple processes:

from __future__ import division
import multiprocessing as mp   

def read_occupation( matrix,i ):
    matrix[i]=[1,2,3]

def main():
    manager=mp.Manager()
    matrix=manager.list([ [0] ] * 4)

    for index in xrange(0,3,2):
        p1 = mp.Process( target=read_occupation, args=(matrix,index) )
        p2 = mp.Process( target=read_occupation, args=(matrix,index+1) )

        p1.start()
        p2.start()

        p1.join()
        p2.join()

    print matrix

if __name__ == '__main__':
    main()

yields

[[1, 2, 3], [1, 2, 3], [1, 2, 3], [1, 2, 3]]

This block is misleading:

p         = [ [0] ] * 3 
matrix    = [ [0] ] * 4

try doing p[0].append(0), and you will get:

[[0, 0], [0, 0], [0, 0]]

because [[0]] * 3 creates a list of three objects all pointing to the same list.

Worse: I'm not sure if your program will work if you fix this + what sylvain said, because p1 and p2 will try to access the same list. You should read carefully the multiprocessing doc before using it. It is a very tricky lib, full of nasty traps.