Android ListView Vanishing Images

I have a ListView that I'm populating with names and images (if one exists) for each contact on my phone. This works fine when the app initially loads, but once I start to scroll through the ListView the images that were originally correct disappear. The only thing I can think of is that the resources are used and then destroyed once they scroll off the screen, but I am not having any luck finding out how to keep them around.

I'm looping through all of my contacts and storing the name and using ContactsContract.Contacts.openContactPhotoInputStream(context.getContentResolver(), photoUri); to retrieve an InputStream of the contact's image. Then, in my custom ArrayAdapter, I am using Drawable.createFromStream() to set the image for the ImageView of my ListView item.

Thanks!

Edit: As requested, here is my getView method

public View getView(int position, View convertView, ViewGroup parent)
{
    LinearLayout contact_view;
    //Get the current alert object
    ContactInfo contact = getItem(position);

    //Inflate the view
    if(convertView==null)
    {
        contact_view = new LinearLayout(getContext());
        String inflater = Context.LAYOUT_INFLATER_SERVICE;
        LayoutInflater vi;
        vi = (LayoutInflater)getContext().getSyste开发者_开发问答mService(inflater);
        vi.inflate(resource, contact_view, true);
    }
    else
    {
        contact_view = (LinearLayout) convertView;
    }

    //Get the fields to populate from the listitem.xml file
    TextView contact_name = (TextView)contact_view.findViewById(R.id.contact_name);
    ImageView contact_image =(ImageView)contact_view.findViewById(R.id.contact_image);

    //Assign the appropriate data from our alert object above
    contact_name.setText(contact.get_name());
    if(contact.get_contact_image() != null) {
        contact_image.setImageDrawable(Drawable.createFromStream(contact.get_contact_image(), "src"));
    } else {
        contact_image.setImageResource(R.drawable.dummy_image);
    }

    return contact_view;
}

Once you have read from an InputStream, you reach its end and can't reread it without requesting a new InputStream from the provider.