Keeping the downloaded torrent in memory rather than file libtorrent

Working with Rasterbar libtorrent I dont want the downloaded data to sit on my hard drive rather a pipe or variable or something Soft so I can redirect it to somewhere else, Mysql, or even trash if it is not what I want, is there anyway of doing this in preferably python binding if not in C++ using Libtorrent?

EDIT:--> I li开发者_如何学运维ke to point out this is a libtorrent question not a Linux file handling or Python file handling question. I need to tell libtorrent to instead of save the file traditionally in a normal file save it to my python pipe or variable or etc.

You can do this by implementing your own storage class to use with libtorrent. Unfortunately this is not possible to do in python, but you can do it in c++. The documentation for it is a bit scarce and can be found here.

Here's a simple example of how to do this by storing all the data in RAM:

struct temp_storage : storage_interface
{
   temp_storage(file_storage const& fs) : m_files(fs) {}
   virtual bool initialize(bool allocate_files) { return false; }
   virtual bool has_any_file() { return false; }
   virtual int read(char* buf, int slot, int offset, int size)
   {
      std::map<int, std::vector<char> >::const_iterator i = m_file_data.find(slot);
      if (i == m_file_data.end()) return 0;
      int available = i->second.size() - offset;
      if (available <= 0) return 0;
      if (available > size) available = size;
      memcpy(buf, &i->second[offset], available);
      return available;
   }
   virtual int write(const char* buf, int slot, int offset, int size)
   {
      std::vector<char>& data = m_file_data[slot];
      if (data.size() < offset + size) data.resize(offset + size);
      std::memcpy(&data[offset], buf, size);
      return size;
   }
   virtual bool rename_file(int file, std::string const& new_name) { assert(false); return false; }
   virtual bool move_storage(std::string const& save_path) { return false; }
   virtual bool verify_resume_data(lazy_entry const& rd, error_code& error) { return false; }
   virtual bool write_resume_data(entry& rd) const { return false; }
   virtual bool move_slot(int src_slot, int dst_slot) { assert(false); return false; }
   virtual bool swap_slots(int slot1, int slot2) { assert(false); return false; }
   virtual bool swap_slots3(int slot1, int slot2, int slot3) { assert(false); return false; }
   virtual size_type physical_offset(int slot, int offset) { return slot * m_files.piece_length() + offset; };
   virtual sha1_hash hash_for_slot(int slot, partial_hash& ph, int piece_size)
   {
      int left = piece_size - ph.offset;
      TORRENT_ASSERT(left >= 0);
      if (left > 0)
      {
         std::vector<char>& data = m_file_data[slot];
         // if there are padding files, those blocks will be considered
         // completed even though they haven't been written to the storage.
         // in this case, just extend the piece buffer to its full size
         // and fill it with zeroes.
         if (data.size() < piece_size) data.resize(piece_size, 0);
         ph.h.update(&data[ph.offset], left);
      }
      return ph.h.final();
   }
   virtual bool release_files() { return false; }
   virtual bool delete_files() { return false; }

   std::map<int, std::vector<char> > m_file_data;
   file_storage m_files;
};

You'd also need a constructor function to pass in through the add_torrent_params struct when adding a torrent:

storage_interface* temp_storage_constructor(
   file_storage const& fs, file_storage const* mapped
   , std::string const& path, file_pool& fp
   , std::vector<boost::uint8_t> const& prio)
{
   return new temp_storage(fs);
}

From this point it should be fairly straight forward to store it in a MySQL database or any other back-end.

If you're on Linux, you could torrent into a tmpfs mount; this will avoid writing to disk. That said, this obviously means you're storing large files in RAM; make sure you have enough memory to deal with this.

Note also that most Linux distributions have a tmpfs mount at /dev/shm, so you could simply point libtorrent to a file there.

I've implemented a torrent client in Go just for this purpose. I wanted to able to handle and control the data directly, for use in writing torrentfs, and to have storage backends to S3 and various databases.

It would be trivial to plug in an in-memory storage backend to this client.

Try giving the library a cStringIO "file handle" instead of a real file handle. That works for most python libraries.