All but the last N elements of iterator in Python

What is the best way to get all 开发者_运维问答but the last N elements of an iterator in Python? Here is an example of it in theoretical action:

>>> list(all_but_the_last_n(range(10), 0))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(all_but_the_last_n(range(10), 2))
[0, 1, 2, 3, 4, 5, 6, 7]

Just for the fun of it, here's a variation on Ignacio's solution that doesn't require a deque.

>>> def truncate(it, n):
...     cache = [next(it) for i in range(n)]
...     index = 0
...     for val in it:
...         val, cache[index] = cache[index], val
...         index = (index + 1) % n
...         yield val

I wasn't especially concerned with speed when I wrote the above... but perhaps this would be a tad faster:

def truncate(it, n):
    cache = [next(it) for i in range(n)]
    index = 0
    for val in it:
        yield cache[index]
        cache[index] = val
        index = (index + 1) % n

Use a collections.deque. Push N items from the source on the first invocation. On each subsequent invocation, pop an item out, push an item in from the source, and yield the popped item.

Based on Ignacio Vazquez-Abrams's description:

from collections import deque

def all_but_the_last_n(iterable, count):
    q = deque()
    i = iter(iterable)
    for n in range(count):
        q.append(i.next())
    for item in i:
        q.append(item)
        yield q.popleft()

I wondered whether it was better to use the deque right to left (append, popleft) or left to right (appendleft, pop). So i timed it with python 2.5.2 and found that rtl was 3.59 usec while ltr was 3.53 usec. The difference of 0.06 usec is not significant. the test was to append a single item and pop a single item.

Using Ignacio's solution.

import collections
def all_but_the_last_n(iterable, n):
    it = iter(iterable)
    fifo = collections.deque()
    for _, i in zip(range(n), it):
        fifo.append(i)
    for i in it:
        fifo.append(i)
        yield fifo.popleft()

print(list(all_but_the_last_n(range(10), 3)))
print(list(all_but_the_last_n('abcdefghijkl', 3)))

It is unfortunate that collections does not have a circular buffer. This would be more efficient from a cache miss standpoint with one.

For a list you could do this:

def all_but_the_last_n(aList, N):
    return aList[:len(aList) - N]

myList = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
N = 4

print(all_but_the_last_n(myList, N))

Will print:

[0, 1, 2, 3, 4, 5]

This is compact (aside from consume recipe taken from itertools recipes) and should run at C speed:

import collections
import operator
from itertools import islice, tee


def truncate(iterable, n):
    a, b = tee(iterable)
    consume(b, n)
    return map(operator.itemgetter(0), zip(a, b))


# From itertools recipes
def consume(iterator, n=None):
    "Advance the iterator n-steps ahead. If n is None, consume entirely."
    # Use functions that consume iterators at C speed.
    if n is None:
        # feed the entire iterator into a zero-length deque
        collections.deque(iterator, maxlen=0)
    else:
        # advance to the empty slice starting at position n
        next(islice(iterator, n, n), None)

I think this should work:

list[:-N]