Python for-loop look-ahead

I have开发者_StackOverflow a python for loop, in which I need to look ahead one item to see if an action needs to be performed before processing.

for line in file:
    if the start of the next line == "0":
        perform pre-processing
        ...
    continue with normal processing
    ...

Is there any easy way to do this in python? My current approach is to buffer the file to an array, however this is not ideal as the file is rather large.

you can get any iterable to prefetch next item with this recipe:

from itertools import tee, islice, izip_longest
def get_next(some_iterable, window=1):
    items, nexts = tee(some_iterable, 2)
    nexts = islice(nexts, window, None)
    return izip_longest(items, nexts)

Example usage:

for line, next_line in get_next(myfile):
    if next_line and next_line.startswith("0"):
        ... do stuff

The code allows you to pass the window parameter as a larger value, if you want to look 2 or more lines ahead.

Along the lines of nosklo's answer, I tend to use the following pattern:

The function pairwise from the excellent itertools recipes is ideal for this:

from itertools import tee

def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = tee(iterable)
    next(b, None)
    return izip(a, b)

Using it in your code gets us:

for line, next_line in pairwise(file):
    if next_line.startswith("0"):
        pass #perform pre-processing
        #...
    pass #continue with normal processing

Generally, for this type of processing (lookahead in the iterable), I tend to use a window function. Pairwise is a special case of a window of size 2.

You can have a prev_line where you store previous line and process that whenever you read a line only given your condition.

Something like:

prev_line = None
for line in file:
    if prev_line is not None and the start of the next line == "0":
        perform pre-processing on prev_line
        ...
    continue with normal processing
    ...
    prev_line = line

You may need to do additional processing for the last line if necessary, depending on your logic.

This should work too. I always prefer calling next over setting something = None for the first round.

prev_line = next(the_file)
for current_line in the_file:
    if current_line.startswith('0'):
        do_stuff( prev_line )
    # continue with normal processing
    # ...
    prev_line = current_line

You simply need to buffer one line.

for line in file:
  if (prevLine is not None):
    //test line as look ahead and then act on prevLine
  prevLine = line

It's much easier than all that...

lines = len(file)
for i in range(0,lines):
    current_line = file[i]
    if i < lines - 1:
        next_line = file[i + 1]
    else:
        next_line = None

    do_your_work(current_line,next_line)

more_itertools has several lookahead tools. Here we will demonstrate some tools and a abstracted function for processing lines of a file. Given:

f = """\
A
B
C
0
D\
"""
lines = f.splitlines()

Code

import more_itertools as mit


def iter_lookahead(iterable, pred):
    # Option 1
    p = mit.peekable(iterable)
    try:
        while True:
            line = next(p)
            next_line = p.peek()
            if pred(next_line):
                # Do something
                pass
            else:
                print(line)
    except StopIteration:
        return


pred = lambda x: x.startswith("0")
iter_lookahead(lines, pred)

Output

A
B
0

Here are other options using the same library that include pairwise and windowed tools mentioned by @Muhammad Alkarouri.

# Option 2
for line, next_line in mit.pairwise(lines):
    if pred(next_line):            
        # Do something
        pass
    else:
        print(line)

# Option 3
for line, next_line in mit.windowed(lines, 2):
    if pred(next_line):            
        # Do something
        pass
    else:
        print(line)

The latter options can be run independently or substitute the logic in the prior function.