Remove Holidays and Weekends in a very long time-serie, how to model time-series in Python?

Is there some function in Python to handle this. GoogleDocs has a Weekday -operation so perhaps there is something like that in Python. I am pretty sure someone must have solved this, similar problems occur in sparse data such as in finance and research. I am basically just trying to organize a huge amount of different sized vectors indexed by days, time-series, I am not sure how I should hadle the days -- mark the first day with 1 and the last day with N or with unix -time or how should that be done? I am not sure whether the time-series should be saved into matrix so I could model them more easily to calculate correlation matrices and such things, any ready thing to do such things?

Let's try to solve this problem without the "practical" extra clutter:

import itertools
seq = range(100000)
criteria  = cycle([True]*10 + [False]*801)
list(compress(seq, criteria))

now have to change them into days and then change the $\mathbb R$ into $( \mathbb R, \mathbb R)$, tuple. So $V : \mathbb R \mapsto \mathbb R^{2}$ missing, investigating.

[Update]

Let's play! Below code solves the subproblem -- creates some test data to test things -- now we need to create arbitrary days and valuations there to try to test it on arbitrary timeseries. If we can create some 开发者_运维知识库function $V$, we are very close to solve this problem...it must consider though the holidays and weekends so maybe not easy (not sure).

import itertools as i
import time
import math
import numpy



def createRandomData():
    samples=[]

    for x in range(5):
        seq = range(5)
        criteria  = i.cycle([True]*x+ [False]*3)

        samples += [list(i.compress( seq, criteria ))] 

    return samples

def createNNtriangularMatrix(data):
    N = len(data)
    return [aa+[0]*(N-len(aa)) for aa in data]


A= createNNtriangularMatrix(createRandomData())
print numpy.array(A)
print numpy.corrcoef(A)

I think you should figure out someway the days you want to INCLUDE, and create a (probably looping) subroutine use slicing operations on your big list.

For discontinuous slices, you can take a look at this question:

Discontinuous slice in python list

Or perhaps you could make the days you do not want receive a null value (zero or None).

Try using pandas. You can create a DateOffset for business days and include your data in a DataFrame (see: http://pandas.pydata.org/pandas-docs/stable/timeseries.html) to analyze it.

I think it depends on the scope of your problem, for a personal calendar, 'day' is good enough for indexing.

One's life is as long as 200 years, about 73000 days, simply calculate and record them all, maybe use a dict, e.g.

day = {}
# day[0] = [event_a, event_b, ...]
# or you may want to rewrite the __getitem__ method like this: day['09-05-2012']

Why would you want to remove the holidays and weekends? Is it because they are outliers or zeroes? If they are zeroes they will be handled by the model. You would want to leave the data in the time series and use dummy variables to model the seasonal effects (ie monthly dummies), day of the week dummies and holiday dummies. Clearly, I am dummfounded. I have season people who are unable to deal with time series analysis even break the weekdays into one time series and the weekends into another which completely ignores the lead and lag impacts around holidays.

If it is trading days you want then you can use the pandas datareader package to download the s&p 500 historical prices for U.S. and use the index of dates as a mask to your data.

Answered on mobile, I'll add links and code later.