how to make Sliding window model for data stream mining?
we have a situation that a stream (data from sensor or click stream data at server) is coming with slidi开发者_如何学运维ng window algorithm we have to store the last (say) 500 samples of data in memory. These samples are then used to create histograms, aggregations & capture information about anomalies in the input data stream.
please tell me how to make such sliding window.
If you are asking how to store and maintain these values in a sliding-window manner, consider this simple example which keep tracks of the running mean of the last 10 values of some random stream of data:
WINDOW_SIZE = 10;
x = nan(WINDOW_SIZE,1);
%# init
counter = 0;
stats = [NaN NaN]; %# previous/current value
%# prepare figure
SHOW_LIM = 200;
hAx = axes('XLim',[1 SHOW_LIM], 'YLim',[200 800]);
hLine = line('XData',1, 'YData',nan, 'EraseMode','none', ...
'Parent',hAx, 'Color','b', 'LineWidth',2);
%# infinite loop!
while true
val = randi([1 1000]); %# get new value from data stream
x = [ x(2:end) ; val ]; %# add to window in a cyclic manner
counter = counter + 1;
%# do something interesting with x
stats(1) = stats(2); %# keep track of the previous mean
stats(2) = nanmean(x); %# update the current mean
%# show and update plot
set(hLine, 'XData',[counter-1 counter], 'YData',[stats(1) stats(2)])
if rem(counter,SHOW_LIM)==0
%# show only the last couple of means
set(hAx, 'XLim', [counter counter+SHOW_LIM]);
end
drawnow
pause(0.02)
if ~ishandle(hAx), break, end %# break in case you close the figure
end
Update
The EraseMode=none
property was deprecated and removed in recent versions. Use the animatedline
function instead for a similar functionality.
精彩评论