How can I use the output of my monitor as the input to a program?

Some context: I am hoping to make a program that can play a videogame based on visual input. While I could just point a webcam at my monitor whilst the game is on the screen, I'd prefer to just have some way to send whatever pixel information is going to the screen to my program.

Specifically, I'm hoping to sample screenshots at a rate ~30fps and compute on them. So far the only thing I can think of is to use Fraps to take screenshots whenever 开发者_JS百科my program 'presses' a certain key, but those can only be taken at a maximum rate of one per second and require using this outside program. I'm hoping there is some way to intercept this screen information more directly.

I'm currently planning on using a combination of Java and Matlab, but I'd be happy to switch to whichever language has a nice way to grab screenshots rapidly. Oh, and I'm doing this on Windows 7, in case this screen grabbing operation is low-level enough for that to matter.

I'll take a stab at answering your question. The basic answer is that the screen data itself is memory mapped, so it resides somewhere in physical memory. However, your operating system has probably cut you off from accessing that memory directly through its virtual memory system. So, the only way to get access is to either:

a) Write a device driver to access it or b) Use a device driver written by someone else to access it

On Windows you could use DirectX to access the screen data. This website explains how to do that:

http://www.dgrigoriadis.net/post/2008/01/29/Taking-screenshots-with-DirectX-90.aspx

If it's a 2D Java game, you'd just send it to the framebuffer as well as your visual input program. If it's 3D and in Java, you could possibly do a similar thing with screenshotting functions in the library the game is written in. If it's a game in another window, you could try using java.awt.Robot (http://download.oracle.com/javase/1,5.0/docs/api/java/awt/Robot.html) and see if the screenshot capability works. You get a BufferedImage, which you would send to the visual input program (like if you were sending the game view from a Java 2D or 3D app).

You can do it , But with complex image processing, gesture recognition and machine learning algorithms. And the response of the system should be real-time if you want to win that game which naturally makes your research complex.

But if the original game is developed by yourself , then you probably don't need any of image processing, ,webcam and FRAPS. In that case all you need is Machine Learning for game playing.