Problem implementing CSS Sprites

This is my image:- http://www.flickr.com/photos/50525207@N02/5064283850/

CSS n html

Now the problem:-

When I hover over links the same image appears when I want different parts of the image to appear. Also the other links shift when I move mouse over one link. What I want is this:-

http://www.flickr.com/photos/50525207@N02/5063686801/

What I want is a grey colored image to appear in the background when the mouse is hovered over "Link1". A green colored image is to appear when the mouse is hovered over "Link2" and so on. What am I doing wrong? I have been trying to make it work since yesterday but in vain.

PS: erm, that's not the actual image BTW. I don't want colors in the background. It's going to be images of products. Oh, and I want that grey image to appear when no link is hovered over. How to do that?

[EDIT]

I added the following in the CSS:-

.sprite Div
{
    width: 728px;
    height: 243px;
}

.sprite a
{
    width: 728px;
    height: 243px;
}

In the HTML IK included the links inside of Div so the height gets fixed:-

<div id="SpriteDiv" class="sprite">
    My links here...
</div>

First, you should set a size of your anchor element without hover, this is what's causing your other links to shift around (the dimensions shouldn't be defined on a:hover):

.sprite a {
  display: block;
  width: 728px; 
  height: 243px; 
}

Next, your image background is assigned to the anchor elements, not the span, so you need to define those positions with the selector like this:

.sp_ID1 a {
  background-position: 0px 0px; 
} 

Corrected according to your comment:

Originally I put the gray background on .container, but that causes strange behavior on Chrome, so I added .sp_ID0

<style type="text/css">
    .sprite { display: block; margin-bottom: 5px; }
    .container, .sp_ID0, .sprite div { width: 600px; height: 203px; }
    .sp_ID0, .sprite:hover div {
        background-image: url(http://farm5.static.flickr.com/4106/5064283850_fc6b5fac15_b.jpg);
        background-repeat: no-repeat;
    }
    .container  { position:relative; }
    .sp_ID0 { z-index: -2; }
    .sprite div { display: none; z-index: -1; }
    .sp_ID0, .sprite:hover div { position: absolute; top: 0px; left: 0px; display: block; }
    .sp_ID1 div { background-position: 0px -203px; }
    .sp_ID2 div { background-position: 0px -406px; }
    .sp_ID3 div { background-position: 0px -609px; }
    .sp_ID4 div { background-position: 0px -812px; }
    .sp_ID5 div { background-position: 0px -203px; }
    .sp_ID6 div { background-position: 0px -406px; }
</style>

<div class="container">
<div class="sp_ID0">&nbsp;</div>
    <a href=# class="sprite sp_ID1">Link1<div>&nbsp;</div></a>
    <a href=# class="sprite sp_ID2">Link2<div>&nbsp;</div></a>
    <a href=# class="sprite sp_ID3">Link3<div>&nbsp;</div></a>
    <a href=# class="sprite sp_ID4">Link4<div>&nbsp;</div></a>
    <a href=# class="sprite sp_ID5">Link5<div>&nbsp;</div></a>
    <a href=# class="sprite sp_ID6">Link6<div>&nbsp;</div></a>
</div>

Old solution.