jQuery - Remove class from any <li> that wasn't just clicked

Lets say I have list like this...

<ul>
  <li class="one">One</li>
  <li class="two">Two</li>
  <li class="three">Three</li>
</ul>

...and I want to be able to toggle class in the li's...

$( "li" ).click(function() {
    $( "li" ).toggleClass( "active" );
});

..also I want the li's except the one that was clicked to never show as '.active' while still maintaining the toggleClass ability in the '.active' li

So I was thinking maybe I want to removeClass from any other li's except the one what was clicked. I don't know how I could do that though...

Any ideas on how to achieve something like that? I'm sure its rather easy but I have no idea as I'm very new to jQuery

Solved - If someone wants to see it.. there we go: http:开发者_开发知识库//jsfiddle.net/6jm2s/17/ also this one... http://jsfiddle.net/6jm2s/18/

Both answers work perfectly.. @jondavidjohn & @hunter Thank you guys... too bad I can't put Checkmark on both answers... but I will let hunter have it as his code is shorter.. :)

Make it so that once something is active, one will always be active:

$("li:not(.active)").live("click", function() {
    $(this).addClass("active");
    $(this).siblings().removeClass("active");
});

focuses only on li's that aren't already active.

To be able to deactivate:

$("li").click(function() {
    $(this).toggleClass("active");
    $(this).siblings().removeClass("active");
});

working example: http://jsfiddle.net/6jm2s/18/

$( "li" ).click( function() {
    if( $(this).is('.active') ) {
        $(this).removeClass( "active" );
    }
    else {
        $( "li.active" ).removeClass( "active" );
        $(this).addClass( "active" );
    }
});