Capybara+Rspec+Rails3.1 Assets not being requested after they were requested a first time

I'm using capybara+rspec to test my requests on a rails 3.1 application, I have two scenarios doing the exact same thing, no matter what the one thats executed first passes the second never passes unless I have config.cache_classes set to false, I realized that when it is set to true on the second scenario the assets are not being requested after they are requested the first time.

What happens is that the first request of the assets happens normally when they are first needed, when theres a second request to a page that requires the assets already loaded there is no request to them, when rails is on development environment they are requested and if there are already cached rails returns a 304. This is not happening on test environment unless cached_classes is set to false, which breaks all my other tests and makes the test suite really slow.