How can I allow users to search data which is stored without reference to its URL?

I was wondering how to setup a search engine with this constraint :

On the website (small traffic website), news content is stored but we have no idea where these pieces of content are actually displayed开发者_Python百科; we can't guess the URL.

Do you have any idea how and which technology I will have to use to achieve this functionality?

I guess I need a search engine which crawls the entire website while searching? Can anybody tell me if sphynx/lucene/solr would be suitable?

No matter what search engine you choose you must supply it with enough information. Hence, with your constraint you cannot use any search engine in the world.

When you submit a document or page to the index you must also supply enough information to be able to create an URL to where the actual content will be displayed.

Lucene is the search API used by Solr. Both are Apache projects of high quality and which to choose depends on your requirements. Solr gives a wide set of features to Lucene such as facetted seach and a REST API for all it's operations.