Joomla noindex,follow PHP code

I have a joomla 开发者_JAVA技巧based news website that has a ton of useless pages showing up in search engine indices. At least as a quick fix until I can look at rebuilding the site from scratch I want to implement a NOINDEX, FOLLOW meta tag on all pages except the home page and article pages that end in .html

Working off various snippets of code found here and elsewhere I have come up with this:

<?php
if ((JRequest::getVar('view') == "frontpage" ) || ($_SERVER['REQUEST_URI']=='*.html' ))    {
echo "<meta name=\"robots\" content=\"index,follow\"/>\n";
} else {
echo "<meta name=\"robots\" content=\"noindex,follow\"/>\n";
}
?> 

I'm still very new to php programming and I'm sure I'm bound to have made a couple of mistakes so I was wondering if a kind soul would be able to give my code the once over and let me know if it's ok to use before I accidentally nuke my site.

Thanks,

Tom

Wouldn't it be better to use the robots.txt file for this?

Some major crawlers support an Allow directive which can counteract a following Disallow directive. This is useful when one disallows an entire directory but still wants some HTML documents in that directory crawled and indexed. While by standard implementation the first matching robots.txt pattern always wins, Google's implementation differs in that Allow patterns with equal or more characters in the directive path win over a matching Disallow pattern. Bing uses the Allow or Disallow directive which is the most specific.

In order to be compatible to all robots, if one wants to allow single files inside an otherwise disallowed directory, it is necessary to place the Allow directive(s) first, followed by the Disallow, for example:

Allow: /folder1/myfile.html
Disallow: /folder1/

This example will Disallow anything in /folder1/ except /folder1/myfile.html, since the latter will match first. In case of Google, though, the order is not important.

This will never match:

$_SERVER['REQUEST_URI']=='*.html'

== is a literal comparison and does not parse wildcards. You may check the end of a string with substr:

substr($_SERVER['REQUEST_URI'], -5) == '.html'

or you can use a regular expression:

//This will match when .html is enywhere inside the string
preg_match('/\.html/', $_SERVER['REQUEST_URI'])

//This will match when .html is at the end of the string, but the
//substr solution is faster in that case
preg_match('/\.html$/', $_SERVER['REQUEST_URI'])

taking advice from the posters here and a friend I have come up with this:

you need to go to /public_html/libraries/joomla/document/html and edit html.php

replace

//set default document metadata
     $this->setMetaData('Content-Type', $this->_mime . '; charset=' . $this->_charset , true );
     $this->setMetaData('robots', 'index, follow' );

with

//set default document metadata
$this->setMetaData('Content-Type', $this->_mime . '; charset=' . $this->_charset , true );

$queryString = $_SERVER['REQUEST_URI'];
if (( $queryString == '' ) || ( $queryString == 'index.php/National-news' ) || ( $queryString == 'index.php/Business' ) || ( $queryString == 'index.php/Sport' ) || ( substr($queryString, -5 ) == '.html' )) {
$this->setMetaData('robots', 'index, follow' );
}else {
$this->setMetaData('robots', 'noindex, follow' );
}

this will update the meta robots tag on every page on the site, removing all the messed up content from search engines and leaving only the content we want to be found in the index.

I'll try running it on a test server in the next few days and report back.