.htaccess and google crawl bug
I'm having trouble with this website since I've added a .htaccess file. Google bot doesn't crawl the new links.
开发者_运维技巧here is my htaccess:
Options +FollowSymlinks
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www.grantsimeonphoto.com$
RewriteRule ^(.*)$ http://www.grantsimeonphoto.com/$1 [R=301]
RewriteRule ^accueil$ index.php?page=accueil
RewriteRule ^services$ index.php?page=services
RewriteRule ^portfolio$ index.php?page=portfolio
RewriteRule ^contact$ index.php?page=contact
RewriteRule ^epreuves$ index.php?page=epreuves
if you make a search on google you will see :
site:grantsimeonphoto.com
here is my robot.txt:
User-agent: *
Allow: /
Here is my sitemap.xml:
<?xml version="1.0" encoding="UTF-8"?>
<urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9
http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<!-- created with Free Online Sitemap Generator www.xml-sitemaps.com -->
<url>
<loc>http://www.grantsimeonphoto.com/</loc>
<priority>1.00</priority>
</url>
<url>
<loc>http://www.grantsimeonphoto.com/accueil</loc>
<priority>0.80</priority>
</url>
<url>
<loc>http://www.grantsimeonphoto.com/services</loc>
<priority>0.80</priority>
</url>
<url>
<loc>http://www.grantsimeonphoto.com/portfolio</loc>
<priority>0.80</priority>
</url>
<url>
<loc>http://www.grantsimeonphoto.com/contact</loc>
<priority>0.80</priority>
</url>
<url>
<loc>http://www.grantsimeonphoto.com/epreuves</loc>
<priority>0.80</priority>
</url>
</urlset>
Thanks for the help
You don't need to add the allow, commonly all search engines will crawl your links when it doesn't explicitly disallow the specified URL.
精彩评论