Update August 13th, 2011: I’ve switched over and am giving a new plugin a try. It’s called Better WordPress Google XML Sitemaps (with sitemapindex and Multi-site support) and so far so good.
What really convinced me to look at this one is that the guys at WP Engine added it to their Curated Plugins list. If they are giving this a try then that’s really all justification I need.
Update July 23rd, 2011: These instructions still work for the newer 3.2.5 version as well. While this simple modification is useful, I plan to switch the to the 4.0 beta.
Update: Arne has posted an early test version on the WordPress support forums here. At this stage it’s not yet ready for release. If you would like to test that version, you can support Arne’s effort by downloading and reporting any issues to him on that thread.
I’m lazy and have not yet tried the early release version. 🙂
——————————————————–
WordPress 3.0’s Multisite feature is cool and useful. The Google (XML) Sitemaps Generator plugin is also cool and useful. Unfortunately Arne Brachhold’s excellent plugin does not work with multisite blogs.
If you activate the plugin on a multisite enabled WordPress install, you get this error message.
Which is a shame, but it did not take me a long time to find a work around. My setup is using multisite with sub-domains and with the Domain Mapping plugin. These instructions are for the 3.2.4 version of the plugin.
1. Modify the plugin
Edit the file wp-content/plugins/google-sitemap-generator/sitemap.php and comment out lines 53 to 58 to look like this
//Check for 3.0 multisite, NOT supported yet!
// if((defined(‘WP_ALLOW_MULTISITE’) && WP_ALLOW_MULTISITE) || (function_exists(‘is_multisite’) && is_multisite())) {
// if(function_exists(‘is_super_admin’) && is_super_admin()) {
// add_action(‘admin_notices’, array(‘GoogleSitemapGeneratorLoader’, ‘AddMultisiteWarning’));
// }
//
// return;
// }
You can delete these lines, but I prefer to comment them out. All this does is prevent the plugin from detecting the multisite setup and not becoming active.
2. Get your rewrite rules ready
In your blog’s .htaccess file add these lines above your WordPress rules
RewriteRule ^sitemap.xml sitemap-%{SERVER_NAME}.xml [L] RewriteRule ^sitemap.xml.gz sitemap-%{SERVER_NAME}.xml.gz [L]
This will come in handy later.
3. Activate the plugin and set these options
Now on each of your sites in your WordPress network, activate the Google (XML) Sitemap plugin. Do not Network Activate it.
Now got to your XML-Sitemap options page and for “Location of your sitemap file” select Custom location.
In the custom location change your change the file name sitemap.xml to sitemap-yourblogservername.xml. My server name is blog.dembowski.net so I’m using sitemap-blog.dembowski.net.xml. Save the options.
Look in your WordPress installation’s root and check if you have sitemap.xml or sitemap.xml.gz. If you do then delete them both.
Now at the top of that options page, click on “rebuild the sitemap manually” and confirm that your new sitemap-yourblogservername.xml has been created.
4. Test and confirm that this worked
Now test two things. Start by pointing your browser to http://yourblogservername/sitemap.xml and then http://yourblogservername/sitemap-yourblogservername.xml. If you setup your .htaccess rewrite rules correctly, then both URLs should work.
Load up your WordPress generated robots.txt file via http://yourblogservername/robots.txt and look at the Sitemap line. Mine reads like this
User-agent: * Disallow: Sitemap: http://blog.dembowski.net/sitemap-blog.dembowski.net.xml.gz
I have that rewrite rule in case the default location for the sitemap is used. When you load the sitemap via the normal location, the rewite rule sends it to the correct location. The second gziped sitemap URL is fine.
There should be no real sitemap.xml or sitemap.xml.gz files in your blog’s root directory. Instead, each sitemap file should have your virtual WordPress network names instead.
5. Rinse and repeat
Repeat this process for each site in your WordPress Network and you can continue to use this sitemap plugin. You will need to activate the plugin in each of your sites and change sitemap.xml to include the site name just as I outline in step 3.
6. The caveat
If this works for you, great! But if it doesn’t, please don’t bug Arne. If and when he makes his plugin multisite compatible, then it will be probably a better solution than this one. I’m not seeing any PHP errors or warning in my log files, but there may be reasons why this is not a good idea.
Also, when you modify your copy of a plugin in the WordPress plugin repository and that plugin is updated you will lose your edits from step 1 when you update the plugin.
In the meanwhile, I can continue to use this excellent plugin.
Abhinav says:
Hello and thanks for this wonderful tutorial. Can you please tell me how to use this plugin with multisite @ directory. I’m not using it @ sub-domain. hence, the above re-write rules are not working.
June 13, 2010 — 7:37 pm
Jan Dembowski says:
Abhinav,
You are entirely welcome.
I have not tried a directory install, and that sounds like a tough one.
Each blog in it’s own directory would use the same sitemap. What you would need is a combined sitemap for each of your sites i.e. http://yoursite.com/blog1/ http://yoursite.com/blog2/ etc.
I don’t think there is a graceful work around for a directory based site with this plugin. It would need Arne to recode it and make it multisite aware.
Before I fixed this problem with my installation, I was successfully using this plugin:
http://wordpress.org/extend/plugins/xml-sitemap-feed/
I preferred Arne’s plugin but that one worked as well. Give it a try, from the documentation it looks like it may work.
Good luck.
June 13, 2010 — 9:55 pm
lonelicloud says:
HERE is the rewrite_rule for directory installation.
http://cnzhx.net/blog/articles/google-xml-sitemaps-and-wordpress-multisite/
You could skip the Chinese chars and look for the rules, or like this:
where $1 is the directories of sites on your network.
like:
July 6, 2010 — 8:10 pm
Jan Dembowski says:
That would work, but whenever I got to that link I get a 404 page.
Edit: Okay, I get it. That site is in China and restricts visitors. Try this link instead for people not in China.
July 6, 2010 — 8:18 pm
lonelicloud says:
It’s odd! But I can click the link and go to the right page. AND, this site is mainly in Chinese but it’s hosted in U.S. with a free host plan. Maybe FREE is the reason.
Anyway, that’s not important because the rules is here and anyone can see it.
July 8, 2010 — 12:29 pm
lonelicloud says:
I found the problem.
Occasionally, this link could not be opened right because I use WPSC to cache the page and compression functionally is enabled. Sometimes, the .gz file is not handled properly. Sorry for that!
But you could just use the code I provide above.
October 2, 2010 — 10:16 am
Alan says:
This worked perfectly. Thank you, you have saved me so much time and I was getting a headache trying to find a plugin that could do this.
I hope Arne can get his plugin updated soon, but for now this is just what I needed.
Thanks Again
June 16, 2010 — 9:01 am
Jan Dembowski says:
You’re welcome.
Thinking about Abhinav’s problem re directory multisite, blanket multisite support maybe problematic.
Subdomain multisite should be a quick check and adjustment, but there may be more to it than I am aware of.
June 16, 2010 — 9:40 am
JAG says:
Jan,
Thank you for posting this!
Had to play around with the subdomain url because I’m on cloud servers but aside from that your instructions are excellent and the plugin works flawlessly.
Cheers!
JAG
June 19, 2010 — 9:28 pm
Jan Dembowski says:
Jag,
You’re welcome.
June 19, 2010 — 10:23 pm
Max says:
Jan, thanks a lot! It’s realy works!
I made it on my web(multi)site, and it works correctly. Any php errors or other bugs.
Thanks again!
June 23, 2010 — 2:08 pm
Jan Dembowski says:
Max,
You’re welcome and I’m glad it works. Multisite is so much easier than maintaining 4 or 5 separate WordPress installations. Having this plugin work is a bonus.
June 23, 2010 — 2:32 pm
Max says:
Jan,
I have once more question. I can’t find, where is robots.txt formed?
As I understand, there is no any live file whith such name. But it’s going generating by Google XML Sitemap plugin due GET query?
So, question is “how can I put my own exclusions to robots.txt output?”
Thank you, again!
June 23, 2010 — 4:45 pm
Jan Dembowski says:
That’s a fun one. You want to add specific data to the artificial robots.txt file for your site. I haven’t tried but give this plugin a look. It might let you do what you want.
http://wordpress.org/extend/plugins/robots-meta/
June 23, 2010 — 10:15 pm
Max says:
Yeah, I got this one http://wordpress.org/extend/plugins/pc-robotstxt/ for my site.
Seems like it works correctly. 🙂
June 25, 2010 — 9:30 am
Oliver says:
There is a more elegant way for networks. You can activate the plugin for network and edit the function GetXmlPath in sitemap-core.php
Create a directory in your root path called “sitemaps” and chmod it to 777. Add this to your .htaccess:
Thats it! The sitemaps will be generated in the sitemap directory.
June 27, 2010 — 8:55 pm
Jan Dembowski says:
Oliver,
That is a better way to do it and I considered making those changes.
However, my PHP is less than reliable so I kept it as simple for me as I could. Hopefully Arne’s solution will be more elegant than my hack. 🙂
June 28, 2010 — 5:49 am
Ivan says:
Error loading stylesheet: An unknown error has occurred (805303f4)http://www.mysite.net/wp-content/plugins/google-sitemap-generator/sitemap.xsl
i did all like you described and still getting errors while creating sitemap for my subdomains?!?!
July 4, 2010 — 8:33 am
Jan Dembowski says:
Ivan,
It should work. Try loading the sitemap.xsl file directly and see if there is some problem with retrieving that file. You may see an error in your web server’s error_log file.
July 4, 2010 — 10:09 am
Ivan says:
i can load sitemap.xsl just fine, i can even access to rootdomain sitemap the only problem is when i try to access to subdomain’s sitemap
July 4, 2010 — 10:46 am
anjan says:
Hi Oliver,
Have you tried those changes yourself.When I activate plugin for network the error message pops up as said in this article.
“Google XML Sitemaps is not multisite compatible.
Unfortunately the Google XML Sitemaps plugin was not tested with the multisite feature of WordPress 3.0 yet. The plugin will not be active until you disable the multisite mode. Otherwise go to active plugins and deactivate the Google XML Sitemaps plugin to make this message disappear.”
So it seems plugin can’t me made active using network activate mode.
August 28, 2010 — 2:14 am
Jan Dembowski says:
Anjan,
The changes Oliver discussed were for after the multi-site check was disabled.
You still need to comment out the checks from lines 53 to 58 at the top of this blog post.
August 28, 2010 — 7:49 am
NTB says:
I think that the .htaccess rewrite rule should be more strict
With your example the url for sitemap.xml.gz is redirected to to sitemap.xml
I suggest using:
July 3, 2010 — 1:56 pm
Jan Dembowski says:
NTB,
The inclusion of the EOL $ character is not a bad idea, and does make it a more specific match in the rewrite rules. I left it as is because it works for me.
Also I prefer to have the sitemaps in the blog root directory. Just my preference, I’m sure you’ll set it as you see fit.
July 3, 2010 — 4:22 pm
zamzam says:
hello,
my wpmu main domain is nol.im, is it correct setting for subdomain ?
Absolute or relative path to the sitemap file, including name.
/home/nol/public_html/jewelry.nol.im/sitemap-jewelry.nol.im.xml
Complete URL to the sitemap file, including name.
http://jewelry.nol.im/sitemap-jewelry.nol.im.xml
July 4, 2010 — 12:12 pm
Rene Silva says:
Thanks for the advice!
July 7, 2010 — 6:10 pm
guialowcost says:
I did all but still have this message: Google XML Sitemaps is not multisite compatible.
July 16, 2010 — 4:18 am
Jan Dembowski says:
You must have missed a step. Commenting out those line will let Google XML Sitemaps work in multi-site mode.
Try downloading a fresh copy of the plugin and make the modifications again.
July 16, 2010 — 5:23 am
Alexandre says:
Hello, thks for the great hack.
After the process, I had both sitemaps, the old and the new one.
After delete the oldest, I check my robots.txt, as you sugested, but I only have “User-agent: *” and “Disallow:” terms. Should it have too a “new sitemap path”, havent it?
I made a .htacess hack putting the RewriteRule above de # BEGIN WordPress line. Sorry for may poor php skill 🙂
Thks again.
Alexandre
July 16, 2010 — 4:23 pm
Jan Dembowski says:
Alexandre,
Do you have a real robots.txt file or are you using the WordPress generated one? If you have a real robots.txt file, rename it to robots-old.txt and the try to load http://your-url/robots.txt
See if that does it.
July 16, 2010 — 10:47 pm
Michael says:
Jan –
What do you do if you’ve got the multisite blog installed in the root directory? There is no directory for the main blog so what would you use for the $1 variable?
July 19, 2010 — 3:50 am
Michael says:
Here’s a follow up question – what if you’re using another sitemap generator for the rest of your site which also leaves a sitemap.xml file in the root directory? It needs to be placed in another subdirectory and that would probably be a prudent measure unless you only run wordpress on your domain.
July 19, 2010 — 3:52 am
Jan Dembowski says:
Michael,
I think you’ve lost me there.
This is a simple solution for using Arne’s plugin for with a multisite install using sub-domains. A sub-domain install is supposed to be in the root directory.
Even if you don’t have an active blog at your-root.com you will have blog1.your-root.com, blog2.your-root.com, etc.
The rewrite rule will make sure that is the default blog1.your-root.com/sitemap.xml, blog2.your-root.com/sitemap.xml, etc. will serve the correct sitemap for the correct blog.
You could put the sitemap.xml somewhere else and delete the sitemap rewite rules.
July 19, 2010 — 6:38 am
OJ says:
Hi,
What do you do with the old sitemap you have submitted under Google’s webmaster’s tool? Do you delete it or what?
Now I have two, the old one and the newly created after making your modifications. Is this going to affect my index or perhaps, my ranking?
Thanks 🙂
July 20, 2010 — 9:31 pm
Jan Dembowski says:
I just checked my Google Webmaster tools for this site and the other in my WordPress network. I’m still getting crawled and my indexing doesn’t seem to have changed. Nice to see that the rewrite rules are working. 🙂
I don’t think you’ll suffer any penalty with Google. Since I’ve done this I still get the same amount of traffic.
July 21, 2010 — 6:16 am
Markus Richert says:
Thanks for this great tutorial.
It’s even more elegant, to create a subdirectory for each set of sitemaps.
So the RewriteRules would look like:
As mentioned before in exact this order, gz first, then the one without. Otherwise the first rule catches them all.
The third step is spare to edit for each blog, if you replace the following into sitemap-core.php in function GetXmlPath, line 1386:
with this:
This will automatically create a subdir for each blog containing the sitemaps therefor.
Step 4 is also spare, if you don’t use a physical robots.txt, but the WordPress virtual one instead. You only need to check “Add the sitemap to the virtual robots.txt” in the XML Sitemap options and are done with your hand work.
August 24, 2010 — 1:02 pm
Jan Dembowski says:
Markus,
I’m glad you liked the post. I’ve added some formatting to your comment so that readers can see more clearly what you are refering to.
Your edits would have been a more elegant solution: fortunately Arne is working on a version of his plugin that visualizes the sitemap.xml files.
No more real files, it’s a more dynamic solution. Of course, this may mean more CPU time but I’m confident that the next release will be worth the wait.
August 24, 2010 — 2:32 pm
Robert says:
Thanks for the tutorial. I also use a subdomain network with domain mapping, but the generated xml file has the original domain in the tags instead of the mapped domain. For example:
Network: network.com
Site: site.network.com
http://www.betterdomain.com is mapped to site.network.com
In the generated sitemap I’m getting this:
I need it to be this instead:
Any ideas? Thanks.
October 6, 2010 — 11:36 am
Robert says:
On the last comment… it’s missing all the tags, but hopefully you get the idea.
October 6, 2010 — 11:37 am
Jan Dembowski says:
I think either the .htaccess rules are not working (and you are getting a different file than you intended) or the domain mapping is not setup correctly.
When the sitemap.xml is generated, does it actually create a file called sitemap-www.betterdomain.com.xml?
October 6, 2010 — 5:25 pm
Robert says:
Thanks for the response. Yes, when the files are generated they use the correct names I assigned in the respective “Custom location” sections, and they are including all the correct url entries. The only problem is the loc tags in each url entry include the original domain instead of the mapped primary domain. (I didn’t specify earlier, but in my example the “www.betterdomain.com” is set to be the Primary domain.)
As for domain mapping, the site is working correctly. i.e., I am able to access my site from either domain with it redirecting to the primary domain.
What should I do so that the Primary domain is used in the url entries of the xml file?
October 6, 2010 — 7:39 pm
Jan Dembowski says:
That’s a tough one. It looks like the only thing that is different is that I don’t use the domain mapping plugin to mask my primary site.
The only thing I can think of is to re-confirm your settings or try out the development version at this location:
http://wordpress.org/extend/plugins/google-sitemap-generator/download/
October 8, 2010 — 5:45 am
Robert says:
Thanks, Jan. I’ll see what I can do.
October 8, 2010 — 8:32 am
Homer says:
Thanks for the info. FYI, one guy reposted your article without mentioning you at [splog URL removed]
October 13, 2010 — 1:30 pm
Jan Dembowski says:
Yep, thanks I saw that too.
What can you do, they’re spammers. They run afoul of my hotlink image protection and are showing people an image of Hello Kitty Superman.
October 13, 2010 — 2:35 pm
Homer says:
Since I’m lazy to make all the changes, I finally decided to try out the beta version from Arne at the post you mentioned. Lastest version was 4.0b2 released two days ago. Unfortunately it doesn’t seem to work with subdomains. I’m waiting for Arne’s reponse now. If it turns out to be true, I’ll have to go for the difficult way in your post, it isn’t too hard to follow though, since you explained everything clearly enough.
October 30, 2010 — 1:57 pm
Felix says:
hello, i followed the steps, but i found that all sub-directory blog share the same site.
Am i doing something wrong?
Anyone can help?
Thanks,
Felix
November 10, 2010 — 3:55 am
fclub.ro says:
10x a lot, seems to work great. It also works with the WordPress MU Sitewide Tags Pages plugin, so everything is great.
January 25, 2011 — 8:12 pm
SpinnerChief says:
Looks like Arne still hasn’t made the multisite version public. Fortunately the beta version works fine.
February 2, 2011 — 5:46 pm
Jan Dembowski says:
Looking at the problem, it’s not really a trivial update.
With 3.1 just around the corner maybe Arne will polish off that version.
February 2, 2011 — 5:53 pm
Angela DiGiovanni | Living Out Loud says:
I went through all the steps, got the same robot.txt file that you got with the gz on the end, but my links to test it still don’t work with or without the gz on the end.
Any idea what I could have done wrong?
I’m not getting the incompatibility error message anymore on my dashboard for the plugin. It appears that my sitemap was manually rebuilt successfully.
I have my custom locations set as follows:
/home/digang/public_html/angeladigiovanni.com/sitemap.xml
http://angeladigiovanni.com/sitemap-angeladigiovanni.com.xml
I commented out the lines in the plugin file with /* */.
I put my rewrite rules in the .htaccess file at the top, looking like this:
Yet I can’t get those test links to work.
Any thoughts?
Thanks in advance!
Angela
February 4, 2011 — 2:26 am
Jan Dembowski says:
Angela,
That looks like that should work, but when I visit your web site I also get a ‘404 Page not found’ error.
Try something: copy the sitemap-angeladigiovanni.com.xml to test.xml and see if you can load that in your web browser.
February 4, 2011 — 8:02 pm
Daplun says:
where is the real sitemap-yourblogservername.xml and/or sitemap-yourblogservername.xml.gz files?
March 23, 2011 — 10:53 pm
Jan Dembowski says:
Daplun,
See step #3. They are stored in my blog’s root directory.
March 24, 2011 — 5:23 am
Sudhir says:
This works like a breeze !
Thanks for sharing.
May 13, 2011 — 7:18 am
Indian Cars Bikes says:
We were not able to find a plugin for XMS sitemap after we upgraded to the WP Multisite functionality.
We patiently followed all the steps mentioned here and this tutorial finally did the trick for us.
It was very helpful.
May 15, 2011 — 2:25 am
Tim says:
Hi,
thanks for the various comments and the detailed tutorial. Save me a lot of grey hairs.
Thanks
Tim
May 29, 2011 — 11:21 pm
John says:
I followed all the step but still can’t get it to work. I am sure it is me. Are there any other oprion or other plugins for multi site WP sitemaps?
Thanks in advance – John
May 30, 2011 — 12:29 am
Jan Dembowski says:
The easiest thing to do is search though http://wordpress.org/extend/plugins/ and look for “XML sitemaps”.
I am sure that there is at least one available that can handle mutlisite installations of WordPress.
May 30, 2011 — 7:53 am
asma says:
every thing is working fine but look the sitemap is not working without full domain name (www) fun portal
and working fine with fun portal
now plz tell me that how can i view my sitemap with both urls with (www) and without (www)
my google webmaster tool is not accepting my sitemap becoz i add my domian there without (www)
August 23, 2011 — 3:06 pm
Jan Dembowski says:
That’s a tough one. This trick depends on you modifying your .htaccess file as well as having your Apache virtual servers setup correctly. It sounds as if that second part is not correct.
The only advice I can offer is to consider switching to another Sitemap plugin. The one I’ve put into the update on top just works without any changes or .htaccess updates.
August 24, 2011 — 6:45 am
Jasper Tielen says:
Hello,
I am installing wordpress multi site on a few empty domains.
my new main http://www.jps-hosting.nl is now with some new empty domains running.
But i would like to have one big sitemap (index) for all domains.
http://jps-hosting.nl/sitemapindex.xml
is that possible?
Regards,
Jasper
July 30, 2012 — 9:07 am
Jan Dembowski says:
I don’t know; I stopped using that plugin for while now. 😉
It’s probably possible bu I suggest you post the question in that plugin’s dedicated support forum.
July 30, 2012 — 9:19 am