Kryptronic Software Support Forum

I have all of my mod-rewrite linked pages showing up in google, but it took creating a static page wwwyoursite/sitemap.html  which is basically a page with all of the rewritten links to every product we offer, then I linked to this page from my index. (small text on the bottom of the page that no one ever looks at) 

This is the page that really got google interested.  Seems lately that google doesn't really wanna go too deep into the site. Since sitemap.html is in the root, it will quickly index everything on that page.  That's the way I got it working.... don't know if it will help you or not, but its worth a shot.

We do have a sitemap.xml for Google and the pages are still not showing up.

I don't understand whether or noe we use someone else's mod for the mod_rewrite, it should work.

Yahoo! and MSN indexed all of the mod_rewrite pages okay.

Are there any other ideas why Google isn't following the links?

Hi Kelly,

The site went live early last summer 2005.
Yes, there are some external links to it. I manage other sites with the exact same architecture but with entirely different content (products). Google managed to index their mod_rewrite pages okay.

Since I started pursung why Google is finicky, a few more pages showed up in the search engine.
HOWEVER, these pages are from when we first launched the site last early summer, and the pages no longer exist.

Get this. Numerous pages are listed okay at Google-UK. It's Google-USA that isn't behaving. The site is a USA business. Go figure. (No wonder we get orders from the UK). (We had to implement mod_rewrite to redirect the wrong pages in Google).

What really puzzles me is that Google reads my sitemap (registered with Google months ago), but still doesn't index the correct pages.

I'm starting to believe that Google is trying way too hard and their indexing system has run amuck.

Or could they know that I am a supporter of AdWords and they don't want to index it, or put it in a low priority to index? That would be hard to believe, but not impossible to do.

I'm still looking for a resolution.

- Baffled BLitzen

Same results if you try allinurl:yoursite.com (no www)?

I agree with Kelly, in this case a url would be helpful.

Got It! We were caught up in the Google Dance.
Now over 160 pages are indexed.

Hear this! Not only are the SE-friendly pages indexed, but so are the dynamic links (...cgi-bin/cp-app.cgi?param=value).

That makes for duplicate pages, now. We can restrict the bots to the ...cgi-bin/cp-app.cgi?param=value files in robots.txt to alleviate this problem.

Thank you for all your consideration.

BTW - How did we know that it was caught up in the Google Dance? The symptoms were
1. On one day, the correct pages showed up. On another day, a different list appeared.

2. On one Google server, the correct pages were indexed. On another Google servce, the old nonexistent one were indexed.

3. When I was consulting with an SEO expert, he saw over 130 pages indexed. As he did his search, I did the same and got the 17 nonexistent pages.

4. Last night, we saw 64 pages indexed. An hour later, over 130 pages were indexed. Today, over 160 pages are indexed.

Here is a good link describing the Google Dance.


This makes me believe that the Google Dance is the cause of the rumours about the Google Sandbox Hypothesis.

Sorry - I know that many of you were asking. We were overwhelmed with this issue. 

We searched advice from a broad area. Most replies we received were the basic SEO strategies that weren't relevant to our case. Some suggestions were red herrings and some were just not applicable. You see, I don't submit a post until the basics are covered (I check that the computer is plugged in before calling for tech support).  Of course, there is the the chance of a simple oversight - as exemplified in this sentence.

We appreciate the input from everyone. Your input encouraged us to reach farther into our problem. After approaching some SEO specialists, we finally found one that led us in the right direction with new-old ideas (e.g., Sandboxing).

Unfortunately, there is nothing that can be done about the Google Dance - except wait. Once G gathers the data, the dance begins. Whether or not sandboxing is a myth and applied to this case, we are happy that the site is on G. The PR'ing effort begins.

You might want to bookmark the following site. There are many Google Server URLs listed here (Note, we had to find G's IPs through searching  on Google).

Bear in mind that G pulls from over 10,000 servers and can change these IP's to keep us awake. We found a site that listed some expired Google IPs.

Happy marketing!
===================

Resolved. Irrelevant content removed.

Comment deleted. Moved to