Kryptronic Software Support Forum

You are viewing this forum as a guest. Login to an existing account, or create a new account, to reply to topics and to create new topics.

#1 01-09-2004 13:15:48

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Creating Search Engine Friendly Pages

This is what I've found out about Apache Mod Rewrite and what we can do to use it to help create search engine friendly urls.

I'm going to take a standard url and rewrite to a friendly version that spiders will pickup and index. This involves writing a rule and placing it in an .htaccess file on your server in your main directory, and then coding links on your site in the rewritten version so that if Google comes by or someone clicks on the link it will display a certain way.

In a nutshell we want to take this link to display all the products in a category:

wwwyoursite.com/shop/cgi-bin/cp-app.cgi? usr=51F6296048&rnd=7528108&rrc=N&affl= &cip=65.32.153.69&act=&aff=&catstr=HOME:Baseball &pg=ste_cat&ref=Baseball

And when spidered by Google or clicked by someone display like this:

wwwyoursite.com/cp-app/ste_cat/Baseball … t/Baseball

This will display the baseball category page and all the products in it up to whatever you have your setting per page set at in admin, say the max at 30.

I'm working with Nick to have him give me something that will allow more than your max number per page, or better yet, output the link (via a mod) at the bottom of the page in this same format so the spider will follow it from page to page.

So for now, this is how it is done, and will work for you if you have less than 30 products in a given category.

1. Create in Notepad a file with this in it:

Options +FollowSymLinks
RewriteEngine on
RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ shop/cgi-bin/cp-app.cgi?pg=$1&ref=$2 [L]

2. Save the file as .htaccess and upload to the top level directory, where you site's main index.html page is.

3. Create a link on your site, homepage, in ste_layout.txt, a sitemap(excellent idea regardless), wherever, like this:

<a href="/cp-app/ste_cat/Baseball">Baseball</a> Baseball being the reference name for the category I've entered when I made the category.

Now when a spider comes by or someone clicks that link it displays as:

wwwyoursite.com/cp-app/ste_cat/Baseball  in the address bar.

Once someone clicks on a product, or sub category it throws all the other variables in there as usual, but for spidering purposes, this page can now be picked up. A prudent method might be to leave your category links on the home index page and store pages as normal, but make a sitemap with among other things, links to all your categories and sub categories rewritten as described above with a couple links from your static home page to your static sitemap so Google will come by and follow the links to eventually spidering each category for you.

After all, aren't we trying to get them spidered somehow right?

Once I hear from Nick on this I will be able to determine if I need to include something like searchrpp=1000 in the links on the sitemap to allow up to 1000 products per page and Google stumbling on the <prev> <next> links at the bottom and stopping......or if nothing is needed and the link as written will display an unlimited number of products per page.

Later today I will post a rewritten product link that can be used if you want to make links work to individual products, and also a search link so you can search on a term and return all the products on that results page with no pagination in a good friendly link.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#2 01-09-2004 15:07:05

TerryA
Member
From: Sanford, Fl
Registered: 07-14-2003
Posts: 1322
Website

Re: Creating Search Engine Friendly Pages

Kool!  Keep us posted on this!

Offline

 

#3 01-09-2004 15:17:20

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

To write a link to a search term, you must write another rule and enter it right below the previous rule in your .htaccess file. There is no need to repeat this part:

Options +FollowSymLinks
RewriteEngine on

Here is a rule to add below the other one for a search term:

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/([^/]*)/?$ shop/cgi-bin/cp-app.cgi? pg=$1&searchtyp=$2&searchrpp=$3&searchstr=$4 [L]

Which can be written on your site as:

href="www.yoursite.com/cp-app/ste_search_results/EXACT/10000/Batting  Gloves">Autographed Baseball Batting Gloves<    adding the additional parts needed to make it a link.

This will display in the address bar as url:

wwwyoursite.com/cp-app/ste_search_resul … 00/Batting Gloves

and provide you with an unpaginated search result page of products for the term "Batting Gloves" up to 10000 products in that category. I just picked a high number so I'd never have a pagination problem or have to worry about it.

If Nick can create a mod to somehow write the <prev> <next> link at the bottom of the page in this rewritten form we again can eliminate the problem of spiders stopping at the pagination and eliminate the need for that variable in the link, but for now it works.  For someone with 10, 15, 50 products in a category it would make for a nice deal with all products on one page.

This search function link style can be handy dandy. You could enter a keyword in the keyword field for a grouping of products and do many links like this on your sitemap page. That way you are criss crossing your entire product base and getting products to be spidered numerous different times and ways.

I think if you are creative with your products, groupings, and use keywords to group them the way your categories are set up, you could have a sitemap or links all over your static index page that will get your important products spidered, or all your products spidered if your catalog is not huge and you make a nicely designed group of links on a sitemap.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#4 01-09-2004 22:11:27

sgreiner
Member
Registered: 07-26-2003
Posts: 78

Re: Creating Search Engine Friendly Pages

Adding the first mod to a link to test the results.  My problem so far is after I create the .htaccess file with the code inside, I upload to my server but it disappears. Maybe I am not allowed to upload these types of files?  Not sure what the problem is.  Eaglewolf, any thoughts..... you host my site smile


Scott

Offline

 

#5 01-09-2004 22:20:53

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

You will not see the file using an ftp program.

If you have a file viewer or such with your host control panel, try and see if it is there that way. If you can edit it there too in the future, thats the way to do it.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#6 01-09-2004 22:29:59

sgreiner
Member
Registered: 07-26-2003
Posts: 78

Re: Creating Search Engine Friendly Pages

Thanks blade... that is exactly how I transferred the file. First time working with this file type. I modified it and it works great.


Scott

Offline

 

#7 01-17-2004 11:21:44

webmaster
Administrator
From: York, PA
Registered: 04-20-2001
Posts: 19798
Website

Re: Creating Search Engine Friendly Pages

Rewrite rule for products & categories:

RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ shop/cgi-bin/cp-app.cgi?pg=$1&ref=$2 [L]

For URLs like this:

wwwitsalreadysigned4u.com/prod/111
wwwitsalreadysigned4u.com/cat/222

Where 'prod' and 'cat' are the CCP page names and the '111' and '222' are the product and category reference strings.

For previous/next results, you'd have to modify the URLs for next/previous links in the first routine in the file ./cgi-bin/library/modules/ste_prod.pl.  You would replace:

Code:


print <<ENDOFTEXT;
<A HREF=\"$common_url&pg=$fd_pg&startrow=$prev_startrow&searchstr=$fd_searchstr_encoded&searchrpp=$store_products_to_display_per_page&searchtyp=$fd_searchtyp_encoded&ref=$fd_ref_encoded&cat=$fd_cat_encoded\">$html_previous_string</A> \| 
ENDOFTEXT

} else {

print <<ENDOFTEXT;
$html_previous_string \| 
ENDOFTEXT

} ######### End of if statement.

if ($fd_startrow < "$rows_count" && $next_startrow < "$rows_count") {

print <<ENDOFTEXT;
<A HREF=\"$common_url&pg=$fd_pg&startrow=$next_startrow&searchstr=$fd_searchstr_encoded&searchrpp=$store_products_to_display_per_page&searchtyp=$fd_searchtyp_encoded&ref=$fd_ref_encoded&cat=$fd_cat_encoded\">$html_next_string</A>
ENDOFTEXT

With:

Code:


print <<ENDOFTEXT;
<A HREF=\"YOUR_DOMAIN_NAME_HERE/$fd_pg/$prev_startrow/$fd_searchstr_encoded/$store_products_to_display_per_page/$fd_searchtyp_encoded/$fd_ref_encoded/$fd_cat_encoded\">$html_previous_string</A> \| 
ENDOFTEXT

} else {

print <<ENDOFTEXT;
$html_previous_string \| 
ENDOFTEXT

} ######### End of if statement.

if ($fd_startrow < "$rows_count" && $next_startrow < "$rows_count") {

print <<ENDOFTEXT;
<A HREF=\"YOUR_DOMAIN_NAME_HERE/$fd_pg/$prev_startrow/$fd_searchstr_encoded/$store_products_to_display_per_page/$fd_searchtyp_encoded/$fd_ref_encoded/$fd_cat_encoded\">$html_next_string</A>
ENDOFTEXT

And use a re-write rule like:

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/([^/]*)/([^/]*)/([^/]*)/([^/]*)/?$ shop/cgi-bin/cp-app.cgi?pg=$1&startrow=$2&searchstr=$3&searchrpp=$4&searchtyp=$5&ref=$6&cat=$7 [L]


Nick Hendler

Offline

 

#8 01-18-2004 12:15:40

shibumik
Member
Registered: 10-19-2003
Posts: 216
Website

Re: Creating Search Engine Friendly Pages

Here is probably a stupid question....

How does this situation apply to Solaris and Linux (or any non-Apache UNIX server) servers with regard to searchability of pages?

Thanks in advance!



Offline

 

#9 01-18-2004 12:18:59

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

I'm not sure exactly. I believe this only applies to an Apache Server with the mod rewrite module installed and enabled.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#10 01-18-2004 21:55:08

scoutch
Banned
Registered: 07-03-2003
Posts: 3167

Re: Creating Search Engine Friendly Pages

I also think it involves mySQL Database.


______________________________________________

THIS USER HAS BEEN BANNED FROM THIS FORUM.

If this post contains any language related to
code samples, advice, etc., please read this
entire thread before making a decision to use
this post as a basis for any change to your
software installation.
______________________________________________

Offline

 

#11 01-19-2004 00:45:45

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

Nick,
     I tried the Rewrite rule for products & categories.  I put this:
RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ shop/cgi-bin/cp-app.cgi?pg=$1&ref=$2 [L]
in a .htaccess file.  I put it in the ccp51 folder. 
my store is in wwwyourdomain.com/ccp51

I created a test link on my splash page.
Here is the link I created:
<a href="http://www.yourdomain.com/cat/100">Test Link</a>

My Main categories start with 100, 110, 120, etc  My sub categories start with 10001, 10002, 11001, 11002, 12001, 12002, etc

When I click on the link the category of 100 doesn't come up.  It says page cannot be found.

Am I doing something wrong?


Charles

Offline

 

#12 01-19-2004 10:17:26

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

First..  .htaccess should be in the highest dir possible, so move to your actual site directory wwwyourdomain.com or public_html whatever your main directory is.

Also, this:

<a href="http://www.yourdomain.com/cat/100">Test Link</a>

Should be this:

<a href="http://www.yourdomain.comcat/100">Test Link</a>

and 100 should be your category ref string NOT the category name, just in case that is confusing too.

Try all that and let me know.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#13 01-19-2004 10:58:09

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

I moved my .htaccess file to the highest dir.
I changed the Test Link to
<a href="http://www.yourdomain.com/cp-app/cat/100">Test Link</a>

100 is my cat. ref. string
It still won't work.  Last night I asked my webhosting company if Mod Rewrite was installed and enabled they said it was installed and enabled.

I was wondering if the word shop should be changed to ccp51 since that is where my store in located.
Options +FollowSymLinks
RewriteEngine on
RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ /cgi-bin/cp-app.cgi?pg=$1&ref=$2 [L]


Charles

Offline

 

#14 01-19-2004 13:43:54

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

Yes yes yes, my bad I didn't catch that.

Your rewrite rule would be:

RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ ccp51/cgi-bin/cp-app.cgi?pg=$1&ref=$2 [L]

Link is ok, that should do it.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#15 01-19-2004 15:01:17

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

Perfect It works now
Thank You very much.

So all I have to do is create more links to my Cat. and Products.
One other question though.
How will spiders know where to get these links.  I know there was something said about a Site Map but where should I put it?  Do I just put the links into a html file in my root directory?     


Charles

Offline

 

#16 01-19-2004 15:12:32

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

I did notice though that when I went to a sub-cat. that the Breadcrumbs don't work all that well.  I think that I will have to create another Rewrite rule based on this:
pg=cat&ref=10001&catstr=HOME:100

I just have to copy the same one but add the part about catstr but I am not sure if that will work or not. 

100 is Collectibles
10001 is African American Heritage

If I go to wwwyourdomain.com/cp-app/cat/10001 in the Breadcrumbs it shows
Home>African American Heritage instead of showing Home>Collectibles>African American Heritage


Charles

Offline

 

#17 01-19-2004 15:27:38

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

I tried this link
and the breadcrumbs work correctly. 
Here is the new Rewrite Rule that I added:

RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ ccp51/cgi-bin/cp-app.cgi?pg=$1&ref=$2&catstr=$3 [L]

I think that the : is messing things up.  I don't know how to put that into the Rewrite Rule


Charles

Offline

 

#18 01-19-2004 15:48:21

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

If you are using the category menu that comes with CCP spiders will not handle those links. I would do one of two things.

1. Eliminate the CCP cat menu and just hard code the links to your home page and your ste_layout file, rewritten as you've done.

2. Leave the CCP menus alone, and do a nice detailed sitemap with rewritten links to categories and important product pages on the sitemap. Put sitemap.html in your root directory along with your main index page, and on your index page, link to your sitemap so google will find it and hit all the links from there. (My preference)

With regards to your breadcrumbs. Dont worry about that. Leave the menus and links written normally in CCP and your homepage. People will not go to your sitemap page to navigate around.  Make a separate link to ALL your categories and sub categories individually on your sitemap. All you want is google to pick up those pages and index all the words to your items on there. By leaving all the other links alone your customers will be just fine along with the breadcrumbs. Just make a link to your sitemap from your index page somewhere at the bottom discretely just so google will find it and follow it. Customers or people coming to your site need not find it or use it at all. These rewritten links are merely for spiders to index your site, not for people to use and move around your site.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#19 01-19-2004 15:57:31

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

Can google spider a link like this:


If not that is what I am trying to fix if I can.  I want to when google spiders it that the Breadcrumbs so up correctly.  I am trying to prevent any problems from happening later on down the road if any at all.


Charles

Offline

 

#20 01-19-2004 16:08:29

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

Charles,01/19/2004 03:57:31 PM wrote:

Can google spider a link like this:


If not that is what I am trying to fix if I can.  I want to when google spiders it that the Breadcrumbs so up correctly.  I am trying to prevent any problems from happening later on down the road if any at all.

Possibly, but not easily.

I might rearrange too and see what happens:

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/?$ ccp51/cgi-bin/cp-app.cgi?pg=$1&catstr=$2&ref=$3 [L]

then the link would be:



You only had 2 variables in the front of your rule and three in the back. The one I posted above now has 3 both front and back.

Worse case do them like this....wait a month or so to see what gets spidered and go from there. Thats the only way you'll truly know for sure if it like the colon or not.

Make a couple links to your sitemap on your index page and any static html pages you have other than that one so it is easy for google to finds the sitemap.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#21 01-19-2004 16:13:34

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

I fixed the ReWrite Rule
I tried the URL:
but it didn't work.
I then tried this URL

It worked sorta but the Breadcrumbs look like this;
HomeHome > Collectibles


Charles

Offline

 

#22 01-19-2004 17:04:51

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

Charles,01/19/2004 04:13:34 PM wrote:

I fixed the ReWrite Rule
I tried the URL:
but it didn't work.
I then tried this URL

It worked sorta but the Breadcrumbs look like this;
HomeHome > Collectibles

Perhaps the colon IS an issue. Like I said before, I wouldn't code the links with it in there anyway. It's really of no value to what we are trying to do here bottom line. Links on your sitemap don't need it in there.


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

#23 01-19-2004 17:57:11

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

BINGO
I figured it out
wwwyourdomain.com/cat/10001/HOME/100 (I know this is spider friendly!!)

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/([^/]*)/?$
ccp51/cgi-bin/cp-app.cgi?pg=$1&ref=$2&catstr=$3:$4 [L]

This will show the correct breadcrumbs.  I had to add another varible in the rule.
My sub-cat. ref string is part of my cat. ref string so I just have to make sure I code the right URL on my links.


Charles

Offline

 

#24 01-19-2004 18:41:41

Charles
Member
From: USA
Registered: 12-24-2002
Posts: 252

Re: Creating Search Engine Friendly Pages

Here is a copy of my Rewrite Rules

Options +FollowSymLinks
RewriteEngine on

RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ ccp51/cgi-bin/cp-app.cgi?pg=$1&ref=$2 [L]
#
# For Categories (with Breadcrumbs)
# wwwyourdomain.com/cp-app/cat/130
#
# For Products (without Breadcrumbs)
# wwwyourdomain.com/cp-app/prod/30209
#

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/([^/]*)/?$
ccp51/cgi-bin/cp-app.cgi?pg=$2&ref=$4&catstr=$1:$3 [L]
#
# For Sub-Categories (with Breadcrumbs)
# wwwyourdomain.com/cp-app/HOME/cat/130/13001
#

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/([^/]*)/([^/]*)/?$
ccp51/cgi-bin/cp-app.cgi?pg=$2&ref=$5&catstr=$1:$3:$4 [L]
#
# For Products (with Breadcrumbs)
# wwwyourdomain.com/cp-app/HOME/prod/130/13001/30209
#

You can remove the cp-app/ in the Rewrite Rule the url then would be
wwwyourdomain.com/cat/130 (with Breadcrumbs)
wwwyourdomain.com/prod/30209 (without Breadcrumbs)
wwwyourdomain.com/HOME/cat/130/13001
Etc.....

Replace ccp51 in the Rewrite rule with the directory your store is in.
wwwyourdomain.com/store is where your store is replace ccp51 with the word store.  There is a space before ccp51!!!! wink  wink


Charles

Offline

 

#25 01-19-2004 18:47:57

theblade24
Member
From: Tampa, Florida
Registered: 11-19-2003
Posts: 384
Website

Re: Creating Search Engine Friendly Pages

Charles,01/19/2004 06:41:41 PM wrote:

Here is a copy of my Rewrite Rules

Options +FollowSymLinks
RewriteEngine on

RewriteRule ^cp-app/([^/]*)/([^/]*)/?$ ccp51/cgi-bin/cp-app.cgi?pg=$1&ref=$2 [L]
#
# For Categories (with Breadcrumbs)
# wwwyourdomain.com/cp-app/cat/130
#
# For Products (without Breadcrumbs)
# wwwyourdomain.com/cp-app/prod/30209
#

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/([^/]*)/?$
ccp51/cgi-bin/cp-app.cgi?pg=$2&ref=$4&catstr=$1:$3 [L]
#
# For Sub-Categories (with Breadcrumbs)
# wwwyourdomain.com/cp-app/HOME/cat/130/13001
#

RewriteRule ^cp-app/([^/]*)/([^/]*)/([^/]*)/([^/]*)/([^/]*)/?$
ccp51/cgi-bin/cp-app.cgi?pg=$2&ref=$5&catstr=$1:$3:$4 [L]
#
# For Products (with Breadcrumbs)
# wwwyourdomain.com/cp-app/HOME/prod/130/13001/30209
#

You can remove the cp-app/ in the Rewrite Rule the url then would be
wwwyourdomain.com/cat/130 (with Breadcrumbs)
wwwyourdomain.com/prod/30209 (without Breadcrumbs)
Etc.....

Replace ccp51 in the Rewrite rule with the directory your store is in.
wwwyourdomain.com/store is where your store is replace ccp51 with the word store.  There is a space before ccp51!!!! wink  wink

Looks like you got the hang of it!!!!!!

Did you do the mod to make the <prev> <next> links work?


CCP 5.1
CCP 5.1
CCP 5.1

Offline

 

Board footer