Kryptronic Software Support Forum

sdn · 02-17-2017 04:30:40

GWT is reporting duplicate Category pages. More and more each day and a problem experienced with CCP8 initially that did great damage to our site ranking until we finally got it fixed.

The Helper suggests the htaccess code should be

RewriteRule ^index.php - [L,PT]
RewriteRule ^$ index.php?COREseourl=/ [L,PT]
RewriteRule ^/$ index.php?COREseourl=/ [L,PT]
RewriteRule ^sitemap\.xml utilities/sitemap.xml [L,PT]
RewriteCond %{REQUEST_URI} ^/
RewriteCond %{REQUEST_URI} !\.(xml|css|js|gif|jpg|jpeg|png|pdf)$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*) index.php?COREseourl=/ [L,PT]

Even with everything else stripped out of htacess and just this code, clicking on any site link changes the url in the browser address bar but the page always stays on the home page. If you start from a different page and upload this htaccess code clicking any link takes you back to the home page.

So we solved the problem by changing the above to the CCP8 code which had $1 in the last line as follows

RewriteRule ^index.php - [L,PT]
RewriteRule ^$ index.php?COREseourl=/ [L,PT]
RewriteRule ^/$ index.php?COREseourl=/ [L,PT]
RewriteRule ^sitemap\.xml utilities/sitemap.xml [L,PT]
RewriteCond %{REQUEST_URI} ^/
RewriteCond %{REQUEST_URI} !\.(xml|css|js|gif|jpg|jpeg|png|pdf)$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*) index.php?COREseourl=/$1 [L,PT]

That at least makes the site usable but I suspect this mod is causing the duplicate content issue which I want to rectify PDQ.

So we need a site that both works and doesn't upset Google. Please advise soonest.

Last edited by sdn (02-17-2017 04:38:08)

webmaster · 02-17-2017 07:32:47

The $1 should be in the helper, I'm adding it in today with the update. Oversight here. GWT should not be reporting duplicate content with that setup (adding the $1 on the last line is the only change needed), so we're going to need a bit more info from you on this. Can you provide two URLs which Google sees as duplicates so we can take a look?

sdn · 02-17-2017 08:32:30

OK so we already fixed the htaccess file. An example is:

https://www.quasarelectronics.co.uk/Cat … lay-boards
https://www.quasarelectronics.co.uk/ind … 00p320c50s

Listed in both duplicate meta decriptions and title tags.

Canonical points to the SEO url for both versions.

Last edited by sdn (02-17-2017 08:33:21)

webmaster · 02-17-2017 11:21:12

That second URL should internally be doing a 301 redirect. If you view the source, you'll see that the canonical URL is coming up correctly. I'm not sure why that's not happening. It looks like something is forcing all URLs to https. Do you have Use SSL For All Requests turned on, or is something else doing that? Do you have both your URLs in your installer set to https URLs?

sdn · 02-17-2017 12:41:32

Yes, we had a CCP8 rule in there forcing the site to https now remmed out. We also have some other rules detailed below. I trust these are OK. Not sure if the RewriteCond %{HTTP_HOST} ^.*$ one is relevant now.

# Force all pages to be delivered as https
#RewriteCond %{SERVER_PORT} 80
#RewriteRule ^(.*)$ https://www.quasarelectronics.co.uk/$1 [R=301,L]

# Redirect all requests to www
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.quasarelectronics\.co\.uk$
RewriteRule (.*) https://www.quasarelectronics.co.uk/$1 [L,R=301]

# Redirect lower to upper case
RewriteRule ^item\/(.*)$ Item/$1 [L,R=301]
RewriteRule ^category\/(.*)$ Category/$1 [L,R=301]

# Redirect Old URLs to new CCP8 URLs
RewriteCond %{HTTP_HOST} ^.*$

We also have a + sign in our symlinks line (can't remember why now).

Options +FollowSymLinks
RewriteEngine on

plus we also use the following to shorten the other rewrites.

RewriteBase /

These are all above the K9 code.

Last edited by sdn (02-17-2017 14:08:15)

webmaster · 02-20-2017 08:33:15

I'm not sure what this does, or if it's hurting. It's a condition only, not a rule, didn't know you could do that:

Code:

RewriteCond %{HTTP_HOST} ^.*$

Those SSL rewrites need to stay commented, or remove it. Under System / Component / Settings / Security Settings, set Use SSL for All Requests to true to handle that.

The duplicate content issue itself should be handled by the canonical URLs, which are being printed correctly to the page. In your case, that doesn't seem to be working, unless you think the site may have been indexed prior to you turning SEO on, in that case this will all shake out over a few days. Regardless, I see an opportunity in the code to combat this further and will put something in the 9.0.20 update (which is due out ASAP) to handle it.

sdn · 02-20-2017 09:29:33

OK thanks.

I thought that the problem had been resolved but Google have reported more pages this morning.

I have just now removed RewriteCond %{HTTP_HOST} ^.*$ as I cannot remember why it went in along with a few other redirects going back to CCP6.

We still have v8 in a folder of its own as we want to refer to it for some styling info. It is set to noindex and the same in robots.txt and almost all products and categories are set to not visible. I did forget to disable the sitemap submission so maybe that was causing Google to get confused.

Do you have any other ideas?

Last edited by sdn (02-21-2017 05:57:57)

sdn · 04-03-2017 08:23:47

GWT added EmailList and EmailListUnSubscribe to pages with duplicate title tags. Where do I go to amend them so they a different?

webmaster · 04-04-2017 08:59:49

Thanks for the report. We're going to have to add noindex tags there. Probably the best way to handle this is to put the following code above the 'jslib' namespace call in your skin:

Code:

if (preg_match('/^emaillist/',$this->globals('core.namespace'))) {$this->globals('core_display.noindex',1);}

sdn · 04-05-2017 22:16:24

OK that works.

Last edited by sdn (04-06-2017 10:16:09)

sdn · 04-06-2017 10:16:38

Are there any other pages we might need to do the same with?

webmaster · 04-07-2017 07:50:13

Not that I can think of. We're using noindex pretty much everywhere in K9 where it should be used in order to avoid these types of issues.

Kryptronic Software Support Forum

#1 02-17-2017 04:30:40

GWT Reporting Duplicate Content

#2 02-17-2017 07:32:47

Re: GWT Reporting Duplicate Content

#3 02-17-2017 08:32:30

Re: GWT Reporting Duplicate Content

#4 02-17-2017 11:21:12

Re: GWT Reporting Duplicate Content

#5 02-17-2017 12:41:32

Re: GWT Reporting Duplicate Content

#6 02-20-2017 08:33:15

Re: GWT Reporting Duplicate Content

Code:

#7 02-20-2017 09:29:33

Re: GWT Reporting Duplicate Content

#8 04-03-2017 08:23:47

Re: GWT Reporting Duplicate Content

#9 04-04-2017 08:59:49

Re: GWT Reporting Duplicate Content

Code:

#10 04-05-2017 22:16:24

Re: GWT Reporting Duplicate Content

#11 04-06-2017 10:16:38

Re: GWT Reporting Duplicate Content

#12 04-07-2017 07:50:13

Re: GWT Reporting Duplicate Content

Board footer