Submitted by rob on Fri, 12/02/2011 - 16:11
[this is a summary of question asked via support email]
I entered the website [URL provided] as a seed, and the crawler ran, but there are no outbound links and in the "add seeds" window the seed URL is listed as one of the "Seed sites you have added that have been accepted by the administrator and are pending crawling". What's wrong?
Forums:
We fixed this problem on 02Dec2011 (version 0.6.1.0). The crawler was not properly identifying the encoding of some web pages and hence it was choking on these pages and not crawling them properly. This was only happening for a small number of web pages where the encoding is declared in an unusual way.
This problem is now fixed.