Printer Friendly Version Print this thread
Email this thread to a friend eMail this thread to a friend
  • How i can get Page Rank? (In: Google)
  • all the results in 1 page (In: SEO Tools and Software Utilities)
  • Featured Web Site Template

    Hundreds More at Free Site Templates.com!

    Web Site Partners
    Sponsored Links
    Jet City Software
     
    Whos Here ?
    Reflects user activity within the last 5 minutes
    Moderator(s): yellowwing
    Forum Index · Search Engine Forums · SEF Community and Networking · Professionals Corner · How far does a spider crawl down a page?
    Member Message

    saschaeh
    Joined: May 02, 2005
    # Posts: 66

    View the profile for saschaeh Send saschaeh a private message

    Posted: 2005-May-12 11:00
    Edit Message Delete Message Reply to this message

    I have generated a site map that pulls out spider friendly links to all the dynamic content in my DB. I have three categories with in each city and there are four cities. I have only pulled out one category of one city and it is already fairly long.

    How many full pages does a spider roughly go down. I read somewhere that spiders don’t always go all the way and that weighting is also lost the lower they go.

    1: If this is true how far would you think that is?
    2: I have made it user friendly and broken it down in a hierarchical fashion. Will i get penalized for this kind of thing?


    Thank you for your time!




    g1smd
    Staff
    Joined: Jul 28, 2002
    # Posts: 10437

    View the profile for g1smd Send g1smd a private message

    Posted: 2005-May-12 13:03
    Edit Message Delete Message Reply to this message

    Google caches only the first 100kB of a page. For a short while, a few months ago, they did cache up to 250kB but are no longer doing so for new pages. The figure has reverted to 100kB again. For larger caches generated a few months back they are still retaining the larger page size for those only.

    How much is actually spidered and indexed is left for you to discover, simply by putting some unique "words" on your page and seeing which are indexed and which are ignored.



    Vinnie
    Joined: Nov 03, 2003
    # Posts: 66

    View the profile for Vinnie Send Vinnie a private message

    Posted: 2005-May-13 15:59
    Edit Message Delete Message Reply to this message

    I may be have misunderstood what you are saying, but if I have not here is my take on it.
    Now this is coming from a brand new site released at the begining of last week with some real good inbound quality links to it. Google indexed the entire front page of 322 words. But having said that the entire site is built in XHTML and div tags, no tables no interuptions at all, clean stylish, optimised images, and readable.
    Now I am wondering if there had been tables etc if it would have done the same! I'm just waiting now until it makes it's damn mind up to get to the other pages in its own good time.



    saschaeh
    Joined: May 02, 2005
    # Posts: 66

    View the profile for saschaeh Send saschaeh a private message

    Posted: 2005-May-13 20:29
    Edit Message Delete Message Reply to this message


    Thanks! Ill def use your guys advice!

    Where did you find out about the 100K thing?
    Awesome tip on putting unique words at different levels of your page.


    All the best things!
    Sascha tongue



    g1smd
    Staff
    Joined: Jul 28, 2002
    # Posts: 10437

    View the profile for g1smd Send g1smd a private message

    Posted: 2005-May-13 21:12
    Edit Message Delete Message Reply to this message



    The 100KB limit has been mentioned in many forums, and might also be found in the Google Guidelines for Webmasters, somewhere on their site.

    They did cache up to 250KB for a short while earlier in the year. The evidence was seen in the SERPs. They now only cache up to 100KB for new pages (as I found out when I briefly tested with a 120KB file last month).


    Forum Index · Search Engine Forums · SEF Community and Networking · Professionals Corner · How far does a spider crawl down a page?
    You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:
    1. You have not yet logged in, or registered properly as a member
    2. You are a member, but no longer have posting rights.
    3. This is a private forum, for which you do not have permissions.

    If you are a recent member, it's possible that you simply have not yet confirmed your account. Please check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions contained within.

    If you cannot find this message, click here to Re-Send it.

    If you are still experiencing problem, please read the Login Assistance Article for some advice on what may be causing your login not to work properly.

    Switch to Advanced Editor and ... Create a New Topic or Reply to this Thread

    New posts Forum is locked
    © 1995  ·  iWeb, Inc  ·  DBA JimWorld Productions