Posts

Showing posts from October, 2014

GoogleBot

Googlebot  is Google's web crawling bot (sometimes also called a "spider"). Crawling is the process by which Googlebot discovers new and updated pages to be added to the Google index. We use a huge set of computers to fetch (or "crawl") billions of pages on the web. Googlebot uses an algorithmic process: computer programs determine which sites to crawl, how often, and how many pages to fetch from each site. Googlebot's crawl process begins with a list of webpage URLs, generated from previous crawl processes and augmented with Sitemap data provided by webmasters. As Googlebot visits each of these websites it detects links (SRC and HREF) on each page and adds them to its list of pages to crawl. New sites, changes to existing sites, and dead links are noted and used to update the Google index. For webmasters: Googlebot and your site How Googlebot accesses your site For most sites, Googlebot shouldn't access your site more than on

Chrome Remote debugging protocol

Image
Chrome Remote debugging protocol Under the hood, Chrome Developer Tools is a web application written in HTML, JavaScript and CSS. It has a special binding available at JavaScript runtime that allows interacting with chrome pages and instrumenting them. Interaction protocol consists of commands that are sent to the page and events that the page is generating. Although Chrome Developer Tools is the only client of this protocol, there are ways for third parties to bypass it and start instrumenting browser pages explicitly. We will describe the ways it could be done below. Note: If you are interested in inspecting remote pages on Chrome for Android, please see the remote debugging documentation . For users wishing to implement custom inspection code using our debugger protocol instead, please use the instructions below. Protocol Interaction protocol consists of JSON commands that are sent to the page and events that the page is generating. We define this protocol in Blink (&quo