Uses of Package
com.norconex.crawler.web.doc.operations.robot

Packages that use com.norconex.crawler.web.doc.operations.robot

Package	Description
com.norconex.crawler.web
com.norconex.crawler.web.doc.operations.delay
com.norconex.crawler.web.doc.operations.delay.impl
com.norconex.crawler.web.doc.operations.robot
com.norconex.crawler.web.doc.operations.robot.impl
com.norconex.crawler.web.doc.pipelines.importer
com.norconex.crawler.web.util

Classes in com.norconex.crawler.web.doc.operations.robot used by com.norconex.crawler.web

Class	Description
RobotsMetaProvider	Responsible for extracting robot information from a page.
RobotsTxtProvider	Given a URL, extract any "robots.txt" rules.

Classes in com.norconex.crawler.web.doc.operations.robot used by com.norconex.crawler.web.doc.operations.delay

Class	Description
RobotsTxt	Immutable holder of robots.txt rules.

Classes in com.norconex.crawler.web.doc.operations.robot used by com.norconex.crawler.web.doc.operations.delay.impl

Class	Description
RobotsTxt	Immutable holder of robots.txt rules.

Classes in com.norconex.crawler.web.doc.operations.robot used by com.norconex.crawler.web.doc.operations.robot

Class	Description
RobotsMeta
RobotsTxt	Immutable holder of robots.txt rules.
RobotsTxt.RobotsTxtBuilder
RobotsTxtFilter	Holds a robots.txt rule.

Classes in com.norconex.crawler.web.doc.operations.robot used by com.norconex.crawler.web.doc.operations.robot.impl

Class	Description
RobotsMeta
RobotsMetaProvider	Responsible for extracting robot information from a page.
RobotsTxt	Immutable holder of robots.txt rules.
RobotsTxtProvider	Given a URL, extract any "robots.txt" rules.

Classes in com.norconex.crawler.web.doc.operations.robot used by com.norconex.crawler.web.doc.pipelines.importer

Class	Description
RobotsMeta

Classes in com.norconex.crawler.web.doc.operations.robot used by com.norconex.crawler.web.util

Class	Description
RobotsTxt	Immutable holder of robots.txt rules.