HCE Project Python language Distributed Tasks Manager Application, Distributed Crawler Application and client API bindings.  2.0.0-chaika
Hierarchical Cluster Engine Python language binding
dc_crawler.RobotsParser.RobotsParser Member List

This is the complete list of members for dc_crawler.RobotsParser.RobotsParser, including all inherited members.

__init__(self, headers=None, isCacheUsing=False, robotsFileDir=None)dc_crawler.RobotsParser.RobotsParser
cacheElementdc_crawler.RobotsParser.RobotsParser
cacheElementKeysdc_crawler.RobotsParser.RobotsParser
checkUrlByRobots(self, url, siteId=None, headers=None)dc_crawler.RobotsParser.RobotsParser
headersdc_crawler.RobotsParser.RobotsParser
HTTP_OK_CODEdc_crawler.RobotsParser.RobotsParserstatic
initFiends(self, headers=None, isCacheUsing=False, robotsFileDir=None)dc_crawler.RobotsParser.RobotsParser
loadRobots(self, url, siteId=None, additionHeaders=None, proxyName=None)dc_crawler.RobotsParser.RobotsParser
localCrawlerDataStoragedc_crawler.RobotsParser.RobotsParser
localParserdc_crawler.RobotsParser.RobotsParser
ROBOTS_PATTERNdc_crawler.RobotsParser.RobotsParserstatic
robotsFileDirdc_crawler.RobotsParser.RobotsParser
USER_AGENT_HEADER_NAMEdc_crawler.RobotsParser.RobotsParserstatic