Inheritance diagram for dc_crawler.CrawledResource.CrawledResource:

Collaboration diagram for dc_crawler.CrawledResource.CrawledResource:

Public Member Functions
def	__init__ (self)

Public Attributes
	html_content

	binary_content

	response_header

	html_request

	content_type

	charset

	error_mask

	crawling_time

	http_code

	bps

	last_modified

	etag

	resource_changed

	meta_content

	cookies

	dynamic_fetcher_type

	dynamic_fetcher_result_type

Detailed Description

Definition at line 19 of file CrawledResource.py.

Constructor & Destructor Documentation

◆ init()

def dc_crawler.CrawledResource.CrawledResource.__init__ ( self )

Definition at line 22 of file CrawledResource.py.

   def __init__(self):
     # rendered unicode content for dynamic fetcher
     self.html_content = ""
     self.binary_content = ""
     self.response_header = ""
     self.html_request = ""
     self.content_type = URL.CONTENT_TYPE_UNDEFINED
     self.charset = ""
     self.error_mask = 0
     self.crawling_time = 0
     self.http_code = 200
     self.bps = 0
     self.last_modified = ""
     self.etag = ""
     self.resource_changed = True
     # before rendered unicode content for dynamic fetcher
     self.meta_content = ""
     self.cookies = {}
     self.dynamic_fetcher_type = None
     self.dynamic_fetcher_result_type = None
 

Member Data Documentation

◆ binary_content

dc_crawler.CrawledResource.CrawledResource.binary_content

Definition at line 25 of file CrawledResource.py.

◆ bps

dc_crawler.CrawledResource.CrawledResource.bps

Definition at line 33 of file CrawledResource.py.

◆ charset

dc_crawler.CrawledResource.CrawledResource.charset

Definition at line 29 of file CrawledResource.py.

◆ content_type

dc_crawler.CrawledResource.CrawledResource.content_type

Definition at line 28 of file CrawledResource.py.

◆ cookies

dc_crawler.CrawledResource.CrawledResource.cookies

Definition at line 39 of file CrawledResource.py.

◆ crawling_time

dc_crawler.CrawledResource.CrawledResource.crawling_time

Definition at line 31 of file CrawledResource.py.

◆ dynamic_fetcher_result_type

dc_crawler.CrawledResource.CrawledResource.dynamic_fetcher_result_type

Definition at line 41 of file CrawledResource.py.

◆ dynamic_fetcher_type

dc_crawler.CrawledResource.CrawledResource.dynamic_fetcher_type

Definition at line 40 of file CrawledResource.py.

◆ error_mask

dc_crawler.CrawledResource.CrawledResource.error_mask

Definition at line 30 of file CrawledResource.py.

◆ etag

dc_crawler.CrawledResource.CrawledResource.etag

Definition at line 35 of file CrawledResource.py.

◆ html_content

dc_crawler.CrawledResource.CrawledResource.html_content

Definition at line 24 of file CrawledResource.py.

◆ html_request

dc_crawler.CrawledResource.CrawledResource.html_request

Definition at line 27 of file CrawledResource.py.

◆ http_code

dc_crawler.CrawledResource.CrawledResource.http_code

Definition at line 32 of file CrawledResource.py.

◆ last_modified

dc_crawler.CrawledResource.CrawledResource.last_modified

Definition at line 34 of file CrawledResource.py.

◆ meta_content

dc_crawler.CrawledResource.CrawledResource.meta_content

Definition at line 38 of file CrawledResource.py.

◆ resource_changed

dc_crawler.CrawledResource.CrawledResource.resource_changed

Definition at line 36 of file CrawledResource.py.

◆ response_header

dc_crawler.CrawledResource.CrawledResource.response_header

Definition at line 26 of file CrawledResource.py.

The documentation for this class was generated from the following file:

sources/hce/dc_crawler/CrawledResource.py

Public Member Functions

Public Attributes