dc_processor.alchemy_extractor.AlchemyExtractor Class Reference

Inheritance diagram for dc_processor.alchemy_extractor.AlchemyExtractor:

Collaboration diagram for dc_processor.alchemy_extractor.AlchemyExtractor:

Public Member Functions
def	__init__ (self, config, templ=None, domain=None, processorProperties=None)
def	extractTags (self, resource, reslt)
Public Member Functions inherited from dc_processor.base_extractor.BaseExtractor
def	__init__ (self, config, templ=None, domain=None, processorProperties=None)
def	__str__ (self)
def	__repr__ (self)
def	loadScraperProperties (self, scraperPropFileName)
def	isTagNotFilled (self, result, tagName)
def	isTagValueNotEmpty (self, tagValue)
def	tagValueElemValidate (self, tagValueElem, conditionElem)
def	tagValueValidate (self, tagName, tagValue)
def	addTag (self, result, tag_name, tag_value, xpath="", isDefaultTag=False, callAdjustment=True, tagType=None, allowNotFilled=False)
def	calculateMetrics (self, response)
def	rankReading (self, exctractorName)

Public Attributes
	name
Public Attributes inherited from dc_processor.base_extractor.BaseExtractor
	config
	processorProperties
	name
	rank
	process_mode
	modules
	data
	db_dc_scraper_db
	DBConnector
	imgDelimiter
	tagsValidator

Additional Inherited Members
Static Public Attributes inherited from dc_processor.base_extractor.BaseExtractor
	properties = None
dictionary	tag
dictionary	tagsMask

Detailed Description

Definition at line 21 of file alchemy_extractor.py.

Constructor & Destructor Documentation

__init__()

def dc_processor.alchemy_extractor.AlchemyExtractor.__init__	(		self,
			config,
			templ = None,
			domain = None,
			processorProperties = None
	)

Definition at line 24 of file alchemy_extractor.py.

  def __init__(self, config, templ=None, domain=None, processorProperties=None):
    BaseExtractor.__init__(self, config, templ, domain, processorProperties)
    self.name = CONSTS.EXTRACTOR_NAME_ALCHEMY
    self.data["extractor"] = CONSTS.EXTRACTOR_NAME_ALCHEMY
    logger.debug("Properties: %s", varDump(self.properties))

    # set module rank from module's properties
    self.rankReading(self.__class__.__name__)

Member Function Documentation

extractTags()

def dc_processor.alchemy_extractor.AlchemyExtractor.extractTags	(		self,
			resource,
			reslt
	)

Definition at line 34 of file alchemy_extractor.py.

  def extractTags(self, resource, reslt):
    try:
      logger.info("AAAAAAA")
      parser = AlchemyAPI()
      logger.info("BBBBBBB")
      text = parser.text("html", resource.raw_html)
      logger.info("CCCCCCC")
      logger.info("Article's corpus: %s", text)
      self.addTag(result=reslt, \
                    tag_name=CONSTS.TAG_CONTENT_UTF8_ENCODED, \
                    tag_value=text)
      logger.info("DDDDDDD")
    except Exception, err:
      logger.info(varDump(err))
    return reslt

Member Data Documentation

name

dc_processor.alchemy_extractor.AlchemyExtractor.name

Definition at line 26 of file alchemy_extractor.py.

---

The documentation for this class was generated from the following file:

• sources/hce/dc_processor/alchemy_extractor.py