dc_db.URLPutTask.URLPutTask Class Reference

Inheritance diagram for dc_db.URLPutTask.URLPutTask:

Collaboration diagram for dc_db.URLPutTask.URLPutTask:

Public Member Functions
def	__init__ (self, keyValueStorageDir, rawDataDir, dBDataTask)
def	process (self, urlPuts, queryCallback)
Public Member Functions inherited from dc_db.BaseTask.BaseTask
def	isSiteExist (self, siteId, queryCallback, userId=None)
def	generateCriterionSQL (self, criterions, additionWhere=None, siteId=None)
def	fetchByCriterions (self, criterions, queryCallback)
def	dbLock (self, mutexName, queryCallback, sleepTime=1, mutexLockTTL=Constants.DEFAULT_LOCK_TTL)
def	dbUnlock (self, mutexName, queryCallback)
def	createUrlsInsertQuery (self, siteId, localKeys, localValues)
def	copyUrlsToDcUrls (self, siteId, queryCallback)
def	statisticLogUpdate (self, localObj, urlMd5, siteId, status, queryCallback, isInsert=False)
def	calculateMd5FormUrl (self, url, urlType, useNormilize=False)

Public Attributes
	uRLCleanUpTask
	dBDataTask

Additional Inherited Members
Static Public Member Functions inherited from dc_db.BaseTask.BaseTask
def	readValueFromSiteProp (siteId, propName, queryCallback, urlMd5=None)

Detailed Description

Definition at line 22 of file URLPutTask.py.

Constructor & Destructor Documentation

__init__()

def dc_db.URLPutTask.URLPutTask.__init__	(		self,
			keyValueStorageDir,
			rawDataDir,
			dBDataTask
	)

Definition at line 28 of file URLPutTask.py.

  def __init__(self, keyValueStorageDir, rawDataDir, dBDataTask):
    super(URLPutTask, self).__init__()
    self.uRLCleanUpTask = URLCleanUpTask(keyValueStorageDir, rawDataDir, dBDataTask)
    self.dBDataTask = dBDataTask

Member Function Documentation

process()

def dc_db.URLPutTask.URLPutTask.process	(		self,
			urlPuts,
			queryCallback
	)

Definition at line 39 of file URLPutTask.py.

  def process(self, urlPuts, queryCallback):  # pylint: disable=W0613
    urlPutResponses = []
    for urlPut in urlPuts:
      urlsCount = 0
      localMd5s = []
      if urlPut.urlMd5 is None:
        logger.debug(">>> urlPuts.urlMd5 is None, fetch by criterions")
        localMd5s = self.uRLCleanUpTask.extractUrlByCriterions(urlPuts.siteId, False, urlPuts.criterions,
                                                               queryCallback, Constants.SECONDARY_DB_ID)
      else:
        localMd5s.append(urlPut.urlMd5)
      logger.debug(">>> [URL_PUT] localUrls size = " + str(len(localMd5s)))

      if "data" in urlPut.putDict and urlPut.contentType != dc.EventObjects.Content.CONTENT_PROCESSOR_CONTENT:
        try:
          urlPut.putDict["data"] = base64.b64decode(urlPut.putDict["data"])
        except TypeError:
          pass
      for localMd5 in localMd5s:
        try:
          urlPut.urlMd5 = localMd5
          urlPutResponses.append(self.dBDataTask.process(urlPut, queryCallback))
          urlsCount = urlsCount + 1
        except Exception as err:
          ExceptionLog.handler(logger, err, ">>> [URL_PUT] Exception:")
          logger.debug(">>> [URL_PUT] Some Type Exception [LOOP] = " + str(type(err)))

    return urlPutResponses

Member Data Documentation

dBDataTask

dc_db.URLPutTask.URLPutTask.dBDataTask

Definition at line 31 of file URLPutTask.py.

uRLCleanUpTask

dc_db.URLPutTask.URLPutTask.uRLCleanUpTask

Definition at line 30 of file URLPutTask.py.

---

The documentation for this class was generated from the following file:

• sources/hce/dc_db/URLPutTask.py