HCE Project Python language Distributed Tasks Manager Application, Distributed Crawler Application and client API bindings.  2.0.0-chaika
Hierarchical Cluster Engine Python language binding
dtma.DTMA.DTMA Class Reference

DTMA Class contents main functional of DTMA application, class inherits from foundation.CementApp. More...

Inheritance diagram for dtma.DTMA.DTMA:
Collaboration diagram for dtma.DTMA.DTMA:

Classes

class  Meta
 

Public Member Functions

def __init__ (self, connectionBuilder=None)
 constructor initialise all class variable and recieve connectionBuilder as param(not mandatory) More...
 
def fillError (self, errorStr, errorCode)
 fillError method calls from error-code point from main processing (...from event handlers) More...
 
def connectionInit (self)
 connectionInit method initializes internal variables that containts network connections/communications More...
 
def generateEmptyResponse (self, task)
 generateEmptyResponse method If here was some critical error, we generate empty response here , instead real response More...
 
def configReader (self)
 configReader method Method try to read config file by prereared paths, return count of readed configs More...
 
def checkAdditionalArgs (self, cmd)
 checkAdditionalArgs method Method checks additions mandatory argument's present More...
 
def transportCommunications (self, cmd, requestObjects)
 taskProcessingDeserialize method Method serializes incoming task to the JSON string More...
 
def loadLogConfigFile (self)
 load logging load logging configuration (log file, log level, filters) More...
 
def setup (self)
 setup method Method calls before run application More...
 
def run (self)
 run method Method contains main application functionality More...
 
def close (self)
 close method Method calls after application run More...
 

Public Attributes

 config
 
 connectionBuilder
 
 localConnection
 
 eventBuilder
 
 dTMAObjectsFiller
 
 errorCode
 
 errorStr
 

Static Public Attributes

 label = CONSTANTS.APP_NAME
 

Detailed Description

DTMA Class contents main functional of DTMA application, class inherits from foundation.CementApp.

Definition at line 36 of file DTMA.py.

Constructor & Destructor Documentation

◆ __init__()

def dtma.DTMA.DTMA.__init__ (   self,
  connectionBuilder = None 
)

constructor initialise all class variable and recieve connectionBuilder as param(not mandatory)

Definition at line 44 of file DTMA.py.

44  def __init__(self, connectionBuilder=None):
45  foundation.CementApp.__init__(self)
46  self.config = ConfigParser.ConfigParser()
47  self.connectionBuilder = connectionBuilder
48  self.localConnection = None
49  self.eventBuilder = EventBuilder()
50  self.dTMAObjectsFiller = DTMAObjectsFiller()
51  self.errorCode = CONSTANTS.ERROR_NOERROR
52  self.errorStr = ""
53 
54 
def __init__(self)
constructor
Definition: UIDGenerator.py:19

Member Function Documentation

◆ checkAdditionalArgs()

def dtma.DTMA.DTMA.checkAdditionalArgs (   self,
  cmd 
)

checkAdditionalArgs method Method checks additions mandatory argument's present

Definition at line 122 of file DTMA.py.

122  def checkAdditionalArgs(self, cmd):
123  ret = True
124  if cmd in CONSTANTS.TASKS[1:3]:
125  if self.pargs.fields is None or self.pargs.classes is None:
126  self.fillError(CONSTANTS.ERROR_STR2, CONSTANTS.ERROR_ARGS2)
127  ret = False
128  elif cmd == CONSTANTS.TASKS[0] or cmd == CONSTANTS.TASKS[3]:
129  if self.pargs.classes is None:
130  optionName = [CONSTANTS.SERVER_CONFIG_OPTION_NAME, CONSTANTS.SERVER_CONFIG_OPTION_NAME2]\
131  [cmd == CONSTANTS.TASKS[3]]
132  try:
133  self.pargs.classes = str(self.config.get(CONSTANTS.SERVER_CONFIG_SECTION_NAME, optionName))
134  except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
135  self.fillError(CONSTANTS.ERROR_STR3.format(CONSTANTS.SERVER_CONFIG_SECTION_NAME, optionName),
136  CONSTANTS.ERROR_CONFIG_SECTION)
137  ret = False
138  return ret
139 
140 
Here is the call graph for this function:
Here is the caller graph for this function:

◆ close()

def dtma.DTMA.DTMA.close (   self)

close method Method calls after application run

Definition at line 259 of file DTMA.py.

259  def close(self):
260  foundation.CementApp.close(self)
261 

◆ configReader()

def dtma.DTMA.DTMA.configReader (   self)

configReader method Method try to read config file by prereared paths, return count of readed configs

Definition at line 109 of file DTMA.py.

109  def configReader(self):
110  configReadList = []
111  if self.pargs.config is None:
112  configReadList = self.config.read(CONSTANTS.DEFAULT_CONFIG_NAME1)
113  if len(configReadList) == 0:
114  configReadList = self.config.read(CONSTANTS.DEFAULT_CONFIG_NAME2)
115  else:
116  configReadList = self.config.read(self.pargs.config)
117  return len(configReadList)
118 
119 
Here is the caller graph for this function:

◆ connectionInit()

def dtma.DTMA.DTMA.connectionInit (   self)

connectionInit method initializes internal variables that containts network connections/communications

Definition at line 65 of file DTMA.py.

65  def connectionInit(self):
66  if self.connectionBuilder is None:
67  self.connectionBuilder = ConnectionBuilderLight()
68  host = str(self.config.get(CONSTANTS.APP_NAME, CONSTANTS.DTM_HOST))
69  if len(host) > 1 and host[0] == "\"":
70  host = host[1:]
71  if host[-1] == "\"":
72  host = host[0:-1]
73  port = str(self.config.get(CONSTANTS.APP_NAME, CONSTANTS.DTM_PORT))
74  addr = host + ":" + port
75  self.localConnection = self.connectionBuilder.build(transport.Consts.CLIENT_CONNECT, addr,
76  transport.Consts.TCP_TYPE)
77 
78 
Here is the caller graph for this function:

◆ fillError()

def dtma.DTMA.DTMA.fillError (   self,
  errorStr,
  errorCode 
)

fillError method calls from error-code point from main processing (...from event handlers)

Definition at line 57 of file DTMA.py.

57  def fillError(self, errorStr, errorCode):
58  self.errorCode = errorCode
59  self.errorStr = errorStr
60  logger.error(self.errorStr)
61 
62 
Here is the caller graph for this function:

◆ generateEmptyResponse()

def dtma.DTMA.DTMA.generateEmptyResponse (   self,
  task 
)

generateEmptyResponse method If here was some critical error, we generate empty response here , instead real response

Definition at line 81 of file DTMA.py.

81  def generateEmptyResponse(self, task):
82  ret = []
83  obj = None
84  if task is None:
86  elif task == CONSTANTS.TASKS[0]:
88  elif task == CONSTANTS.TASKS[1]:
90  elif task == CONSTANTS.TASKS[2]:
92  elif task == CONSTANTS.TASKS[3]:
94  elif task == CONSTANTS.TASKS[4]:
96  elif task == CONSTANTS.TASKS[5]:
98  elif task == CONSTANTS.TASKS[6]:
99  obj = dtm.EventObjects.CustomResponse(None, None, None)
100 
101  obj.errorCode = self.errorCode
102  obj.errorMessage = self.errorStr
103  ret.append(obj)
104  return json.dumps([ret], default=lambda o: o.__dict__, sort_keys=True, indent=4)
105 
106 
GeneralResponse event object, represents general state response for multipurpose usage.
AdminConfigVars event object, for admin set or get config variables from any of threaded classes or a...
AdminStatData event object, for admin fetch stat fields and possible data from any threaded classes i...
Here is the caller graph for this function:

◆ loadLogConfigFile()

def dtma.DTMA.DTMA.loadLogConfigFile (   self)

load logging load logging configuration (log file, log level, filters)

Definition at line 194 of file DTMA.py.

194  def loadLogConfigFile(self):
195  global logger # pylint: disable=W0603
196  try:
197  logIniFileName = self.config.get(CONSTANTS.LOG_CONFIG_SECTION_NAME, CONSTANTS.LOG_CONFIG_OPTION_NAME)
198  if logIniFileName != None:
199  logging.config.fileConfig(logIniFileName, disable_existing_loggers=False)
200  except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
201  self.errorStr = CONSTANTS.ERROR_STR8
202  self.errorCode = CONSTANTS.ERROR_LOG_SECTION_ERROR
203  except Exception, err:
204  self.errorStr = CONSTANTS.ERROR_STR10 + ': ' + str(err)
205  self.errorCode = CONSTANTS.ERROR_LOG_INIT
206  logger = Utils.MPLogger().getLogger()
207 
208 
209 
Here is the call graph for this function:
Here is the caller graph for this function:

◆ run()

def dtma.DTMA.DTMA.run (   self)

run method Method contains main application functionality

Definition at line 218 of file DTMA.py.

218  def run(self):
219  foundation.CementApp.run(self)
220  isHelpArg = False
221  if '-h' in self.argv or '--help' in self.argv:
222  isHelpArg = True
223  requestObjects = None
224  jsonBuf = None
225  if self.configReader() > 0:
226  self.loadLogConfigFile()
227 
228  if self.errorCode == CONSTANTS.ERROR_NOERROR:
229  try:
230  self.connectionInit()
231  except (ConfigParser.NoSectionError, ConfigParser.NoOptionError):
232  self.fillError(CONSTANTS.ERROR_STR3, CONSTANTS.ERROR_CONFIG_SECTION)
233  if not isHelpArg and self.pargs.cmd is None:
234  self.fillError(CONSTANTS.ERROR_STR1, CONSTANTS.ERROR_ARGS1)
235  elif self.pargs.cmd is not None and self.pargs.cmd in CONSTANTS.TASKS and \
236  self.checkAdditionalArgs(self.pargs.cmd):
237  try:
238  requestObjects = self.dTMAObjectsFiller.generateObjectsList(self.pargs.cmd, self.pargs.fields,
239  self.pargs.classes)
240  responseObjects = self.transportCommunications(self.pargs.cmd, requestObjects)
241  jsonBuf = json.dumps(responseObjects, default=lambda o: o.__dict__, sort_keys=True, indent=4)
242  except (DTMAExceptions.DTMAEmptyFields, DTMAExceptions.DTMANameValueException):
243  self.fillError(CONSTANTS.ERROR_STR4, CONSTANTS.ERROR_FIELDS_ARG)
244  except DTMAExceptions.DTMAEmptyClasses:
245  self.fillError(CONSTANTS.ERROR_STR5, CONSTANTS.ERROR_CLASSES_ARG)
246  else:
247  self.errorStr = CONSTANTS.ERROR_STR9
248  self.errorCode = CONSTANTS.ERROR_NO_CONFIG
249 
250  if jsonBuf is None:
251  jsonBuf = self.generateEmptyResponse(self.pargs.cmd)
252  sys.stdout.write(jsonBuf)
253  # Finish logging
254  logger.info(APP_CONSTS.LOGGER_DELIMITER_LINE)
255 
256 
Here is the call graph for this function:

◆ setup()

def dtma.DTMA.DTMA.setup (   self)

setup method Method calls before run application

Definition at line 212 of file DTMA.py.

212  def setup(self):
213  foundation.CementApp.setup(self)
214 
215 

◆ transportCommunications()

def dtma.DTMA.DTMA.transportCommunications (   self,
  cmd,
  requestObjects 
)

taskProcessingDeserialize method Method serializes incoming task to the JSON string

Definition at line 143 of file DTMA.py.

143  def transportCommunications(self, cmd, requestObjects):
144  responses = []
145  retEvent = None
146  eventType = None
147  timeout = None
148  responseTuple = []
149  emptyResponse = None
150  try:
151  timeout = self.config.get(CONSTANTS.APP_NAME, CONSTANTS.TCP_TIMEOUT_CONFIG_NAME)
152  except ConfigParser.NoSectionError:
153  timeout = CONSTANTS.TCP_TIMEOUT
154  except ConfigParser.NoOptionError:
155  timeout = CONSTANTS.TCP_TIMEOUT
156 
157  for requestObject in requestObjects:
158  if cmd == CONSTANTS.TASKS[0]:
159  eventType = EVENT_TYPES.ADMIN_FETCH_STAT_DATA
160  emptyResponse = dtm.EventObjects.AdminStatData(requestObject.className)
161  elif cmd == CONSTANTS.TASKS[1]:
162  eventType = EVENT_TYPES.ADMIN_SET_CONFIG_VARS
163  emptyResponse = dtm.EventObjects.AdminConfigVars(requestObject.className)
164  elif cmd == CONSTANTS.TASKS[2]:
165  eventType = EVENT_TYPES.ADMIN_GET_CONFIG_VARS
166  emptyResponse = dtm.EventObjects.AdminConfigVars(requestObject.className)
167  elif cmd == CONSTANTS.TASKS[3]:
168  eventType = EVENT_TYPES.ADMIN_STATE
169  emptyResponse = dtm.EventObjects.GeneralResponse()
170  elif cmd == CONSTANTS.TASKS[4]:
171  eventType = EVENT_TYPES.ADMIN_SUSPEND
172  emptyResponse = dtm.EventObjects.GeneralResponse()
173  elif cmd == CONSTANTS.TASKS[5]:
174  eventType = EVENT_TYPES.ADMIN_SYSTEM
175  emptyResponse = dtm.EventObjects.GeneralResponse()
176  elif cmd == CONSTANTS.TASKS[6]:
177  eventType = EVENT_TYPES.ADMIN_SQL_CUSTOM
178  emptyResponse = dtm.EventObjects.CustomResponse(None, None, None)
179 
180  event = self.eventBuilder.build(eventType, requestObject)
181  self.localConnection.send(event)
182  if self.localConnection.poll(timeout) == 0:
183  responseTuple = [emptyResponse, CONSTANTS.ERROR_NETWORK]
184  else:
185  retEvent = self.localConnection.recv()
186  responseTuple = [retEvent.eventObj, CONSTANTS.ERROR_NOERROR]
187  responses.append(responseTuple)
188  return responses
189 
190 
GeneralResponse event object, represents general state response for multipurpose usage.
AdminConfigVars event object, for admin set or get config variables from any of threaded classes or a...
AdminStatData event object, for admin fetch stat fields and possible data from any threaded classes i...
Here is the caller graph for this function:

Member Data Documentation

◆ config

dtma.DTMA.DTMA.config

Definition at line 46 of file DTMA.py.

◆ connectionBuilder

dtma.DTMA.DTMA.connectionBuilder

Definition at line 47 of file DTMA.py.

◆ dTMAObjectsFiller

dtma.DTMA.DTMA.dTMAObjectsFiller

Definition at line 50 of file DTMA.py.

◆ errorCode

dtma.DTMA.DTMA.errorCode

Definition at line 51 of file DTMA.py.

◆ errorStr

dtma.DTMA.DTMA.errorStr

Definition at line 52 of file DTMA.py.

◆ eventBuilder

dtma.DTMA.DTMA.eventBuilder

Definition at line 49 of file DTMA.py.

◆ label

dtma.DTMA.DTMA.label = CONSTANTS.APP_NAME
static

Definition at line 39 of file DTMA.py.

◆ localConnection

dtma.DTMA.DTMA.localConnection

Definition at line 48 of file DTMA.py.


The documentation for this class was generated from the following file: