HCE Project Python language Distributed Tasks Manager Application, Distributed Crawler Application and client API bindings.  2.0.0-chaika
Hierarchical Cluster Engine Python language binding
dtm.ResourcesManager.ResourcesManager Class Reference

Class contents ResourcesManager module implementation. More...

Inheritance diagram for dtm.ResourcesManager.ResourcesManager:
Collaboration diagram for dtm.ResourcesManager.ResourcesManager:

Public Member Functions

def __init__ (self, config, connectionBuilder=None)
 constructor initialise all class variable and recieve config as param More...
 
def connectionInit (self)
 connectionInit method initializes internal inproc connection More...
 
def dbInit (self)
 dbInit method initializes internal database API More...
 
def badEventType (self, msg, event)
 eventProcessing method Method contains main error processing for incoming events msg - incomig error message event - incoming event More...
 
def onUpdateResourcesData (self, event)
 onUpdateResourcesData event handler method Processing UPDATE_RESOURCES_DATA incoming request event - incoming event More...
 
def onGetAVGResources (self, event)
 onGetAVGResources event handler method Processing GET_AVG_RESOURCES incoming request event - incoming event More...
 
- Public Member Functions inherited from app.BaseServerManager.BaseServerManager
def __init__ (self, poller_manager=None, admin_connection=None, conectionLightBuilder=None, exceptionForward=False, dumpStatVars=True)
 constructor More...
 
def addConnection (self, name, connection)
 
def setEventHandler (self, eventType, handler)
 set event handler rewrite the current handler for eventType More...
 
def send (self, connect_name, event)
 send event More...
 
def reply (self, event, reply_event)
 wrapper for sending event in reply for event More...
 
def poll (self)
 poll function polling connections receive as multipart msg, the second argument is pickled pyobj More...
 
def process (self, event)
 process event call the event handler method that was set by user or on_unhandled_event method if not set More...
 
def run (self)
 
def is_connection_registered (self, name)
 check is a connection was registered in a instance of BaseServerManager i object More...
 
def on_poll_timeout (self)
 function will call every time when ConnectionTimeout exception arrive More...
 
def on_unhandled_event (self, event)
 function will call every time when arrive doesn't set handler for event type of event.evenType More...
 
def build_poller_list (self)
 
def clear_poller (self)
 
def onAdminState (self, event)
 onAdminState event handler process admin SHUTDOWN command More...
 
def onAdminFetchStatData (self, event)
 onAdminState event handler process admin command More...
 
def onAdminSuspend (self, event)
 onAdminState event handler process admin command More...
 
def getStatDataFields (self, fields)
 getStatDataFields returns stat data from storage More...
 
def getSystemStat (self)
 getSystemStat returns stat data for system indicators: RAMV, RAMR and CPU More...
 
def getConfigVarsFields (self, fields)
 getConfigVarsFields returns config vars from storage More...
 
def onAdminGetConfigVars (self, event)
 onAdminGetConfigVars event handler process getConfigVars admin command, fill and return config vars array from internal storage More...
 
def onAdminSetConfigVars (self, event)
 onAdminSetConfigVars event handler process setConfigVars admin command More...
 
def setConfigVars (self, setConfigVars)
 processSetConfigVars sets config vars in storage More...
 
def sendAdminReadyEvent (self)
 send ready event to notify adminInterfaceService More...
 
def createLogMsg (self, event)
 from string message from event object More...
 
def initStatFields (self, connect_name)
 add record in statFields More...
 
def updateStatField (self, field_name, value, operation=STAT_FIELDS_OPERATION_ADD)
 update values of stat field - default sum More...
 
def processSpecialConfigVars (self, name, value)
 send ready event to notify adminInterfaceService More...
 
def getLogLevel (self)
 Get log level from first of existing loggers. More...
 
def setLogLevel (self, level)
 Set log level for all loggers. More...
 
def saveStatVarsDump (self)
 Save stat vars in json file. More...
 
def loadStatVarsDump (self)
 Load stat vars in json file. More...
 
def getStatVarsDumpFileName (self)
 Get stat vars file name. More...
 
def createDBIDict (self, configParser)
 

Public Attributes

 moduleName
 
 servIndex
 
 eventBuilder
 
 connectionBuilder
 
 config
 
 dbi
 
 resourcesRecalculating
 
- Public Attributes inherited from app.BaseServerManager.BaseServerManager
 dumpStatVars
 
 poller_manager
 
 eventBuilder
 
 exit_flag
 
 pollTimeout
 
 connections
 
 event_handlers
 
 statFields
 stat fields container More...
 
 configVars
 
 exceptionForward
 

Static Public Attributes

string RESOURCE_MANAGER_SERV_NAME = "Server"
 
string RESOURCE_MANAGER_SERV_CONFIG_NAME = "ServerName"
 
- Static Public Attributes inherited from app.BaseServerManager.BaseServerManager
string ADMIN_CONNECT_ENDPOINT = "Admin"
 
string ADMIN_CONNECT_CLIENT = "Admin"
 
int POLL_TIMEOUT_DEFAULT = 3000
 
int STAT_FIELDS_OPERATION_ADD = 0
 
int STAT_FIELDS_OPERATION_SUB = 1
 
int STAT_FIELDS_OPERATION_SET = 2
 
int STAT_FIELDS_OPERATION_INIT = 3
 
string POLL_TIMEOUT_CONFIG_VAR_NAME = "POLL_TIMEOUT"
 
string LOG_LEVEL_CONFIG_VAR_NAME = "LOG_LEVEL"
 
string STAT_DUMPS_DEFAULT_DIR = "/tmp/"
 
string STAT_DUMPS_DEFAULT_NAME = "%APP_NAME%_%CLASS_NAME%_stat_vars.dump"
 
dictionary LOGGERS_NAMES = {APP_CONSTS.LOGGER_NAME, "dc", "dtm", "root", ""}
 

Detailed Description

Class contents ResourcesManager module implementation.

Definition at line 33 of file ResourcesManager.py.

Constructor & Destructor Documentation

◆ __init__()

def dtm.ResourcesManager.ResourcesManager.__init__ (   self,
  config,
  connectionBuilder = None 
)

constructor initialise all class variable and recieve config as param

Definition at line 41 of file ResourcesManager.py.

41  def __init__(self, config, connectionBuilder=None):
42  BaseServerManager.__init__(self)
43  self.moduleName = self.__class__.__name__
44  self.servIndex = 1
45  self.eventBuilder = EventBuilder()
46  self.setEventHandler(EVENT.UPDATE_RESOURCES_DATA, self.onUpdateResourcesData)
47  self.setEventHandler(EVENT.GET_AVG_RESOURCES, self.onGetAVGResources)
48  self.connectionBuilder = connectionBuilder
49  self.config = config
50  self.dbi = None
51  self.connectionInit()
52  self.dbInit()
53  self.resourcesRecalculating = ResourcesRecalculating()
54 
55 
def __init__(self)
constructor
Definition: UIDGenerator.py:19

Member Function Documentation

◆ badEventType()

def dtm.ResourcesManager.ResourcesManager.badEventType (   self,
  msg,
  event 
)

eventProcessing method Method contains main error processing for incoming events msg - incomig error message event - incoming event

Definition at line 86 of file ResourcesManager.py.

86  def badEventType(self, msg, event):
87  errorStr = msg + str(event.eventType)
88  logger.error(LogFormatterEvent(event, [], errorStr))
89 # raise Exception(errorStr)
90 
91 
Here is the caller graph for this function:

◆ connectionInit()

def dtm.ResourcesManager.ResourcesManager.connectionInit (   self)

connectionInit method initializes internal inproc connection

Definition at line 58 of file ResourcesManager.py.

58  def connectionInit(self):
59  if self.connectionBuilder == None:
60  self.connectionBuilder = ConnectionBuilderLight()
61  try:
62  localConnection = self.connectionBuilder.build(transport.Consts.SERVER_CONNECT, \
63  self.config.get(self.moduleName, self.RESOURCE_MANAGER_SERV_CONFIG_NAME))
64  self.addConnection(self.RESOURCE_MANAGER_SERV_NAME + str(self.servIndex), localConnection)
65  self.servIndex = self.servIndex + 1
66  except ConfigParser.NoSectionError:
67  logger.error(">>> ResourcesManager can't read config - Section Error")
68  except ConfigParser.NoOptionError:
69  logger.error(">>> ResourcesManager can't read config - Option Error")
70 
71 
Here is the call graph for this function:
Here is the caller graph for this function:

◆ dbInit()

def dtm.ResourcesManager.ResourcesManager.dbInit (   self)

dbInit method initializes internal database API

Definition at line 74 of file ResourcesManager.py.

74  def dbInit(self):
75  if self.config != None:
76  dic = None
77  dic = dict(self.config.items(CONSTANTS.DB_CONFIG_SECTION))
78  if dic != None:
79  self.dbi = DBI(dic)
80 
81 

◆ onGetAVGResources()

def dtm.ResourcesManager.ResourcesManager.onGetAVGResources (   self,
  event 
)

onGetAVGResources event handler method Processing GET_AVG_RESOURCES incoming request event - incoming event

Definition at line 124 of file ResourcesManager.py.

124  def onGetAVGResources(self, event):
125  logger.debug(LogFormatterEvent(event, [], ">>> ResourcesManager [GET_AVG_RESOURCES] Handler start"))
126  if event.eventType != EVENT.GET_AVG_RESOURCES:
127  self.badEventType(">>> Wrong Event type [GET_AVG_RESOURCES] != ", event)
128  resourcesAVG = self.resourcesRecalculating.getResourcesAVG()
129  retEventType = EVENT.GET_AVG_RESOURCES_RESPONSE
130  retEvent = self.eventBuilder.build(retEventType, resourcesAVG)
131  self.reply(event, retEvent)
132  logger.debug(LogFormatterEvent(event, [], ">>> ResourcesManager [GET_AVG_RESOURCES] Handler finish"))
133 
Here is the call graph for this function:

◆ onUpdateResourcesData()

def dtm.ResourcesManager.ResourcesManager.onUpdateResourcesData (   self,
  event 
)

onUpdateResourcesData event handler method Processing UPDATE_RESOURCES_DATA incoming request event - incoming event

Definition at line 95 of file ResourcesManager.py.

95  def onUpdateResourcesData(self, event):
96  logger.debug(LogFormatterEvent(event, [], ">>> ResourcesManager [UPDATE_RESOURCES_DATA] Handler start"))
97  if event.eventType != EVENT.UPDATE_RESOURCES_DATA:
98  self.badEventType(">>> Wrong Event type [UPDATE_RESOURCES_DATA] != ", event)
99  response = GeneralResponse()
100  wasUpdate = False
101  for updateItem in event.eventObj:
102  data = ResourcesTable(updateItem)
103  try:
104  self.dbi.insertOnUpdate(data, "id=%s" % data.nodeId)
105  except DBIErr as ex:
106  response.statuses.append(False)
107  errorStr = ">>> Some DBI error in ResourcesManager.onUpdateResourcesData [" + str(ex) + "]"
108  logger.error(LogFormatterEvent(event, [], errorStr))
109  else:
110  self.resourcesRecalculating.addUpdateResources(data)
111  wasUpdate = True
112  response.statuses.append(True)
113  if wasUpdate:
114  self.resourcesRecalculating.recalculate()
115  retEventType = EVENT.UPDATE_RESOURCES_DATA_RESPONSE
116  retEvent = self.eventBuilder.build(retEventType, response)
117  self.reply(event, retEvent)
118  logger.debug(LogFormatterEvent(event, [], ">>> ResourcesManager [UPDATE_RESOURCES_DATA] Handler finish"))
119 
120 
Here is the call graph for this function:

Member Data Documentation

◆ config

dtm.ResourcesManager.ResourcesManager.config

Definition at line 49 of file ResourcesManager.py.

◆ connectionBuilder

dtm.ResourcesManager.ResourcesManager.connectionBuilder

Definition at line 48 of file ResourcesManager.py.

◆ dbi

dtm.ResourcesManager.ResourcesManager.dbi

Definition at line 50 of file ResourcesManager.py.

◆ eventBuilder

dtm.ResourcesManager.ResourcesManager.eventBuilder

Definition at line 45 of file ResourcesManager.py.

◆ moduleName

dtm.ResourcesManager.ResourcesManager.moduleName

Definition at line 43 of file ResourcesManager.py.

◆ RESOURCE_MANAGER_SERV_CONFIG_NAME

string dtm.ResourcesManager.ResourcesManager.RESOURCE_MANAGER_SERV_CONFIG_NAME = "ServerName"
static

Definition at line 36 of file ResourcesManager.py.

◆ RESOURCE_MANAGER_SERV_NAME

string dtm.ResourcesManager.ResourcesManager.RESOURCE_MANAGER_SERV_NAME = "Server"
static

Definition at line 35 of file ResourcesManager.py.

◆ resourcesRecalculating

dtm.ResourcesManager.ResourcesManager.resourcesRecalculating

Definition at line 53 of file ResourcesManager.py.

◆ servIndex

dtm.ResourcesManager.ResourcesManager.servIndex

Definition at line 44 of file ResourcesManager.py.


The documentation for this class was generated from the following file: