HCE Project Python language Distributed Tasks Manager Application, Distributed Crawler Application and client API bindings.
2.0.0-chaika
Hierarchical Cluster Engine Python language binding
|
The TasksStateUpdateService class, is a listener of tasks state updates from DRCE FO of cluster nodes. More...
Public Member Functions | |
def | __init__ (self, configParser, connectionBuilderLight=None, connectionBuilder=None) |
constructor initialize fields More... | |
def | onUpdateTaskFieldsResponse (self, event) |
onUpdateTaskFieldsResponse event handler More... | |
def | onTCPServerRequest (self, event) |
onTCPServerRequest handler of TCP server requests. More... | |
def | onReceiveAllTaskIds (self, event) |
onReceiveAllTaskIds handler for receive running tasks from TasksManager More... | |
def | on_poll_timeout (self) |
on_poll_timeout handler, now just send CheckState to EEManager More... | |
def | tryCheckTasksState (self) |
send CheckState message to EEManager if don't have cached task id, then fetch from TasksManager More... | |
def | processUpdateTaskFields (self, taskResponse) |
Send UpdateTasksData event to the TasksManager. More... | |
Public Member Functions inherited from app.BaseServerManager.BaseServerManager | |
def | __init__ (self, poller_manager=None, admin_connection=None, conectionLightBuilder=None, exceptionForward=False, dumpStatVars=True) |
constructor More... | |
def | addConnection (self, name, connection) |
def | setEventHandler (self, eventType, handler) |
set event handler rewrite the current handler for eventType More... | |
def | send (self, connect_name, event) |
send event More... | |
def | reply (self, event, reply_event) |
wrapper for sending event in reply for event More... | |
def | poll (self) |
poll function polling connections receive as multipart msg, the second argument is pickled pyobj More... | |
def | process (self, event) |
process event call the event handler method that was set by user or on_unhandled_event method if not set More... | |
def | run (self) |
def | is_connection_registered (self, name) |
check is a connection was registered in a instance of BaseServerManager i object More... | |
def | on_poll_timeout (self) |
function will call every time when ConnectionTimeout exception arrive More... | |
def | on_unhandled_event (self, event) |
function will call every time when arrive doesn't set handler for event type of event.evenType More... | |
def | build_poller_list (self) |
def | clear_poller (self) |
def | onAdminState (self, event) |
onAdminState event handler process admin SHUTDOWN command More... | |
def | onAdminFetchStatData (self, event) |
onAdminState event handler process admin command More... | |
def | onAdminSuspend (self, event) |
onAdminState event handler process admin command More... | |
def | getStatDataFields (self, fields) |
getStatDataFields returns stat data from storage More... | |
def | getSystemStat (self) |
getSystemStat returns stat data for system indicators: RAMV, RAMR and CPU More... | |
def | getConfigVarsFields (self, fields) |
getConfigVarsFields returns config vars from storage More... | |
def | onAdminGetConfigVars (self, event) |
onAdminGetConfigVars event handler process getConfigVars admin command, fill and return config vars array from internal storage More... | |
def | onAdminSetConfigVars (self, event) |
onAdminSetConfigVars event handler process setConfigVars admin command More... | |
def | setConfigVars (self, setConfigVars) |
processSetConfigVars sets config vars in storage More... | |
def | sendAdminReadyEvent (self) |
send ready event to notify adminInterfaceService More... | |
def | createLogMsg (self, event) |
from string message from event object More... | |
def | initStatFields (self, connect_name) |
add record in statFields More... | |
def | updateStatField (self, field_name, value, operation=STAT_FIELDS_OPERATION_ADD) |
update values of stat field - default sum More... | |
def | processSpecialConfigVars (self, name, value) |
send ready event to notify adminInterfaceService More... | |
def | getLogLevel (self) |
Get log level from first of existing loggers. More... | |
def | setLogLevel (self, level) |
Set log level for all loggers. More... | |
def | saveStatVarsDump (self) |
Save stat vars in json file. More... | |
def | loadStatVarsDump (self) |
Load stat vars in json file. More... | |
def | getStatVarsDumpFileName (self) |
Get stat vars file name. More... | |
def | createDBIDict (self, configParser) |
Public Attributes | |
expect_response | |
clientTasksManagerName | |
serverHost | |
serverPort | |
clientExecutionEnvironmentManager | |
drceIdGenerator | |
drceCommandConvertor | |
lastCheckStateTs | |
timestamp of last send CheckState request More... | |
checkStateInterval | |
checkStateNum | |
fetchTaskNum | |
taskIdsForCheckState | |
list of task ids for check state More... | |
Public Attributes inherited from app.BaseServerManager.BaseServerManager | |
dumpStatVars | |
poller_manager | |
eventBuilder | |
exit_flag | |
pollTimeout | |
connections | |
event_handlers | |
statFields | |
stat fields container More... | |
configVars | |
exceptionForward | |
Static Public Attributes | |
string | CONFIG_SERVER_HOST = "serverHost" |
string | CONFIG_SERVER_PORT = "serverPort" |
string | CONFIG_TASKS_MANAGER_CLIENT = "clientTasksManager" |
string | CONFIG_EE_MANAGER = "clientExecutionEnvironmentManager" |
string | CONFIG_CHECK_STATE_NUM = "checkStateNum" |
string | CONFIG_CHECK_STATE_INTERVAL = "checkStateInterval" |
string | CONFIG_FETCH_TASK_NUM = "checkStateTasks" |
string | ERROR_HCE_RESPONSE_PROCESSING_EXCEPTION = "Update request error, possible wrong json format!" |
string | ERROR_TASK_FIELDS_UPDATE = "Update of task fields response error." |
int | UPDATE_TYPE_TASK_STATE = 100 |
int | UPDATE_TYPE_RESOURCES_STATE = 101 |
Static Public Attributes inherited from app.BaseServerManager.BaseServerManager | |
string | ADMIN_CONNECT_ENDPOINT = "Admin" |
string | ADMIN_CONNECT_CLIENT = "Admin" |
int | POLL_TIMEOUT_DEFAULT = 3000 |
int | STAT_FIELDS_OPERATION_ADD = 0 |
int | STAT_FIELDS_OPERATION_SUB = 1 |
int | STAT_FIELDS_OPERATION_SET = 2 |
int | STAT_FIELDS_OPERATION_INIT = 3 |
string | POLL_TIMEOUT_CONFIG_VAR_NAME = "POLL_TIMEOUT" |
string | LOG_LEVEL_CONFIG_VAR_NAME = "LOG_LEVEL" |
string | STAT_DUMPS_DEFAULT_DIR = "/tmp/" |
string | STAT_DUMPS_DEFAULT_NAME = "%APP_NAME%_%CLASS_NAME%_stat_vars.dump" |
dictionary | LOGGERS_NAMES = {APP_CONSTS.LOGGER_NAME, "dc", "dtm", "root", ""} |
The TasksStateUpdateService class, is a listener of tasks state updates from DRCE FO of cluster nodes.
TasksStateUpdateService object and related classes definitions. This object acts as listener of updates of tasks states inside DRCE Execution Environment. The DRCE Functional objects callback connects to this service and send update message when task changes its state. This call initiated by DRCE node FO watchdog.
Definition at line 49 of file TasksStateUpdateService.py.
def dtm.TasksStateUpdateService.TasksStateUpdateService.__init__ | ( | self, | |
configParser, | |||
connectionBuilderLight = None , |
|||
connectionBuilder = None |
|||
) |
constructor initialize fields
configParser | config parser object |
connectBuilderLight | network transport connection builder light |
connectionBuilder | network transport connection builder |
Definition at line 72 of file TasksStateUpdateService.py.
def dtm.TasksStateUpdateService.TasksStateUpdateService.on_poll_timeout | ( | self | ) |
on_poll_timeout handler, now just send CheckState to EEManager
Definition at line 183 of file TasksStateUpdateService.py.
def dtm.TasksStateUpdateService.TasksStateUpdateService.onReceiveAllTaskIds | ( | self, | |
event | |||
) |
onReceiveAllTaskIds handler for receive running tasks from TasksManager
event | instance of Event object |
Definition at line 174 of file TasksStateUpdateService.py.
def dtm.TasksStateUpdateService.TasksStateUpdateService.onTCPServerRequest | ( | self, | |
event | |||
) |
onTCPServerRequest handler of TCP server requests.
Requests done by DRCE FO clients and send task state update, according with DRCE FO response protocol specification
event | instance of Event object |
Definition at line 152 of file TasksStateUpdateService.py.
def dtm.TasksStateUpdateService.TasksStateUpdateService.onUpdateTaskFieldsResponse | ( | self, | |
event | |||
) |
onUpdateTaskFieldsResponse event handler
event | instance of Event object |
Definition at line 137 of file TasksStateUpdateService.py.
def dtm.TasksStateUpdateService.TasksStateUpdateService.processUpdateTaskFields | ( | self, | |
taskResponse | |||
) |
Send UpdateTasksData event to the TasksManager.
taskResponse | The DRCE response data object |
Definition at line 217 of file TasksStateUpdateService.py.
def dtm.TasksStateUpdateService.TasksStateUpdateService.tryCheckTasksState | ( | self | ) |
send CheckState message to EEManager if don't have cached task id, then fetch from TasksManager
Definition at line 194 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.checkStateInterval |
Definition at line 118 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.checkStateNum |
Definition at line 119 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.clientExecutionEnvironmentManager |
Definition at line 89 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.clientTasksManagerName |
Definition at line 86 of file TasksStateUpdateService.py.
|
static |
Definition at line 57 of file TasksStateUpdateService.py.
|
static |
Definition at line 56 of file TasksStateUpdateService.py.
|
static |
Definition at line 55 of file TasksStateUpdateService.py.
|
static |
Definition at line 58 of file TasksStateUpdateService.py.
|
static |
Definition at line 52 of file TasksStateUpdateService.py.
|
static |
Definition at line 53 of file TasksStateUpdateService.py.
|
static |
Definition at line 54 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.drceCommandConvertor |
Definition at line 112 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.drceIdGenerator |
Definition at line 110 of file TasksStateUpdateService.py.
|
static |
Definition at line 60 of file TasksStateUpdateService.py.
|
static |
Definition at line 61 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.expect_response |
Definition at line 74 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.fetchTaskNum |
Definition at line 120 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.lastCheckStateTs |
timestamp of last send CheckState request
Definition at line 117 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.serverHost |
Definition at line 87 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.serverPort |
Definition at line 88 of file TasksStateUpdateService.py.
dtm.TasksStateUpdateService.TasksStateUpdateService.taskIdsForCheckState |
list of task ids for check state
Definition at line 128 of file TasksStateUpdateService.py.
|
static |
Definition at line 64 of file TasksStateUpdateService.py.
|
static |
Definition at line 63 of file TasksStateUpdateService.py.