HCE Project Python language Distributed Tasks Manager Application, Distributed Crawler Application and client API bindings.  2.0.0-chaika
Hierarchical Cluster Engine Python language binding
app.Profiler.Profiler Class Reference

Class which provides functionality for use profiling. More...

Inheritance diagram for app.Profiler.Profiler:
Collaboration diagram for app.Profiler.Profiler:

Public Member Functions

def __init__ (self)
 constructor More...
 
def readConfig (self, configName)
 load log config file of prifiler More...
 
def start (self)
 start profiling More...
 
def stop (self)
 stop profiling More...
 

Public Attributes

 parser
 
 logger
 
 isStarted
 
 status
 
 sortby
 
 limit
 
 traceback
 
 tracebackLoggerMode
 
 errorMsg
 
 pr
 

Static Public Attributes

string MSG_ERROR_PARSE_CMD_PARAMS = "Error parse command line parameters."
 Constants error messages used in class. More...
 
string MSG_ERROR_EMPTY_CONFIG_FILE_NAME = "Config file name is empty."
 
string MSG_ERROR_WRONG_CONFIG_FILE_NAME = "Config file name is wrong"
 
string MSG_ERROR_LOAD_APP_CONFIG = "Error loading application config file."
 
string MSG_ERROR_READ_LOG_CONFIG = "Error read log config file."
 
string MSG_ERROR_WRONG_CONFIG_OPTION = "Wrong format option"
 
string MSG_ERROR_WRONG__OPTION_SORTBY = "Read wrong value of 'sortby' from config"
 
string PROFILER_OPTION_LOG = "log"
 Constans profiler options read from config. More...
 
string PROFILER_OPTION_STATUS = "profile"
 
string PROFILER_OPTION_SORTBY = "sortby"
 
string PROFILER_OPTION_LIMIT = "limit"
 
string PROFILER_OPTION_TRACEBACK = "traceback"
 
string PROFILER_OPTION_TRACEBACK_LOGGER_MODE = "tracebackLoggerMode"
 
int PROFILER_OPTION_STATUS_DEFAULT = 0
 Constans default values of profiler options. More...
 
string PROFILER_OPTION_SORTBY_DEFAULT = "cumulative"
 
float PROFILER_OPTION_LIMIT_DEFAULT = 1.0
 
int PROFILER_OPTION_TRACEBACK_DEFAULT = 0
 
int PROFILER_OPTION_TRACEBACK_LOGGER_MODE_DEFAULT = 1
 
list PROFILER_OPTION_SORTBY_ALLOWED_LIST = ['stdname', 'calls', 'time', 'cumulative']
 Constans allowed values of 'sortby'. More...
 
string MESSAGES_ITEMS_DELIMITER = ","
 Constans for global message list of strings in the final message for profiler records. More...
 
dictionary tracebackOptions
 Constans traceback config options (key - options name, value - default. More...
 

Private Member Functions

def __initTrackbackOptions (self, config, section)
 initialize traceback options from config file More...
 
def __loadAppConfig (self, configName)
 load application config file More...
 
def __parseParams (self)
 parsing paramers More...
 

Detailed Description

Class which provides functionality for use profiling.

Definition at line 32 of file Profiler.py.

Constructor & Destructor Documentation

◆ __init__()

def app.Profiler.Profiler.__init__ (   self)

constructor

Constructor

Definition at line 94 of file Profiler.py.

94  def __init__(self):
95  '''
96  Constructor
97  '''
98  self.parser = None
99  self.logger = None
100  self.isStarted = False
101  self.status = self.PROFILER_OPTION_STATUS_DEFAULT
102  self.sortby = self.PROFILER_OPTION_SORTBY_DEFAULT
103  self.limit = self.PROFILER_OPTION_LIMIT_DEFAULT
104  self.traceback = self.PROFILER_OPTION_TRACEBACK_DEFAULT
105  self.tracebackLoggerMode = self.PROFILER_OPTION_TRACEBACK_LOGGER_MODE_DEFAULT
106  self.errorMsg = ""
107  #create instance of profiler
108  self.pr = cProfile.Profile()
109  try:
110  #load configuration files and initialization
111  self.__loadAppConfig(self.__parseParams())
112  except Exception, err:
113  self.errorMsg = str(err)
114  self.status = self.PROFILER_OPTION_STATUS_DEFAULT
115 
116  if self.traceback > 0:
117  #used self.tracebackLoggerMode already reinit from config
118  if self.tracebackLoggerMode > 0 and Utils.tracebackLogger is None:
119  Utils.tracebackLogger = Utils.MPLogger().getLogger(APP_CONSTS.LOGGER_NAME_TRACEBACK)
120 
121  sys.settrace(tracefunc)
122 
123 
def __init__(self)
constructor
Definition: UIDGenerator.py:19

Member Function Documentation

◆ __initTrackbackOptions()

def app.Profiler.Profiler.__initTrackbackOptions (   self,
  config,
  section 
)
private

initialize traceback options from config file

Parameters
config- config parser
section- section name
Returns
- None

Definition at line 129 of file Profiler.py.

129  def __initTrackbackOptions(self, config, section):
130 
131  for key, value in self.tracebackOptions.items():
132  opt = Utils.getConfigParameter(config, section, key, value)
133 
134  if opt and opt != value:
135  try:
136  exec('Utils.' + str(key) + '=' + str(opt)) # pylint: disable=W0122
137  except Exception:
138  raise Exception(self.MSG_ERROR_WRONG_CONFIG_OPTION + ': ' + str(key))
139 
140 
Here is the caller graph for this function:

◆ __loadAppConfig()

def app.Profiler.Profiler.__loadAppConfig (   self,
  configName 
)
private

load application config file

Parameters
configName- name of application config file
Returns
- profiler config file name

Definition at line 145 of file Profiler.py.

145  def __loadAppConfig(self, configName):
146  #variable for result
147  confLogFileName = ""
148 
149  try:
150  config = ConfigParser.ConfigParser()
151  config.optionxform = str
152 
153  readOk = config.read(configName)
154 
155  if len(readOk) == 0:
156  raise Exception(self.MSG_ERROR_WRONG_CONFIG_FILE_NAME + ": " + configName)
157 
158  if config.has_section(APP_CONSTS.CONFIG_APPLICATION_SECTION_NAME):
159  confLogFileName = str(config.get(APP_CONSTS.CONFIG_APPLICATION_SECTION_NAME, self.PROFILER_OPTION_LOG))
160 
161  if config.has_section(APP_CONSTS.CONFIG_PROFILER_SECTION_NAME):
162  self.status = int(config.get(APP_CONSTS.CONFIG_PROFILER_SECTION_NAME, self.PROFILER_OPTION_STATUS))
163  self.sortby = str(config.get(APP_CONSTS.CONFIG_PROFILER_SECTION_NAME, self.PROFILER_OPTION_SORTBY))
164  self.limit = float(config.get(APP_CONSTS.CONFIG_PROFILER_SECTION_NAME, self.PROFILER_OPTION_LIMIT))
165  self.traceback = int(config.get(APP_CONSTS.CONFIG_PROFILER_SECTION_NAME, self.PROFILER_OPTION_TRACEBACK))
166  if self.traceback > 0:
167  self.__initTrackbackOptions(config, APP_CONSTS.CONFIG_PROFILER_SECTION_NAME)
168  if config.has_option(APP_CONSTS.CONFIG_PROFILER_SECTION_NAME, self.PROFILER_OPTION_TRACEBACK_LOGGER_MODE):
169  self.tracebackLoggerMode = int(Utils.getConfigParameter(config, APP_CONSTS.CONFIG_PROFILER_SECTION_NAME,
170  self.PROFILER_OPTION_TRACEBACK_LOGGER_MODE,
171  self.PROFILER_OPTION_TRACEBACK_LOGGER_MODE_DEFAULT))
172  else:
173  pass
174  else:
175  pass
176 
177  except Exception, err:
178  raise Exception(self.MSG_ERROR_LOAD_APP_CONFIG + ' ' + str(err))
179 
180  if self.sortby not in self.PROFILER_OPTION_SORTBY_ALLOWED_LIST:
181  raise Exception(self.MSG_ERROR_WRONG__OPTION_SORTBY)
182 
183  return confLogFileName
184 
185 
Here is the call graph for this function:
Here is the caller graph for this function:

◆ __parseParams()

def app.Profiler.Profiler.__parseParams (   self)
private

parsing paramers

Returns
- configName read from command line argument

Definition at line 206 of file Profiler.py.

206  def __parseParams(self):
207  configName = ""
208  try:
209  self.parser = argparse.ArgumentParser(description='Process command line arguments.', add_help=False)
210  self.parser.add_argument('-c', '--config', action='store', metavar='config_file', help='config ini-file')
211 
212  args = self.parser.parse_known_args()
213 
214  if args is None or args[0] is None or args[0].config is None:
215  raise Exception(self.MSG_ERROR_EMPTY_CONFIG_FILE_NAME)
216 
217  configName = str(args[0].config)
218 
219  except Exception, err:
220  raise Exception(self.MSG_ERROR_PARSE_CMD_PARAMS + ' ' + str(err))
221 
222  return configName
223 
224 

◆ readConfig()

def app.Profiler.Profiler.readConfig (   self,
  configName 
)

load log config file of prifiler

Parameters
configName- name of log profiler config file
Returns
- None

Definition at line 190 of file Profiler.py.

190  def readConfig(self, configName):
191  try:
192  if isinstance(configName, str) and len(configName) == 0:
193  raise Exception(self.MSG_ERROR_EMPTY_CONFIG_FILE_NAME)
194 
195  logging.config.fileConfig(configName)
196  #call rotation log files and initialization logger
197  self.logger = Utils.MPLogger().getLogger()
198 
199  except Exception, err:
200  raise Exception(self.MSG_ERROR_READ_LOG_CONFIG + ' ' + str(err))
201 
202 
Here is the call graph for this function:

◆ start()

def app.Profiler.Profiler.start (   self)

start profiling

Returns
- None

Definition at line 228 of file Profiler.py.

228  def start(self):
229  if self.isStarted is False:
230  self.pr.enable()
231  self.isStarted = True
232 
233 

◆ stop()

def app.Profiler.Profiler.stop (   self)

stop profiling

Returns
- None

Definition at line 237 of file Profiler.py.

237  def stop(self):
238  msgStr = self.MESSAGES_ITEMS_DELIMITER.join(messagesList)
239 
240  if self.isStarted is True and self.status > 0:
241  self.isStarted = False
242  self.pr.disable()
243  s = StringIO.StringIO()
244  ps = pstats.Stats(self.pr, stream=s).sort_stats(self.sortby)
245  ps.print_stats(self.limit)
246 
247  #call rotation log files
248  self.logger = Utils.MPLogger().getLogger(APP_CONSTS.LOGGER_NAME_PROFILER)
249  #dump profile information to log
250  if self.logger:
251  self.logger.debug("%s\n%s", msgStr, str(s.getvalue()))
252  self.logger.debug("%s", APP_CONSTS.LOGGER_DELIMITER_LINE)
253 
254  if self.traceback > 0:
255  if Utils.tracebackLogger is None:
256  #call rotation log files
257  self.logger = Utils.MPLogger().getLogger(APP_CONSTS.LOGGER_NAME_TRACEBACK)
258  #dump traceback information to log
259  if self.logger:
260  self.logger.debug("%s\n%s", msgStr, "\n".join(Utils.tracebackList))
261  self.logger.debug("%s", APP_CONSTS.LOGGER_DELIMITER_LINE)
262 
Here is the call graph for this function:

Member Data Documentation

◆ errorMsg

app.Profiler.Profiler.errorMsg

Definition at line 106 of file Profiler.py.

◆ isStarted

app.Profiler.Profiler.isStarted

Definition at line 100 of file Profiler.py.

◆ limit

app.Profiler.Profiler.limit

Definition at line 103 of file Profiler.py.

◆ logger

app.Profiler.Profiler.logger

Definition at line 99 of file Profiler.py.

◆ MESSAGES_ITEMS_DELIMITER

string app.Profiler.Profiler.MESSAGES_ITEMS_DELIMITER = ","
static

Constans for global message list of strings in the final message for profiler records.

Definition at line 62 of file Profiler.py.

◆ MSG_ERROR_EMPTY_CONFIG_FILE_NAME

string app.Profiler.Profiler.MSG_ERROR_EMPTY_CONFIG_FILE_NAME = "Config file name is empty."
static

Definition at line 36 of file Profiler.py.

◆ MSG_ERROR_LOAD_APP_CONFIG

string app.Profiler.Profiler.MSG_ERROR_LOAD_APP_CONFIG = "Error loading application config file."
static

Definition at line 38 of file Profiler.py.

◆ MSG_ERROR_PARSE_CMD_PARAMS

string app.Profiler.Profiler.MSG_ERROR_PARSE_CMD_PARAMS = "Error parse command line parameters."
static

Constants error messages used in class.

Definition at line 35 of file Profiler.py.

◆ MSG_ERROR_READ_LOG_CONFIG

string app.Profiler.Profiler.MSG_ERROR_READ_LOG_CONFIG = "Error read log config file."
static

Definition at line 39 of file Profiler.py.

◆ MSG_ERROR_WRONG__OPTION_SORTBY

string app.Profiler.Profiler.MSG_ERROR_WRONG__OPTION_SORTBY = "Read wrong value of 'sortby' from config"
static

Definition at line 41 of file Profiler.py.

◆ MSG_ERROR_WRONG_CONFIG_FILE_NAME

string app.Profiler.Profiler.MSG_ERROR_WRONG_CONFIG_FILE_NAME = "Config file name is wrong"
static

Definition at line 37 of file Profiler.py.

◆ MSG_ERROR_WRONG_CONFIG_OPTION

string app.Profiler.Profiler.MSG_ERROR_WRONG_CONFIG_OPTION = "Wrong format option"
static

Definition at line 40 of file Profiler.py.

◆ parser

app.Profiler.Profiler.parser

Definition at line 98 of file Profiler.py.

◆ pr

app.Profiler.Profiler.pr

Definition at line 108 of file Profiler.py.

◆ PROFILER_OPTION_LIMIT

string app.Profiler.Profiler.PROFILER_OPTION_LIMIT = "limit"
static

Definition at line 47 of file Profiler.py.

◆ PROFILER_OPTION_LIMIT_DEFAULT

float app.Profiler.Profiler.PROFILER_OPTION_LIMIT_DEFAULT = 1.0
static

Definition at line 54 of file Profiler.py.

◆ PROFILER_OPTION_LOG

string app.Profiler.Profiler.PROFILER_OPTION_LOG = "log"
static

Constans profiler options read from config.

Definition at line 44 of file Profiler.py.

◆ PROFILER_OPTION_SORTBY

string app.Profiler.Profiler.PROFILER_OPTION_SORTBY = "sortby"
static

Definition at line 46 of file Profiler.py.

◆ PROFILER_OPTION_SORTBY_ALLOWED_LIST

list app.Profiler.Profiler.PROFILER_OPTION_SORTBY_ALLOWED_LIST = ['stdname', 'calls', 'time', 'cumulative']
static

Constans allowed values of 'sortby'.

Definition at line 59 of file Profiler.py.

◆ PROFILER_OPTION_SORTBY_DEFAULT

string app.Profiler.Profiler.PROFILER_OPTION_SORTBY_DEFAULT = "cumulative"
static

Definition at line 53 of file Profiler.py.

◆ PROFILER_OPTION_STATUS

string app.Profiler.Profiler.PROFILER_OPTION_STATUS = "profile"
static

Definition at line 45 of file Profiler.py.

◆ PROFILER_OPTION_STATUS_DEFAULT

int app.Profiler.Profiler.PROFILER_OPTION_STATUS_DEFAULT = 0
static

Constans default values of profiler options.

Definition at line 52 of file Profiler.py.

◆ PROFILER_OPTION_TRACEBACK

string app.Profiler.Profiler.PROFILER_OPTION_TRACEBACK = "traceback"
static

Definition at line 48 of file Profiler.py.

◆ PROFILER_OPTION_TRACEBACK_DEFAULT

int app.Profiler.Profiler.PROFILER_OPTION_TRACEBACK_DEFAULT = 0
static

Definition at line 55 of file Profiler.py.

◆ PROFILER_OPTION_TRACEBACK_LOGGER_MODE

string app.Profiler.Profiler.PROFILER_OPTION_TRACEBACK_LOGGER_MODE = "tracebackLoggerMode"
static

Definition at line 49 of file Profiler.py.

◆ PROFILER_OPTION_TRACEBACK_LOGGER_MODE_DEFAULT

int app.Profiler.Profiler.PROFILER_OPTION_TRACEBACK_LOGGER_MODE_DEFAULT = 1
static

Definition at line 56 of file Profiler.py.

◆ sortby

app.Profiler.Profiler.sortby

Definition at line 102 of file Profiler.py.

◆ status

app.Profiler.Profiler.status

Definition at line 101 of file Profiler.py.

◆ traceback

app.Profiler.Profiler.traceback

Definition at line 104 of file Profiler.py.

◆ tracebackLoggerMode

app.Profiler.Profiler.tracebackLoggerMode

Definition at line 105 of file Profiler.py.

◆ tracebackOptions

dictionary app.Profiler.Profiler.tracebackOptions
static
Initial value:
= {'tracebackIdent':Utils.tracebackIdent,
'tracebackIdentFiller':Utils.tracebackIdentFiller,
'tracebackMessageCall':Utils.tracebackMessageCall,
'tracebackMessageExit':Utils.tracebackMessageExit,
'tracebackmessageDelimiter':Utils.tracebackmessageDelimiter,
'tracebackTimeMark':Utils.tracebackTimeMark,
'tracebackTimeMarkFormat':Utils.tracebackTimeMarkFormat,
'tracebackTimeMarkDelimiter':Utils.tracebackTimeMarkDelimiter,
'tracebackIncludeInternalCalls':Utils.tracebackIncludeInternalCalls,
'tracebackIncludeLineNumber':Utils.tracebackIncludeLineNumber,
'tracebackIncludeLineNumberDelimiter':Utils.tracebackIncludeLineNumberDelimiter,
'tracebackIncludeFileNumber':Utils.tracebackIncludeFileNumber,
'tracebackIncludeFileNumberDelimiter':Utils.tracebackIncludeFileNumberDelimiter,
'tracebackFunctionNameDelimiter':Utils.tracebackFunctionNameDelimiter,
'tracebackExcludeModulePath':Utils.tracebackExcludeModulePath,
'tracebackExcludeFunctionName':Utils.tracebackExcludeFunctionName,
'tracebackExcludeFunctionNameStarts':Utils.tracebackExcludeFunctionNameStarts,
'tracebackIncludeExitCalls':Utils.tracebackIncludeExitCalls,
'tracebackRecursionlimit':Utils.tracebackRecursionlimit,
'tracebackRecursionlimitErrorMsg':Utils.tracebackRecursionlimitErrorMsg,
'tracebackIncludeLocals':Utils.tracebackIncludeLocals,
'tracebackIncludeArg':Utils.tracebackIncludeArg,
'tracebackIncludeLocalsPrefix':Utils.tracebackIncludeLocalsPrefix,
'tracebackIncludeArgPrefix':Utils.tracebackIncludeArgPrefix,
'tracebackElapsedTimeDelimiter':Utils.tracebackElapsedTimeDelimiter,
'tracebackElapsedTimeFormat':Utils.tracebackElapsedTimeFormat,
'tracebackUnknownExceptionMsg':Utils.tracebackUnknownExceptionMsg}

Constans traceback config options (key - options name, value - default.

Definition at line 65 of file Profiler.py.


The documentation for this class was generated from the following file: