Sometimes default log level does not allow to find the exact reason of the error. It is necessary to increase logs verbosity in order to receive more information about exact operation.
PPAB/BA logs can be found in the directory
/usr/local/bm/log/ on Billing Service Node/BA Aplication server. Operation logging mostly stored in the file
With the following command you can retrieve current log level of the PPAB/BA processes:
# /usr/local/stellart/bin/logsrv_ctl --command list Current log levels (* - default): *:4 WWW:4 XMLRPCD:4 _amt_service_:4 apsgate:4 generic_worker:4 logsrv_ctl:4 scheduler:4
PPAB/BA log level is an integer value from 0 to 7:
- 0 - FATAL, only fatal error that cannot be, even theoretically, processed by the application, for example, a system error like 'unable to allocate memory' are written to the log.
- 1 - ERR, any error that can be theoretically processed by the application, for example, by the caller, but the module which caught it cannot process the error, is written to log.
- 2 - WARN, any error that was caught and processed by the module.
- 3 - NOTE, is used for logging key application execution points, like incoming RPC calls.
- 4 - TRACE, is the DEFAULT logging level, when LogLevels is set to "*:4". Diagnostic messages are displayed by calling 'tracing arguments'. WARNING: If you remove LogLevels variable from the .global.conf and restart the service 'pba', then new default level is set as '3' ('NOTE')
- 5 - INFO, records the information about key code branching points, e.g. calls to external systems with their addresses.
- 6 - DEBUG, records all key code points with the data, for example, 'entering critical section' with the value of mutex used. At this level, database statements are recorded. The level is often used for troubleshooting.
- 7 - ALL, is the last chance before starting a debugger data values, for example, it requires full response text from external systems or data fetched from the database. Rarely used for troubleshooting as a lot of excessive data might appear in the log.
logsrv_ctl utility allows to change log level without application restart. For example, to set log level of the generic_worker container to 7:
# /usr/local/stellart/bin/logsrv_ctl --command set --level 7 --container generic_worker Log level 7 for container generic_worker has been changed Current log levels (* - default): *:4 WWW:4 XMLRPCD:4 _amt_service_:4 apsgate:4 generic_worker:7 logsrv_ctl:4 scheduler:4
Upon completion of work with logs change the log level to the default level:
# /usr/local/stellart/bin/logsrv_ctl --command set-default Default log levels has been restored Current log levels (* - default): *:4 WWW:4 XMLRPCD:4 _amt_service_:4 apsgate:4 generic_worker:4 logsrv_ctl:4 scheduler:4
General command description:
# logsrv_ctl --help logsrv_ctl options: --help produce help message --command arg set - sets new log level for specific container. list - shows current log level for all containers. set-default - returns log levels of all containers to their default values. --container arg Container name to apply log level. --level arg Log Level to be applied: 0 - 7 (0 - FATAL, 1 - ERR, 2 - WARN, 3 - NOTE, 4 - TRACE, 5 - INFO, 6 - DEBUG, 7 - ALL)
IMPORTANT! There is a bug #PBA-63879 ("Database queries are not always logged") which prevents the database queries being logged when the log level changed from default to debug with the logsrv_ctl command. If you need to see the queries, until the bug is fixed, you have 2 options:
- Change the log level with restart.
- Remove LogLevels parameter from
.global.conffiles, restart pba service (WARNING: new default level will be '3' then, so you might want then to add in the /etc/init.d/pba the command logsrv_ctl with the required log level different from '3'). Then you may use logsrv_ctl to change log level "on-the-fly".