Article ID: 127598, created on Nov 28, 2015, last review on Dec 28, 2016

  • Applies to:
  • Operations Automation


There are failed tasks for VE backups in customer CP.

The tasks fail with timeout with no callback received from vm2vf daemon. In /var/log/IM/PACI-im.log the following error is found:

2015-11-23 15:21:19,589 () ERROR Vm2VfProviderBase [VM2VF callbacks watchdog thread] - No registered callback found: ActiveCallbackId [timestamp=2015-11-22 23:21:19.484, reqId=162737, corbaReqId=927068, info=backup ve [{246853d3-b2e6-4758-afb1-150bbcb3bc9c}] to the node [], contextId=(95fd9507-9dd8-4ede-9502-631beafb9c85)]

The following pattern appears for the same issue on OA 7.0:

2016-11-21 16:11:18,054 (330f4970-6848-4b69-bed1-3d66704108c4) ERROR GenericVm2VfTask [Shared executor thread #7 @1 @INTERACTIVE] - timeout()
2016-11-21 16:11:18,054 (330f4970-6848-4b69-bed1-3d66704108c4) WARN  GenericVm2VfTask [Shared executor thread #7 @1 @INTERACTIVE] - VM2VF operation [BACKUP] (reqId=4070) finished with rc=-1 (0xFFFFFFFF)
2016-11-21 16:11:18,054 (330f4970-6848-4b69-bed1-3d66704108c4) WARN  GenericVm2VfTask [Shared executor thread #7 @1 @INTERACTIVE] - VM2VF operation [BACKUP] (reqId=4070) finished with rc=-1 (0xFFFFFFFF)

In /var/log/IM/PACI-vm2vf.log at the same time it is clear that the backup task is still in progress:

2015-11-23 15:21:40,678 (95fd9507-9dd8-4ede-9502-631beafb9c85) DEBUG NativeVm2VfCode [Thread-4272309] - [162737:927068] Callback invocation: in_progress(TIMEOUT: Dispatcher is still alive, 0)
2015-11-23 15:22:43,051 (95fd9507-9dd8-4ede-9502-631beafb9c85) INFO  NativeVm2VfCode [Thread-4272346] - [162737:927068] INF callback: job [__PrlSrv_CreateVmBackup] cbp 0x7f553ccdce08 index [162088] h type [PHT_EVENT/PET_DSP_EVT_BACKUP_PROGRESS_CHANGED] h/job [0x6f97d8/0x69fbac] (backup_action, backup.c, 492) @[common/generic_sdk_cb.c][366][generic_sdk_cb][51918])
2015-11-23 15:22:43,052 (95fd9507-9dd8-4ede-9502-631beafb9c85) DEBUG NativeVm2VfCode [Thread-4272347] - [162737:927068] Callback invocation: in_progress(PET_DSP_EVT_BACKUP_PROGRESS_CHANGED, 78)


The backup operation lasts longer than the task timeout.


To solve the issue, find out if it is expected that the backup task lasts so long. Normal rate is 1 hour for a 100GB virtual machine. If the VM is indeed huge and requires more than 4 hours to be completed, edit the task timeout in /usr/local/share/PACI-im/IM-config.xml:


<vm2vf callback-timeout-ms="14400000" log="true" mode="remote">


<vm2vf callback-timeout-ms="21600000" log="true" mode="remote">

to increase the timeout to 6 hours. Use "callback-timeout-ms" since "callback-timeout-min" is ignored due to software related issue #CCU-13883 that is planned to be fixed in one of the Product future versions.

Then restart the service:

# service PACI-im fast-restart

Note: on 7.0 the line related to vm2vf will contain mode="on-node" pattern instead

Search Words

ERROR ClusterImpl$Updater [Batch DB updater] - Failed to persist VE update


clone failed

scheduled backup

backup Failed

Callback timeout detected:

creation in progress


No registered callback found:

backup fail

backup vm


5356b422f65bdad1c3e9edca5d74a1ae caea8340e2d186a540518d08602aa065 e12cea1d47a3125d335d68e6d4e15e07

Email subscription for changes to this article
Save as PDF