Article ID: 130184, created on Jan 19, 2017, last review on Jan 19, 2017

  • Applies to:
  • Operations Automation 6.0

Symptoms

Backup tasks are stuck in task manager for 24 hours, not showing any progress, and fail with timeout as a result:

   36596021        Backup 'MySQL Database "db1000010_content_hub_he"' into group 'more updates'    Mar-23-2015 13:27:07    sc02databases.db010     1075196         Enabled Running
    36596020        Backup 'Webspace "www.NG1075196-2445.ccccloud.com"' into group 'more updates'   Mar-23-2015 13:27:07    sc03apache.ds21       1075196         Enabled Running

Tasks get rescheduled with the output:

Request has been timed out, details:
 system exception, ID 'IDL:omg.org/CORBA/TIMEOUT:1.0'
 TAO exception, minor code = 3e (timeout during recv; low 7 bits of errno: 62 Timer expired), completed = MAYBE

Task error upon failure:

Message:'Destination host 'winbackup.hosting.local' (#12), IP '10.10.10.14' : Operation execution is aborted by timeout. Please increase timeout for this operation and try again.

Backups are done from Linux hosting nodes to a Windows backup node.

The following command, run from the Management Node for the Windows backup node host_id, hangs indefinitely:

# /usr/local/pem/bin/pleskd_ctl ping 12

Cause

HCL operations get stuck when performing several simultaneous backup operations from Linux node(s) to a single Windows node. The behavior is recognized as a product issue with internal id POA-92544.

Resolution

There are 2 possible workarounds:

  • regularly (once a day) restart "pem" service on the slave backup node:

    net stop pem && net start pem
    
  • configure a Linux backup node for backing up Linux webspaces and DBs

Note: the trigger for this issue is a high number of simultaneous backup tasks. So when they get failed in large amount, after pem service restart on the backup node, it is recommended to run the failed tasks one by one in order to avoid the same hang-up again.

Search Words

Request has been timed out, details: system exception, ID 'IDL:omg.org/CORBA/TIMEOUT:1.0'

stuck tasks in POA task manager - website/database backup

backup tasks stuck

backup task

POA-92544

CORBA/TIMEOUT

HCL

5356b422f65bdad1c3e9edca5d74a1ae caea8340e2d186a540518d08602aa065 e12cea1d47a3125d335d68e6d4e15e07 956c448bddc7e1f3585373687602379f 6f1456866eed87488c0f02b298a741c0

Email subscription for changes to this article
Save as PDF