Article ID: 117842, created on Oct 10, 2013, last review on Sep 3, 2016

  • Applies to:
  • Operations Automation

Symptoms

The Operations Automation (OA) Provider or Customer Control Panel shows the incorrect location or status for the Cloud Infrastructure (CI) server (Virtual Environment, or VE).

For example, the Provider initiates a migration of a CI server between Hardware Nodes. The server is successfully migrated and is up-and-running on the target Node.

However, the OA Control Panel still shows the server as being located on the source Hardware Node and the status of the CI server is incorrect: "Migration failed", "Migration in progress", or "Failed to start".

A similar problem may be caused by CI server backup or restore operations, or by creating a VM image.

Cause

There are several possible causes of this issue:

  • The OACI Instance Manager (IM) server did not get a callback or was not able to process the callback from the Hardware Node after the requested operation (backup, restore, migration) was completed. For example, the callback timed out.
  • The IM server was restarted, or crashed while processing a callback from the corresponding Hardware Node.
  • There was a node crash and VE failed to get relocated by the backend High Availability routines.

Resolution

  1. LOCATION.

    In order to synchronize the location of the VE, it is needed to move it manually to the hardware node, to which it is bound in Control Panel. Use Virtuzzo utilities to do that directly on the hardware node:

    • To relocate a container (CT):

      # pmigrate c [VEID|UUID] c [TARGET_NODE] --online
      

      where [VEID|UUID] - either the container ID (displayed by vzlist) or the container UUID (displayed by prlctl list) --online option is used to initiate a "hot" migration to minimize the downtime of VE services

      Real-life example:

      # pmigrate c 101 c 192.168.0.10 --online
      
    • To relocate a virtual machine (VM):

      # pmigrate v [VM_NAME|UUID] v [TARGET_NODE]
      

      where [VM_NAME|UUID] - either the name or UUID of the virtual machine (displayed by prlctl list)

      Real-life example:

      # pmigrate v test_vm v 192.168.0.10
      

      Note: for virtual machines, "online" migration mode is initiated by default in case the VM is in running state.

  2. STATUS.

    In order to synchronize the status of the VE, use Virtuozzo VE management utilities to bring the VE to the same state, as it is displayed in CP. In case of any difficulties, contact Odin Technical Support to resolve the situation.

  3. UUID mismatch.

    Sometimes after manual actions with VEs on Virtuozzo backend, the VE may end up with wrong UUID, as opposed to OACI IM records. As a result, there's the following error logged to /var/log/IM/PACI-im-errors.log:

    2016-09-03 00:00:18,502 () ERROR SnmpManagerImpl [VF executor thread #28] - VE uuid doesn't match: 2a90296c-3f8a-4182-a9a6-00917e3be9cb != f2bcb6db-72bc-4913-917e-a89486b2c0ee for VeId [customerId=1010101, name=server-1002003], update ignored
    

    Follow these steps to fix the problem:

    1. At convenient time, suspend the VM (all OS operations inside the VM will get freezed):

      # prlctl suspend 1010101.server-1002003
      
    2. Unregister it:

      # prlctl unregister 1010101.server-1002003
      
    3. Register the VM back with the required UUID:

      # prlctl register 1010101.server-1002003 --uuid 2a90296c-3f8a-4182-a9a6-00917e3be9cb
      
    4. Resume the VM:

      # prlctl resume 1010101.server-1002003
      

Search Words

paci status

Import container failed

PACI IM manager not starting

PACI migration hangs

server stuck backup in progress

PACI in unknown state

migration

PACI VE status inconsistency

CI server

paci migrate

migration in progress

Not able to resize disk and container hanging in "starting" status

paci stuck wrong status

VE status inconsistency

paci Creation in progress

Backing up

PACI-im

restore backup

VE uuid doesn't match

paci

VE is busy

Server was not found virtualization node

unable to migrate cloud server

wrong location after migration

Migration failed

paci failover inconsistency

backing up status paci

import failed

bckup in progress

vm migration

terracotta ehcache

backup in progress paci

terracotta ehache

UNCERTAIN status for VM's

instance manager shutdown

Backup in progress

1015011644.CNGAPP01

wrong location of the cloud server

transitional status

wrong location of cloud server after migration

vm stuck in Failed to restore state

The requested operation cannot be performed on the Server in-web-a-08 while it is in the current state.

stop in progress

stopping

caea8340e2d186a540518d08602aa065 5356b422f65bdad1c3e9edca5d74a1ae e12cea1d47a3125d335d68e6d4e15e07

Email subscription for changes to this article
Save as PDF