This page outlines different situations that a user might encounter when using Coriolis.
No matching migration image type found for OS type ‘windows’
When Replicating/Migrating a Windows VM, the above error can be seen if a Windows template has not been specified for the Temporary migration worker. in the Target options menu, under the Advanced screen, both Linux and Windows templates have to be specified. This is required as Disk cloning is performed using the Linux template and OSMorphing is performed using the Windows template.
Troubleshooting the Docker-based Coriolis appliance
Should the Docker-based appliance be experiencing issues, one should first start by checking the status of the containers by running the following:
Container status command
1 |
$ docker ps --all |
All of the containers on the appliance are configured to restart upon the failure of the container service and should appear as “Up” the whole time, and thus containers that fail to start to appear as “Restarting” or “Up 1 second”
Should a container be in “Exited” status, one can attempt to start it back up by running the following:
Starting a stopped container
1 |
$ docker start $CONTAINER_NAME |
As a last resort, the container may be killed completely and the deployment run again via running the following:
Stopping a container and redeploying it
1 2 |
$ docker kill $CONTAINER_NAME $ ~/coriolis-docker/coriolis-ansible deploy |
The command ‘~/coriolis-docker/coriolis-ansible deploy’ uses Ansible to deploy all the Coriolis components (API, conductor, replica-cron, worker, web, and web-proxy). Running it will only redeploy the containers which have been previously killed, and is idempotent (may safely be run as many times as needed)
Troubleshooting OSMorphing on the Docker-based Coriolis appliance
The OSMorphing process is the procedure undergone by the guest OS being migrated in order to ensure it will boot on the destination platform.
OSMorphing always occurs on the destination platform, where Coriolis will create a temporary VM (either Linux or Windows, depending on the guest OS being migrated) in order to perform the OSMorphing steps on the already-synced disks.
Considering that OSMorphing errors may be a direct result of an incompatibility between the specific setup of the guest OS and Coriolis or the destination platform, they are relatively more common during a Migration, and the only concrete way to debug the issues would be to login to the temporary OSMorphing VM Coriolis is controlling in order to see exactly what Coriolis sees.
Configuration:
There is a dedicated ‘debug_os_morphing_errors’ flag in the ‘[conductor]’ section of /etc/coriolis/coriolis.conf:
debug_os_morphing_errors example
1 2 3 |
# NOTE: this option must be appended within the '[conductor]' section: [conductor] debug_os_morphing_errors = false |
If set to ‘true’, the Coriolis Conductor will skip performing the cleanup steps on the temporary resources on the destination platform and will log out the connection info for the temporary VM in the Coriolis-conductor.log.
NOTE please remember to run a docker restart Coriolis-conductor
for any changes to debug_os_morphing_errors
to take effect.
Accessing the Coriolis temporary worker VM
Below is a sample of the connection info for the temporary worker VM as seen in Coriolis-conductor.log
:
debug_os_morphing_errors log output example
1 2 3 4 5 6 7 8 |
# NOTE: the private keys for the worker VMs may have the passphrase # configured in coriolis.conf set to them: $ grep "temp_keypair_password" /etc/coriolis/coriolis.conf temp_keypair_password = qJDxLBbdFCvRKP3J8qTxcvuh # NOTE: the IP and connection info the temp VM will be logged in the Conductor logs: $ grep "have been cancelled to allow for OSMorphing debugging." /var/log/coriolis/coriolis-conductor.log 2019-10-16 21:53:53.007 WARNING coriolis.conductor.rpc.server [req-05f6b9fa-6d78-45fc-9bee-a1e4de05ac49 ] All subtasks for Migration 'ece4b02d-4ab8-405c-b33e-c36c63a20993' have been cancelled to allow for OSMorphing debugging. The connection info for the worker VM is: {'ip': '138.91.73.150', 'port': 22, 'username': 'coriolis', 'password': None, 'pkey': '<pkey_data>'} |
WARNING if ‘debug_os_morphing_errors’ is set, the lifecycle of the temporary OSMorphing worker VM and all of its associated resources (disks, NICs, public IPs, etc…) will no longer be managed by Coriolis.
All the temporary OSMorphing resources on the destination platform will need to be manually cleaned up after investigations have concluded.
After logging into the temp VM, the following things to check may be of importance:
- if all of the disks of the migrated VM are visible in the temp VM
- if all of the disks are mountable within the temp VM (reviewing
dmesg
output for new mounts is recommended) - if the order of the disks is the same as the order reported by the destination platform’s UI/API
- if the disks are using any non-standard partitioning layouts/formats (ex: BitLocker/dm-crypt, Dynamic Disks on Windows, etc…)
- if Coriolis properly identified and mounted the guest OS being migrated (there should be some non-root mount points on the worker)
- try to run the
chroot
command attempted by Coriolis which leads to the OSMorphing error (can be seen in Coriolis’ error logs)
AWS – 500 Server Error: Internal Server Error for url: https://IP:5566/api/v1/dev/xvdf/acquire
Some AWS instance types use SSD-backing storage, and although the guest OS internally has the disks named /dev/nvmeX
, the AWS API returns the disk labels as /dev/xvdf
.
To overcome this, if the error is encountered when using AWS as a Destination Cloud, consider changing the Instance Type for the Migration Mapping used for the Coriolis Temporary worker.
For example, if the error is seen when using t3.medium, try to use c5.large instead, or other types available, depending on the region.
Backing up all service Databases from the Docker-based Coriolis appliance
Both Coriolis and most of its supporting services (ex: Keystone, Barbican) save their state in a Relational database.
The standard Docker-based Coriolis appliance ships with MariaDB (a MySQL spin-off) deployed by Kolla as a Docker container that is running locally on the appliance.
Thusly, it is highly recommended that periodic database backups be performed from the Coriolis appliance so as to save any existing data, making it easily recoverable on another appliance in case the main one goes bad.
The backup procedure for all Databases can be run using the following:
Dumping all service databases from the mariadb
container
1 2 3 |
$ DBPASS="$(grep -E '^(database_password)' /etc/kolla/passwords.yml | awk '{print $2}')" $ docker exec -ti mariadb mysqldump -u root -p"$DBPASS" --all-databases > /root/coriolis_appliance_dbs_backups.sql $ ### copy the '/root/coriolis_appliance_dbs_backups.sql' file somewhere safe |
WARNING the below steps will completely replace any existing data in the current DB
To import a previously-dumped database backup to a new appliance, the following may be run:
Reinstating all service databases from a previous SQL dump
1 2 3 |
$ DBPASS="$(grep -E '^(database_password)' /etc/kolla/passwords.yml | awk '{print $2}')" $ cat /root/coriolis_appliance_dbs_backups.sql > docker exec -ti mariadb mysql -u root -p"$DBPASS" $ ### presuming '/root/coriolis_appliance_dbs_backups.sql' is the previously-dumped database file |
Change the admin password
The following command can be used to change the password for the admin user:
1 |
$ openstack user set --password-prompt admin |
Once done, consider setting also the corresponding password in /etc/kolla/admin-openrc.sh:
1 |
$ export OS_PASSWORD=YourNewPassword |
For multi-tenant configurations, the following command can be used to change the password for any user:
1 |
$ openstack user set --name admin --domain Default --project admin --project-domain Default --password "new password" admin |
Root partition on BTRFS Subvolume
A guest VM that has the root partition on a Btrfs subvolume might fail to migrate under Coriolis.
Several Linux distributions come with a default partition scheme that uses Btrfs on the root subvolume, so we highly recommend checking the guest VM partition scheme prior to starting the migration. The support for Btrfs subvolumes will be added in a future Coriolis release, please contact our Coriolis Support Team for more details.