Independent disks cannot be hot backed up

clicream · 2018-05-23 13:49:06

xsibackup pro 10.3.4

I'm getting vmdk lock and independent disk error but it is dependent. How to fix this?

2018-05-23T10:10:38| The file "/vmfs/volumes/Datastore-RAID10/MAIL2/MAIL2_3-000002-delta.vmdk" is locked. Independent disks cannot be hot backed up, issue a cold backup for or exlude the disk from the backup. Make sure that the VM state (on/off) was not manually changed during the backup
ERROR: cannot locate .ERR file at [/vmfs/volumes/C220-mailesxi02/xsibackup-dir/.ERR-fe619c1144925223ca7eb164fd5870ff56ae7eb4]

admin · 2018-05-23 14:51:27

The file cannot be read, that's why you are getting this error.
It cannot be read because some other process is holding it.
The way to find which other process is holding the file is to run this command right before you get that error:

ps -c | grep "$( lsof | grep "MAIL2/MAIL2_3-000002-delta.vmdk" | awk '{print $1}' )"

It will most probably be open by the VM process, so make sure VMWare Tools are installed and running and that you meet all the requirements for the system to be quiesced properly. If you are running a Mail server, it will most probably be using a database too. If it's a Windows server, make sure you have all main services configured the right way:

[url]https://33hops.com/troubleshooting-windows-snapshots-in-esxi.html[/url]

The database server might need additional dependent services to be able to quiesce it along with the OS FS.
We have found Windows servers under heavy load (in the middle of a file level backup) to still be problematic to quiesce.

In case you cannot get your OS to be quiesced properly and you can asume some seconds of downtime, you can use [b]warm backup type[/b], which will ensure all files are backed up in a consistent state.

clicream · 2018-05-23 15:12:49

these are output that I got

ps -c | grep "$( lsof | grep "MAIL2/MAIL2_3-000002-delta.vmdk" | awk '{print
$1}' )"
20477343 20477343 vmx /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477353 20477343 vmx-vthread-12 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477354 20477343 vmx-vthread-13:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477355 20477343 vmx-vthread-14:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477356 20477343 vmx-vthread-15:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477357 20477343 vmx-mks:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477358 20477343 vmx-svga:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477359 20477343 vmx-vcpu-0:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477360 20477343 vmx-vcpu-1:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477361 20477343 vmx-vcpu-2:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477362 20477343 vmx-vcpu-3:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477363 20477343 vmx-vcpu-4:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477364 20477343 vmx-vcpu-5:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477365 20477343 vmx-vcpu-6:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx
20477366 20477343 vmx-vcpu-7:MAIL2 /bin/vmx -s sched.group=host/user -# product=2;name=VMware ESX;version=6.0.0;buildnumber=4192238;licensename=VMware ESX Server;licenseversion=6.0; -@ transport=3;msgs=ui /vmfs/volumes/59f9401a-c459a0c2-6d8a-70708b1ed678/MAIL2/MAIL2.vmx

VM is a Zimbra mail server that is using mariadb database
It have vmware-tools installed.

admin · 2018-05-23 16:36:57

Please, post the backup command you are using so that we know something more.

Well, as expected its the /bin/vmx process that is holding the file.
The most probable cause is some VM malfunction that 99% of the times is helped by:

1/ Deleting all snapshots and rebooting the VM.
2/ Restarting the host service: # [b]/etc/init.d/hostd restart[/b]
3/ Rebooting the host.

You should take action in the above order until you fix the issue.

If you want to try to dig a bit more into the problem:

1 - Delete all snapshots.
2 - Take a quiesced snapshot (no memory included). If you can't, then there's some issue with the VMWare Tools service, follow advice above.
3 - Inspect the VM configuration while the VM is running and see what are the disk names the VM is running on, they should be:

MAIL2-000001.vmdk
MAIL2_1-000001.vmdk
MAIL2_2-000001.vmdk
MAIL2_3-000001.vmdk
...

We are again guessing, but as the problematic snapshot is named 000002, is it correct to suppouse you are using OneDiff to backup your VM?

If not, you should wonder why you have two snapshots present instead of just the backup snapshot.

clicream · 2018-05-23 16:49:20

I had problem with snapshot before where I cancelled backup delete halfway and ended up have to edit vmx file to skip a snapshot to be able to boot up that VM (MAIL2)

admin · 2018-05-23 18:44:55

Then do make sure that you have recomposed your VMX file the way it should be. Being able to run a VM under ESXi does not neccessarily mean that it's correctly configured. You might be running one of the disks on top of a snapshot.

clicream · 2018-05-24 06:02:42

So I need to reconcile vm snapshot first?
Because when a backup schedule failed, I found that hard disk is pointed to -0001.vmdk file instead of just MAIL2_1.vmdk

admin · 2018-05-24 09:16:14

We don't even know if you are able to take and delete a snapshot so far.
That depends on what kind of backup you want to perform, but as you have doubts about the state of your VM, first thing to do is delete all and make sure you can create and delete snapshots before going any further.
You cannot drive a car if you are not sure whether it has all wheels on or there some missing.

clicream · 2018-05-24 09:23:10

I'm able to reconcile snapshots but did you mean delete from within VM folder?

admin · 2018-05-24 14:35:32

Deleting a snapshot does not mean that the data contained in it will be deleted, deleting a snapshot merges data in it into the base disks, but this is common terminology.

So, delete the snapshot, merge the data in the snapshot or reconcile it, and once you have performed that operation, do make sure you can create quiesced snapshots and delete them before even trying to make a backup.

Should your VM become rebelious (that could happen if VMWare Tools cannot handle MariaDB properly) due to heavy load during the snapshot operation, ponder whether performing a non-quiesced snapshot or a warm one. You have many other possibilities before going for non-quiesced or warm in any case. Just delve into how MariaDB handles data and how it interacts with LVM and VMWare Tools during the quiescing process.

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

#1 2018-05-23 13:49:06

Independent disks cannot be hot backed up

#2 2018-05-23 14:51:27

Re: Independent disks cannot be hot backed up

#3 2018-05-23 15:12:49

Re: Independent disks cannot be hot backed up

#4 2018-05-23 16:36:57

Re: Independent disks cannot be hot backed up

#5 2018-05-23 16:49:20

Re: Independent disks cannot be hot backed up

#6 2018-05-23 18:44:55

Re: Independent disks cannot be hot backed up

#7 2018-05-24 06:02:42

Re: Independent disks cannot be hot backed up

#8 2018-05-24 09:16:14

Re: Independent disks cannot be hot backed up

#9 2018-05-24 09:23:10

Re: Independent disks cannot be hot backed up

#10 2018-05-24 14:35:32

Re: Independent disks cannot be hot backed up

Board footer