#1 2017-12-27 07:16:17

matz
Member
Registered: 2017-06-28
Posts: 29

Cannot copy VMSD file details

Hello.

I have two servers with esxi free 6.0U2. On both servers I have installed xsibackup-pro 10.2.1.
When I try to backup virtual machines from one to another box (backup prog rsync or xsidiff) i get error.

My example backup command:

./xsibackup  --backup-prog=rsync --backup-point="x.x.x.x:22:/vmfs/volumes/xsinas/xsit-$( date +'%Y-%m')" --backup-type=all
 --mail-from=x@gmail.com --mail-to=x --smtp-srv=smtp.gmail.com --smtp-usr=x --smtp-pwd=x --smtp-port=465

Errors from mail report:

• [ Tue Dec 26 19:07:39 UTC 2017 ] ERROR (CLCPNVMS) details [iRedMail] error: cannot copy VMSD file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/iRedMail/iRedMail.vmsd: nonexistent directory

• [ Tue Dec 26 19:07:39 UTC 2017 ] ERROR (CLCPNVMX) details [iRedMail] error: cannot copy VMX file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/iRedMail/iRedMail.vmx: nonexistent directory

• [ Tue Dec 26 20:12:58 UTC 2017 ] ERROR (CLCPNVMX) details [Cycki] error: cannot copy VMX file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/Cycki/Cycki_Thin.vmx: nonexistent directory

• [ Tue Dec 26 20:12:59 UTC 2017 ] ERROR (CLCPNVMS) details [Cycki] error: cannot copy VMSD file det ails: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/Cycki/Cycki_Thin.vmsd: nonexistent directory

• [ Tue Dec 26 21:52:12 UTC 2017 ] ERROR (CLCPNVMS) details [KFD-SERVER.KFD.local] error: cannot copy VMSD file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/KFD-SERVER.KFD.local/KFD-SERVER.KFD.local.vmsd: nonexistent directory

• [ Tue Dec 26 21:52:12 UTC 2017 ] ERROR (CLCPNVMX) details [KFD-SERVER.KFD.local] error: cannot copy VMX file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/KFD-SERVER.KFD.local/KFD-SERVER.KFD.local.vmx: nonexistent directory

• [ Tue Dec 26 22:40:10 UTC 2017 ] ERROR (CLCPNVMS) details [MAC_INVEST] error: cannot copy VMSD file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/MAC _INVEST/MAC_INVEST.vmsd: nonexistent directory

• [ Tue Dec 26 22:40:10 UTC 2017 ] ERROR (CLCPNVMX) details [MAC_INVEST] error: cannot copy VMX file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/MAC_INVEST/MAC_INVEST.vmx: nonexistent directory

• [ Tue Dec 26 23:16:07 UTC 2017 ] ERROR (CLCPNVMS) details [W2K12R2] error: cannot copy VMSD file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/W2K12R2/W2K12R2.vmsd: nonexistent directory

• [ Tue Dec 26 23:16:07 UTC 2017 ] ERROR (CLCPNVMX) details [W2K12R2] error: cannot copy VMX file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/W2K12R2/W2K12R2.vmx: nonexistent directory

• [ Wed Dec 27 01:24:45 UTC 2017 ] ERROR (CLCPNVMS) details [CHR] error: cannot copy VMSD file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/CHR/CHR.vmsd: nonexistent directory

• [ Wed Dec 27 01:24:45 UTC 2017 ] ERROR (CLCPNVMX) details [CHR] error: cannot copy VMX file details: ./xsibackup: eval: line 1: can't create /vmfs/volumes/xsinas/xsit-2017-12/CHR/CHR.vmx: nonexistent directory
• [ Wed Dec 27 01:24:43 UTC 2017 ] WARNING (CLVMXFPA) details No .vmxf file found for CHR (6)

All directories from errors are existent on remote machine:

[root@ns3068065:~] cd /vmfs/volumes/xsinas/xsit-2017-12/CHR
[root@ns3068065:/vmfs/volumes/d868bde3-5007f8ba/xsit-2017-12/CHR]

Regards.
Damian.

Offline

#2 2017-12-27 09:21:47

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Cannot copy VMSD file details

You are mixing --backup-prog=rsync with a dynamic folder on the other server. This type of path:

/vmfs/volumes/xsinas/xsit-$( date +'%Y-%m')

Can be used for local paths only.
On top of that, Rsync is not the best --backup-prog option to transfer files between servers.

Offline

#3 2017-12-27 21:56:45

matz
Member
Registered: 2017-06-28
Posts: 29

Re: Cannot copy VMSD file details

Yes, right. Its working now. Thank You. smile

I know rsync is not the best option but xsidiff never works for me. I always get error with consistency check.

[W2K12R2] info: (source) first 500M hash is: e72e0ed4b1c2af7e31806c89789ec2d2e38b5d44
[W2K12R2] info: (target) first 500M hash is: e72e0ed4b1c2af7e31806c89789ec2d2e38b5d44
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[W2K12R2] info: first 500M check | OK [ e72e0ed4b1c2af7e31806c89789ec2d2e38b5d44 ]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[W2K12R2] error DIFQMSH3: data size mistmatch: 101680 NE 101116...
[CHR] info: comparing first 500M of [CHR-flat.vmdk]...
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[CHR] info: (source) first 500M hash is: 3697a163bc34f6dc73a7070216bc38641137ccc7
[CHR] info: (target) first 500M hash is: 3697a163bc34f6dc73a7070216bc38641137ccc7
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[CHR] info: first 500M check | OK [ 3697a163bc34f6dc73a7070216bc38641137ccc7 ]
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[CHR] error DIFQMSH3: data size mistmatch: 128 NE 120...

Last edited by matz (2017-12-27 21:57:06)

Offline

#4 2017-12-28 11:46:40

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Cannot copy VMSD file details

Have you downloaded your XSIDiff license and installed it?.
What you are posting is (c) OneDiff output. First 500M check is working O.K., what you are getting is a used size mistmatch, which could be due to using an NFS datastore wich is formatted with a different file system than VMFS. When using OneDiff, try to always backup to VMFS or parse the --certify-backup=yes option to make sure backed up disks are an exact copy of the originals, as used sizes could vary due to using different cluster sizes.

XSIBACKUP-PRO 10.2.1 will detect the FS and remove the "used size" check on non VMFS datastores, but for some reason it's not detecting  your target datastore as something different.

I see that you datastore is named /vmfs/volumes/xsinas, which leads me to think you are probably using the bundled XSINAS deduplication appliance as the target of your (c) OneDiff backups. There's nothing wrong from a theoretical point of view, but combining both won't offer you any real advantage, and will on the other side reduce your backup speed and add a huge overload with very little advantage in return.

Offline

#5 2017-12-31 09:55:54

matz
Member
Registered: 2017-06-28
Posts: 29

Re: Cannot copy VMSD file details

So now, I have "half" worked solution smile

I try to make backups via xsidiff to remote esxi 6.0U2 box, and have retention with deduplication on that box.
On 1st esxi server I have runing machines that I wanted to backup to second esxi server.
So on 1st I run:

./xsibackup --backup-prog=onediff  --certify-backup=yes --backup-point="x:22:/vmfs/volumes/datastore2" --backup-type=all --mail-from=x@gmail.com --mail-to=x@x.pl --smtp-srv=smtp.gmail.com --smtp-usr=x@gmail.com --smtp-pwd=x --smtp-port=465

And I get success. On second servers I have machines with _XSIBAK.


3TFSdwK5RXCrEAEO0svLDg.png

jyDuTjIeQQyURAfTR9_-4w.png

So all ok for me. But I want backup retency, so I run on 1st server:

./xsibackup --host=ip.second.server --backup-point="/vmfs/volumes/xsinas/" --certify-backup=yes --backup-type=custom --backup-vms="CHR,Cycki,iRedMail,MAC_INVEST,W2K12R2,KFD-SERVER.KFD.local" --mail-from=x@gmail.com --mail-to=x@x.pl --smtp-srv=smtp.gmail.com --smtp-usr=x@gmail.com --smtp-pwd=x --smtp-port=465

But xsibackup dont see machines backuped by xsidiff and I get error:

dvWPAk8PRVCwQM9QZl5_JA.png

So I check VM's on second server by running ./xsibackup locally on server and I get:

rRjzYuguQq_OMo9bfCFsWw.png

Offline

#6 2017-12-31 16:46:41

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Cannot copy VMSD file details

Well, lets split thing up, there are many questions on top of the table:

1 - XSIBACKUP-PRO 10.2.1 addresses size mismatch errors when backing up to NFS, upgrade if you have not already done so. There are three builds of this release, so having the same version number does not mean you have the latest. If you use thick disks vs thin virtual disks, AKA sparse files, you will always get a size mismatch if you backup locally, new upcoming version 10.2.2 will address this. The easiest workaround by now, until we release 10.2.2 is to use the --certify-backup argument.

2 - _XSIBAK VMs are ignored, to avoid backing them up by default. You can nevertheless make XSIBackup aware of those VMs by using the --override=xsibakfilter. This is in the Man Page: https://33hops.com/xsibackup-help-man-p … l#override

3 - The last screenshot is showing a "broken pipe" error, which is undoubtedly informing you of a transmission error, which could be, in turn, due to a hardware problem, faulty cable, or be just a derivate error due to a timeout or some other problem. Apart from that, the program output is telling you it's not finding the --backup-point you have set, which is definitive on its own.

4 - I see you are focusing very much on XSINAS, which can be a great tool. Many people try it and they like it so much that they try to use it for almost everything. As any other tool has its pros and cons.

XSINAS was the first deduplication method we released along with XSIBackup, then support for Borg backup was added and in the end we released XSITools, which is the newest deduplication program available in XSIBackup. In fact XSITools is the recommended method to archive VMs, as it's faster and much more powerful than the other two. Borg is a great deduplication program, but as XSIBackup cannot run a true Borg client, due to ESXi not being compatible with compiled Python binaries, you are limited to sending all data over the network, which is a serious inconvenient.

XSINAS is an inline deduplication FS based on FUSE, this means that the data stored there is directly accesible through the file system. Inline deduplication file systems have their own disadvantages though: they need to do the hash search plus the eventual compression on the fly as data is written to disk. This leads to making them extremely resource hungry, and this literally means that they will use as much memory and CPU as you can add, and that they will clogg as you add more data into them, as they will need to look for chunk's hashes in a growing database.

This makes inline deduplication filesystems not so ideal to store huge volumes of data, as you will be using most resources in a feature that is not so interesting at the time to archive data, being real time or inline is not something fundamental to storing data. What is then XSINAS useful for?: well, you can use it to store limited amounts of deduplicated data, to serve as a file server, to store VM system disks, which don't change much over time and are basically read devices, etc...

On the contrary, XSITools has been designed with a single purpose in mind, namely to store or archive VMs. The block size is very big, default is 50M, and the database is the very same FS. It just uses the amount of memory to manage the deduplicated chunks, one at a time, this makes XSITools extremely lightweight and fast.

Offline

#7 2018-01-04 08:45:15

matz
Member
Registered: 2017-06-28
Posts: 29

Re: Cannot copy VMSD file details

Thank You for Your support. So now I change my tactics.

I backing up vms from 1st esxi server to another by:

./xsibackup --backup-prog=onediff --backup-point="x:22:/vmfs/volumes/datastore2" --backup-type=all
 --mail-to=x@x.pl --use-smtp=1 --certify-backup=yes

And then I backup machines on remote esxi server (second) to local deduplicated storage (xsitools) by:

--host=x --backup-prog=xsitools:z --backup-point="/vmfs/volumes/datastore2/xsit-$( date +'%Y-%m')" --certify-backup=yes --backup-type=all --override=xsibakfilter --mail-to=x@x.pl --use-smtp=1

All macines except one goes OK. When I try to backup this one machine i get error (this machine working on Windows 20008 server foundation and has been converted from psyhical to virtual via vmware converter)

Logs:
https://pastebin.com/wTchjxuh

Last edited by matz (2018-01-04 08:45:33)

Offline

#8 2018-01-04 09:55:00

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Cannot copy VMSD file details

Windows Server OSs (and Windows OSs in general) are trickier to backup than *NIXes, I won't delve into the details now, but the most delicate part is quiescing, which XSIBackup does by default. If you can afford to, I would simplify my life and parse the --snapshot=dontquiesce flag to the Windows machine in a separate backup job, that would ensure it's not quiesced and multiply your chances to take a snapshot without errors.

In case you do need to quiesce the VM, then you'll need to deepen into the matter by checking the event log in search of any flaw in any service (VMWare Tools, VSS, Virtual Disk, etc...) upon which quiescing depends. Most of the times, it's lack of room in the Windows guest that causes trouble, in some other cases some services like Exchange or MS SQL, which have their own shadowing services, might need to be patched or upgraded. Read this post:

https://33hops.com/esxi-snapshot-errors … tions.html

Offline

#9 2018-01-04 11:54:53

matz
Member
Registered: 2017-06-28
Posts: 29

Re: Cannot copy VMSD file details

I dont think this is snapshot problem. I add --snapshot=dontquiesce flag and I have same error: (only with this one machine, weird)

4oSKqKWVQWyQP3XIJ54wVg.png

[KFD-SERVER] CLXSIDF1 error: file transfer error: Segmentation fault
packet_write_wait: Connection to 79.137.71.94: Broken pipe

Last edited by matz (2018-01-04 11:55:24)

Offline

#10 2018-01-04 15:17:08

admin
Administrator
Registered: 2017-04-21
Posts: 1,366

Re: Cannot copy VMSD file details

We aren't able to see anything in the screenshot. The error below is an XSIDiff segmentation fault, which is very weird. I would rebuild that VM by cloning its disks via vmkfstools. If I were you I would turn them into thin provisioned disks, AKA sparse files, as they will occupy less space and will allow you to back them up faster too.

Cloning disk to thin format:

vmkfstools -i "/vmfs/volumes/Datastore/examplevm/examplevm.vmdk" "/vmfs/volumes/Datastore 2/newexamplevm/newexamplevm.vmdk" -d thin

I would also make sure that the newly generated disk is an exact multiple of 1M (1048576 bytes), which is the VMFS block size, by extending them to the closest 1M multiple.

vmkfstools -X 80000m /vmfs/volumes/storage1/win2000/win2000.vmdk

Offline

Board footer