You are not logged in.
Good morning.
In my first test of DC to replicate a VM over IP, it seems to be creating a thick version of the vm at the remote site. Looking at the docs it seems that this should not be the case and wonder where I made a mistake.
the replica command being used:
./xsibackup --replica "VMs(Radius)" root@X.X.X.X:22:/volume1/backup --ssh-ciphers=aes128-ctr,aes192-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
so far my progress is:
Backup start date: 2020-06-14 13:42:37
-----------------------------------------------------------------------------------------------------------
2020-06-14 13:42:37 | Backing up 15 files, total size is 64425634867
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/15 Radius-flat.vmdk 60.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 55.37% done | block 34022 out of 61440 | Done 55.37%
looking at the source flat file specifically with du:
15.4G Radius-flat.vmdk
looking st the current destination flat file:
34G Radius-flat.vmdk.tmp
What am i missing here? Thank you.
Offline
We believe that has more to do with your remote OS, FS and du implementation than with DC binary. What it does is to create an empty sparse file, then it fills it with the non-zero data.
Even if you don't host your VM in a VMFS volume, (c)XSIBackup-DC will still jump over zeros in the replica, thus the resulting replicated -flat.vmdk file is always a sparse (thin in (c)VMWare's terminology) file.
Try to run this alternative command and see if that returns the non-zero data on the leftmost column.
ls -lash Radius-flat.vmdk
Offline
Sadly ls gave the same result. Also judging on data speed and time taken I believe it transferred the whole amount:
60G -rw-r--r-- 1 admin root 60G Jun 15 22:21 Radius-flat.vmdk
More troubling and maybe something indicating a different issue my setup has that is causing this, the replica vmdk seems to be unusable. I brought that replica online in the remote esxi environment and the disk was indicated to be unformatted. I am unsure if this would really work, but i put the vmdk-flat file through less it seems blank.
When I do the same command but --backup instead of --replica, the backup goes very fast, and creates an extremely small repository. I was able to get the same result over ssh or local NFS mount. Doing a restore of the backup creates a blank 60 GB vmdk as well.
I have included the output of various commands below to help in figuring out what is going wrong? These commands were directed at a much smaller VM for quicker testing - thin provisioned and had the same result. The host is running ESXi 6.7.0 (Build 8169922).
/vmfs/volumes/SSD/XSIBackup-DC/xsibackup --replica "VMs(Auvik)" root@X.X.X:22:/volume1/vertical --ssh-ciphers=aes128-ctr
replica:
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
SSH: Mon Jun 15 21:33:06 EDT 2020
-----------------------------------------------------------------------------------------------------------
Remote system: linux
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2100191, Running job as: root
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC replicating data to /volume1/vertical
-----------------------------------------------------------------------------------------------------------
Virtual Machine Name: Auvik
-----------------------------------------------------------------------------------------------------------
Creating snapshot VM : Auvik (powered on)
-----------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-----------------------------------------------------------------------------------------------------------
New backup/replica: Auvik | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-16 01:44:28
-----------------------------------------------------------------------------------------------------------
2020-06-16 01:44:28 | Backing up 24 files, total size is 10872008991
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/24 Auvik-flat.vmdk 8.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
2/24 Auvik.vmdk 565.00 B | Done 79.01%
-----------------------------------------------------------------------------------------------------------
3/24 Auvik_1-flat.vmdk 1024.00 KB | Done 79.01%
-----------------------------------------------------------------------------------------------------------
4/24 Auvik_1.vmdk 560.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
5/24 Auvik.nvram 8.48 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
6/24 vmware-8.log 201.22 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
7/24 vmware-9.log 196.03 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
8/24 Auvik.vmsd.tmp 43.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
9/24 Auvik.vmx.tmp 2.53 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
10/24 vmware-10.log 193.21 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
11/24 vmware-11.log 116.99 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
12/24 vmware-12.log 223.09 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
13/24 vmx-Auvik-3446253671-1.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
14/24 Auvik.vmx.lck [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
15/24 vmware-13.log 168.58 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
16/24 vmware.log 230.47 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
17/24 Auvik-cd69a867.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
18/24 Auvik-Snapshot4.vmsn 19.16 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
19/24 Auvik-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
20/24 Auvik-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
21/24 Auvik_1-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
22/24 Auvik_1-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
23/24 Auvik.vmx 2.54 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
24/24 Auvik.vmsd 501.00 B | Done 79.03%
-----------------------------------------------------------------------------------------------------------
Total size: 8194 MB | Done 100.00%
-----------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-----------------------------------------------------------------------------------------------------------
Backup end date: 2020-06-16 02:03:50
-----------------------------------------------------------------------------------------------------------
Time taken: 1162 sec.
-----------------------------------------------------------------------------------------------------------
Total time: 1162 sec.
-----------------------------------------------------------------------------------------------------------
Full file speed: 7.05 mb/s
-----------------------------------------------------------------------------------------------------------
Real data speed: 7.92 mb/s
-----------------------------------------------------------------------------------------------------------
Item backup completed
-----------------------------------------------------------------------------------------------------------
Data processing completed successfully
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup OK
-----------------------------------------------------------------------------------------------------------
SSH session was closed OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------
source:
[root@esxi2:/vmfs/volumes/662879e1-2d0c7453-0000-000000000000/Auvik] du -ksh Auvik-flat.vmdk
4.1G Auvik-flat.vmdk
[root@esxi2:/vmfs/volumes/662879e1-2d0c7453-0000-000000000000/Auvik] ls -lash Auvik-flat.vmdk
4290308 -rw------- 1 root root 8.0G Jun 16 02:33 Auvik-flat.vmdk
dest:
sh-4.3# du -ksh Auvik-flat.vmdk
8.0G Auvik-flat.vmdk
sh-4.3# ls -lash Auvik-flat.vmdk
8.0G -rw-r--r-- 1 root root 8.0G Jun 15 21:56 Auvik-flat.vmdk
backup:
[root@esxi2:~] /vmfs/volumes/SSD/XSIBackup-DC/xsibackup --backup "VMs(Auvik)" root@X.X.X:22:/volume1/vertical/xsi --ssh-ciphers=aes128-ctr
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
SSH: Mon Jun 15 21:53:01 EDT 2020
-----------------------------------------------------------------------------------------------------------
Remote system: linux
-----------------------------------------------------------------------------------------------------------
User defined ciphers: aes128-ctr
-----------------------------------------------------------------------------------------------------------
PID: 2100963, Running job as: root
-----------------------------------------------------------------------------------------------------------
LZJB compression has been enabled
-----------------------------------------------------------------------------------------------------------
Block size is 1048576 bytes (1024.00 KB)
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC setting repository at /volume1/vertical/xsi
-----------------------------------------------------------------------------------------------------------
.xsitools file created at: root@X.X.X.X:22:/volume1/vertical/xsi/.xsitools
-----------------------------------------------------------------------------------------------------------
Virtual Machine Name: Auvik
-----------------------------------------------------------------------------------------------------------
Creating snapshot VM : Auvik (powered on)
-----------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-----------------------------------------------------------------------------------------------------------
New backup/replica: Auvik | folder
-----------------------------------------------------------------------------------------------------------
Backup start date: 2020-06-16 02:04:27
-----------------------------------------------------------------------------------------------------------
2020-06-16 02:04:27 | Backing up 24 files, total size is 10872042917
-----------------------------------------------------------------------------------------------------------
NUMBER FILE SIZE PROGRESS
-----------------------------------------------------------------------------------------------------------
1/24 Auvik-flat.vmdk 8.00 GB | Done 0.00%
-----------------------------------------------------------------------------------------------------------
::: detail ::: 100.00% done | block 8192 out of 8192 | Done 79.01%
-----------------------------------------------------------------------------------------------------------
2/24 Auvik.vmdk 565.00 B | Done 79.01%
-----------------------------------------------------------------------------------------------------------
3/24 Auvik_1-flat.vmdk 1024.00 KB | Done 79.01%
-----------------------------------------------------------------------------------------------------------
4/24 Auvik_1.vmdk 560.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
5/24 Auvik.nvram 8.48 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
6/24 vmware-8.log 201.22 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
7/24 vmware-9.log 196.03 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
8/24 Auvik.vmsd.tmp 43.00 B | Done 79.02%
-----------------------------------------------------------------------------------------------------------
9/24 Auvik.vmx.tmp 2.53 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
10/24 vmware-10.log 193.21 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
11/24 vmware-11.log 116.99 KB | Done 79.02%
-----------------------------------------------------------------------------------------------------------
12/24 vmware-12.log 223.09 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
13/24 vmx-Auvik-3446253671-1.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
14/24 Auvik.vmx.lck [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
15/24 vmware-13.log 168.58 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
16/24 vmware.log 263.37 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
17/24 Auvik-cd69a867.vswp [open excluded]| Done 79.03%
-----------------------------------------------------------------------------------------------------------
18/24 Auvik-Snapshot5.vmsn 19.16 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
19/24 Auvik-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
20/24 Auvik-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
21/24 Auvik_1-000001-delta.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
22/24 Auvik_1-000001.vmdk [skipped excluded]
-----------------------------------------------------------------------------------------------------------
23/24 Auvik.vmx 2.54 KB | Done 79.03%
-----------------------------------------------------------------------------------------------------------
24/24 Auvik.vmsd 500.00 B | Done 79.03%
-----------------------------------------------------------------------------------------------------------
Total size: 8194 MB | Done 100.00%
-----------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-----------------------------------------------------------------------------------------------------------
Backup end date: 2020-06-16 02:04:46
-----------------------------------------------------------------------------------------------------------
Time taken: 19 sec.
-----------------------------------------------------------------------------------------------------------
Total time: 19 sec.
-----------------------------------------------------------------------------------------------------------
Full file speed: 431.28 mb/s
-----------------------------------------------------------------------------------------------------------
Real data speed: 2.36 mb/s
-----------------------------------------------------------------------------------------------------------
Item backup completed
-----------------------------------------------------------------------------------------------------------
Differential blocks were added to the .blocklog database
-----------------------------------------------------------------------------------------------------------
Data processing completed successfully
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup OK
-----------------------------------------------------------------------------------------------------------
SSH session was closed OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------
destination repo info:
sh-4.3# xsibackup --info /volume1/vertical/xsi
Missing SSL key pair, generating...
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
||| (c)XSIBackup-DC 1.3.0.1: Backup & Replication Software |||
||| (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
(c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: Linux, Kernel 4 Major 4 Minor 59 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
PID: 30509, Running job as: root
-----------------------------------------------------------------------------------------------------------
Deduplicated repository information
-----------------------------------------------------------------------------------------------------------
Compression: 1
Block size: 1048576
Block count: 15
-----------------------------------------------------------------------------------------------------------
Retrieving block information...
-----------------------------------------------------------------------------------------------------------
| Real data | Hosted data |
-----------------------------------------------------------------------------------------------------------
Block number | 15 | 8208 |
Data volume | 362.69 KB | 8.00 GB |
Compression | 100.00% | - |
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir OK
-----------------------------------------------------------------------------------------------------------
Removed PID OK
-----------------------------------------------------------------------------------------------------------
Thanks for the help - I really would love to get this to work.
Offline
Sparseness depends on the OS kernel and FS, nothing to do with (c)XSIBackup-DC.
15 blocks is same as nothing.
All you are describing is really awkward. Please contact support for some personalized help, we would need to reproduce your issue by using the same target OS. This doesn't look like something that could be generalized or help others until we know the details of the target OS.
Offline
Thank you. Since I have not purchased yet at this point what would be the best point of contact for support? I see a contact form on the website is that what I should use? Thank you.
Offline