©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#1 Re: © OneDiff » ERROR: cannot locate .ERR file at ... » 2019-08-07 13:46:55

The source VMFS version is 5.60, the target VMFS version is 6.82. Looks like a mismatch.
Do we need an exact version match or should the major number be fine?

Should it work with VMFS (local host) towards NFS (remote host), or NFS<->NFS?

#2 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-08-07 08:26:59

ok. I sent it to "info@33hops.com" and "support@33hops.com".
Please let me know if there is another dedicated address.

#3 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-08-05 12:42:01

We have cloned the disk with vmkfstools and tried again. Unfortunately, still getting the "segmentation fault" error.

Anything else we could do/try?
What VMWare version are you using?

#4 © OneDiff » ERROR: cannot locate .ERR file at ... » 2019-08-05 12:35:36

tom_22
Replies: 4

On the first run of XSIBACKUP-PRO with OneDiff, there are messages, which we don't know how to treat (errors?). Still, the VM seems to be copied fine.

...
[SBS_2011] info: backup complete
[SBS_2011] info: finished doing a first copy in a diff scope, subsequent backups will be differential
[SBS_2011] info: should you need to use the mirror VM, register it manually at the backup ESXi host
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The ESXi configuration was saved to "192.168.0.48:22:/vmfs/volumes/datastore1"
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sh: bad number
sh: bad number
sh: bad number
Backup finished
Tip: no chained backups scheduled, set --on-success and/or --on-error arguments to chain a backup
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Killed
~ # 

The second run of XSIBACKUP-PRO with OneDiff, within 10 minutes after the first run finished, raises errors:

...
[SBS_2011] info: (source) first 500M hash is: fd528979c6cac3b920b8967b10042e27630226c9
[SBS_2011] info: (target) first 500M hash is: a4f29011274816f08323ce8554e5bb94ab988e44
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[SBS_2011] error DIFQMSH4: first 500M mistmatch...
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ERROR: cannot locate .ERR file at [/vmfs/volumes/datastore1/xsi-dir/.ERR-cae3f449bce288c854b1f45ac159a736b26c74e9]
[SBS_2011] measure DIFRMVMX: took measure, renamed remote .vmx file to reinitialize OneDiff
ERROR: cannot locate .ERR file at [/vmfs/volumes/datastore1/xsi-dir/.ERR-cae3f449bce288c854b1f45ac159a736b26c74e9]
The ESXi configuration was saved to "192.168.0.48:22:/vmfs/volumes/datastore1"
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
sh: bad number
sh: bad number
sh: bad number
Backup finished
Tip: no chained backups scheduled, set --on-success and/or --on-error arguments to chain a backup
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Killed
~ #

What else we have done/observer:

  • file/folder permissions should be fine ("chmod -R 0700 src xsibackup EULA bin conf");

  • logged-in and executed as root;

  • there is an .ERR -file in the folder, however, the file-name/ID is different;

  • the versions of the source/target VMWare are different, 5.5 vs. 6.7 (--> do we need identical built-versions on both machines?);

  • obviously, the next run of XSIBACKUP-PRO with OneDiff copies over the entire machine again, not just a differential part;

Have we missed anything?

#5 Re: © XSITools » XSIDiff: Error writing struct offset_out position xxx » 2019-07-29 15:20:05

Yes, there shouldn't be any cross-posting. This wasn't meant to happen.

We do consider your suggestions seriously, therefore we only ask question about the errors which we observe.
In the end, we are happy, when you can provide support in order to make XSIDiff work in client/server mode.

#6 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-29 14:52:55

The NFS Servers run with V3. V4 is disabled.

The (edit: 'relevant') files are all stored on the local datastore, so they are not on mounted (NFS) folders.
The one in question is 128GB large. We have tried another file with 100GB, but unfortunately, when XSDiff is launched on that file, it also throws an 'Segmentation fault' error. (for the sake of completeness: the file with 40GB doesn't result in 'Segmentation fault' )

All vmware servers were rebooted as well.
Also checked the permissions on the files.
Tried on different versions of VMware. Which version are you using?

#7 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-28 15:11:27

https://33hops.com/xsidiff-vmware-esxi-file-replication.html wrote:

© XSIDiff is as fast as vmkfstools and can copy files over IP.
...
All registered XSIBACKUP-PRO users can request a free license of © XSIDiff to be used as a separate binary tool to copy -flat.vmdk files manually. From the day of launch of XSIBACKUP-PRO 10 (sched. aug 2017), © XSIDiff will be used as a core binary, and may be used separately too. A binary license.key file is necessary to operate with files bigger than 10 gb.

As posted in the other thread: the network looks fine. Copied 120GB file through terminal (cp) to NFS share, from multiple host. We are using Meraki, CISCO and HP-E (as backup) devices. The Switches don't report any lost packet or other issues. No other issues with the network whatsoever. No users reported issues with the network. All other applications work fine.
What other things could be verified in order to see, if there is an issue with the network?

Generally ,'Segmentation fault' is a memory 'access violation' caused/raised by XSDiff.
'Error writing struct info to SOCKET' is related to the socket, which was created by XSIDiff.

We cannot debug the socket, as it is created by XSDiff, but it looks like an overflow.

#8 Re: © XSITools » XSIDiff: Error writing struct offset_out position xxx » 2019-07-28 14:45:45

Copied 120GB file through terminal (cp) to NFS share, from multiple host. Network works properly. The Switch doesn't report any lost packet or other issues. No issues with the network whatsoever.

'Segmentation fault' is a memory 'access violation'.
'Error writing struct info to SOCKET' is related to the socket, which was created by XSIDiff.

Yes, looks like an issue while data written into the _local_ socket. What are the socket options, while creation?
The offset positions don't look random. What are buffer-/cache-size, XSIDiff is working with?

A variable overflow would makes sense, as it explains as well the 'Segmentation Fault' error.

#9 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-18 16:56:03

XSIDiff man page

(c) XSIDiff Man Page:
...
--source
Sets the file to sync/ copy, stdin is allowed. --source can be a local path, stdin or an IP:port combination in case of acting as an IP server.
--target
Sets the destination where the data is to be sent. --target can be a local path, stdout or an IP:port when acting as an IP client.
...

USAGE:
...
IP Server :
./xsidiff --source=10.0.0.1:33 --target=[file2]
IP Client :
./xsidiff --source=[file1] --target=10.0.0.1:33

EXAMPLES:
...
Set server to listen at local IP 192.168.0.100 port 33, copy receiving file to [file2] Once the server is listening run the client setting [file1] as source and point it to the server

./xsidiff --source=192.168.0.100:33 --target=[file2]

Run client when server is listening
./xsidiff --source=[file1] --target=10.0.0.1:33

My take on this situation: we shouldn't quarrel, we should find a solution.

#10 Re: © XSITools » XSIDiff: Error writing struct offset_out position xxx » 2019-07-18 16:47:31

Correct. I had to purchase XSIBACKUP-PRO in order to get the XSIDiff license. As it is not a product which you can split, I have to return the entirety.

We do follow the documentation in terms of intended use and syntax:

"(c)XSIBackup: using backup programs" (Intention)

... (c) XSIDiff is a standalone binary that requires its own license key and that may be used separately to copy files manually between datastores, different ESXi hosts or from an ESXi host to a Linux system. ...

"XSIDiff man page" (Syntax)

IP Server :
./xsidiff --source=10.0.0.1:33 --target=[file2]
IP Client :
./xsidiff --source=[file1] --target=10.0.0.1:33

Unfortunately, this returns a "Segmentation fault"

#11 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-18 13:11:20

Well, you haven't provided a solution or support on how to make it run as advertised.

You mentioned security standards, that's why you don't load a TCP server.
On the other hand, rsync is loaded that way as client/server. This seems contradictory to us.

We still can't get the tools working (please see other post) as advertised. How to conclude? Should we return the license?

#12 Re: © XSITools » XSIDiff: Error writing struct offset_out position xxx » 2019-07-18 13:00:05

Restarting the job doesn't seem to be right solution: The error messages are literally flushed in one go, then the copy-process starts. This is why you could think, the tool is recovering from those errors.

Now, this is the situation, how we see it:
We have tried a 10GB file and it worked fine.
We bought the license to have XSIDiff/XSIBackup working with larger files.
With a file of around 40GB we get "Error writing struct".
We now tried it on a 120GB file and we get "Segmentation fault".

The instructions "XSIDiff man page" were followed and it doesn't work. No other documents with any requirements or limitations were provided.

#13 Re: © XSITools » XSIDiff: Error writing struct offset_out position xxx » 2019-07-17 21:59:15

Can this error be ignored?
As long as the packets are re-transmitted and no data is lost, I wouldn't bother too much, but still like to get rid of it.

These are the commands I use:

Server
"./xsidiff --source=ServerIP:33 --target=/vmfs/volumes/datastore1/XXX/XXX-flat.vmdk"

Client
"./xsidiff --source=/vmfs/volumes/datastore1/XXX/XXX-flat.vmdk --target= ServerIP:33"

I guess, nothing is missed with those commands?

#14 © XSITools » XSIDiff: Error writing struct offset_out position xxx » 2019-07-17 17:26:26

tom_22
Replies: 10

Hi,

when I run XSIDiff, I get below error messages (around 140).
Any idea how to fix it?

Thanks,
Tom

(c) XSIDiff 1.1.1.4 licensed:
Meta-data available, zero aware copy
Reading from file /vmfs/volumes/datastore1/XXX/XXX-flat.vmdk
Writing to socket, block size 16384
Connected to socket 5
Error writing struct info to SOCKET 5Set input file offset to 0 bytes
Error writing struct offset_out position 0
Error writing struct offset_out position 19922944
Error writing struct offset_out position 40894464
Error writing struct offset_out position 103809024
...
...
Error writing struct offset_out position 39596326912
Error writing struct offset_out position 40133197824
Error writing struct offset_out position 40670068736

#15 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-17 12:49:49

Agree, we should stay with the facts: if the CPU is the bottleneck, the numbers should be different.
Please be aware as well, that I haven't written "bad implementation" nor did I expressed this in any other way.

It's advertised, that XSDiff is outperforming rsync. I can reproduce this, if XSDiff is run in client/server mode. The CPUs stay almost in IDLE.
However, XSDiff is slow -even slower than rsync- when used as backup-prog option in XSIBackup, as it is run through a SSH tunnel.

There is a better approach/algorithm in order to achieve, what is advertised. You actually already do this with rsync.

The relevant lines in the script -please correct if wrong- might be this

[XSDIFF]
RSYNCOUT=$( ("$PWD/bin/xsidiff" --source="${flatorigin}" --target=stdout "$XSIDIFFSILENT" | eval ssh "$SSHOPTS" -p$baksrvport ${defremusr}@$baksrvaddr "\"${RMXSIDFPATH//\ /\\ }\" --source=stdin --target=\"${flattarget//\ /\\ }\"" >&16) 2>&1 )

and this

[RSYNC]
RSYNCOUT=$( ("$PWD/bin/xsibackup-rsync" -rlpDv --progress ${rsyncoptions} --whole-file --sparse --rsh="ssh $SSHOPTS -p$baksrvport" --rsync-path="${RMRSYNCPATH}" "$flatorigin" ${defremusr}@$baksrvaddr:"${flattarget}" >&16) 2>&1 )

The rsync command line gives a blueprint (--rsh (-e) option ) of the steps of how to run the XSIDiff command in the very fast client/server mode:

// 1) fire up XSIDiff as server on remote host
ssh name@host "./xsidiff --source=server_IP:33 --target=[file2]"

// 2) run XSIDiff to do the copy/transfer
./xsidiff --source=[file1] --target=server_IP:33

// 3) kill XSIDiff server on remote host
ssh name@host "kill $xsidiff_PID"

What is your thought?

#16 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-16 22:12:24

Well, I don't agree.

XSIDiff works in client/server mode without performance issues. The CPUs almost IDLE.
It's just the implementation into XSIBackup, which is not appropriate. But even then, the CPU/Process values don't peak. The process is around 80% (%USED), but could go well beyond 100% because of dynamic CPU clocking. So one CPU-core is under noticeable workload, but not max'ed out.

There is no indication [(c)XSIBackup Classic: using backup programs], that XSIDiff, when used with XSIBackup, is run in a way, so it parse the results from stdin/stdout through ssh & encryption to the other host. How can it then seriously "overtake the initial transfers made by means of Rsync so far"?
Actually, I just tried '--backup-prog=rsync' - it's even faster (31.10MB/s).

Could you please have another look into XSIBackup, so it'll make full use of the advertised features of XSIDiff as a 'backup-prog' option.
I bought the license a few days ago and currently feel a bit disappointed.

#17 Re: General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-16 13:19:25

I have used esxtop on both hosts.

Generally, the source host is a C2750 Avoton (8-core), the target host is a C3558 Denverton (4-core).

With just XSIDiff executing in client/server mode, the CPU load average on the source is about 0.14. There are as well 4 VMs running (IDLE). So, I would say hardly no load. This wasn't an issue anyway.

With 'XSIBackup -backup-prog XSIDiff' the CPU load average is about 0.20, the PCPU AVG between 25% and 35%. I've observed a short peak of 68% of one single core, which is probably related to some VM activity.
The target host has a CPU load average of 0.27 and a PCPU AVG of about 30%

The SSH Process takes around 80% (%USED) on the source and around 50% (%USED) on the target. Not sure if this yet is a sign of overload?

I'll have a look at XSIBackup-Datacenter, but would like to get XSIBackup & XSIDiff well performing, as I just bought the license...

#18 General matters » using XSIDiff in client/server mode as XSIBackup backup-prog » 2019-07-15 13:11:28

tom_22
Replies: 18

Hi,

Is there a way to use XSIDiff in client/server mode as backu-prog for XSIBackup?

If I use XSIDiff in client/server mode, it's very fast. It seems the network link (1Gb) is the limiting device. A 40GB vmdk file takes 207 seconds (197.87 mb/s) to be copied between 2 ESXi Servers.

However, using XSIDiff as backup-prog for XSIBackup, it appears slow. The same 40GB vmdk file takes 1664 seconds (24.62 mb/s), even with the suggested ":l" option applied. I assume the ssh-overhead takes its toll.

Does anyone know how to run XSIDiff in client/server mode for XSIBackup or how to speed up things?

Thanks,
Tom

Board footer