#1 2019-07-17 17:26:26

tom_22
Member
Registered: 2019-07-15
Posts: 18

XSIDiff: Error writing struct offset_out position xxx

Hi,

when I run XSIDiff, I get below error messages (around 140).
Any idea how to fix it?

Thanks,
Tom

(c) XSIDiff 1.1.1.4 licensed:
Meta-data available, zero aware copy
Reading from file /vmfs/volumes/datastore1/XXX/XXX-flat.vmdk
Writing to socket, block size 16384
Connected to socket 5
Error writing struct info to SOCKET 5Set input file offset to 0 bytes
Error writing struct offset_out position 0
Error writing struct offset_out position 19922944
Error writing struct offset_out position 40894464
Error writing struct offset_out position 103809024
...
...
Error writing struct offset_out position 39596326912
Error writing struct offset_out position 40133197824
Error writing struct offset_out position 40670068736

Offline

#2 2019-07-17 18:27:04

admin
Administrator
Registered: 2017-04-21
Posts: 1,362

Re: XSIDiff: Error writing struct offset_out position xxx

That seems to be a simple broken pipe

Offline

#3 2019-07-17 21:59:15

tom_22
Member
Registered: 2019-07-15
Posts: 18

Re: XSIDiff: Error writing struct offset_out position xxx

Can this error be ignored?
As long as the packets are re-transmitted and no data is lost, I wouldn't bother too much, but still like to get rid of it.

These are the commands I use:

Server
"./xsidiff --source=ServerIP:33 --target=/vmfs/volumes/datastore1/XXX/XXX-flat.vmdk"

Client
"./xsidiff --source=/vmfs/volumes/datastore1/XXX/XXX-flat.vmdk --target= ServerIP:33"

I guess, nothing is missed with those commands?

Offline

#4 2019-07-18 11:47:31

admin
Administrator
Registered: 2017-04-21
Posts: 1,362

Re: XSIDiff: Error writing struct offset_out position xxx

I would not ignore it for obvious reasons.

XSIDiff, despite its name, is not a differential tool, it just copies data as is. It is zero aware, but it is only differential when used from within Onediff.

When you use it you are copying all data each time. If you get a broken pipe condition in the middle of a copy, restart the job.

Offline

#5 2019-07-18 13:00:05

tom_22
Member
Registered: 2019-07-15
Posts: 18

Re: XSIDiff: Error writing struct offset_out position xxx

Restarting the job doesn't seem to be right solution: The error messages are literally flushed in one go, then the copy-process starts. This is why you could think, the tool is recovering from those errors.

Now, this is the situation, how we see it:
We have tried a 10GB file and it worked fine.
We bought the license to have XSIDiff/XSIBackup working with larger files.
With a file of around 40GB we get "Error writing struct".
We now tried it on a 120GB file and we get "Segmentation fault".

The instructions "https://33hops.com/xsidiff-man-page.html" were followed and it doesn't work. No other documents with any requirements or limitations were provided.

Other tools work, so it seems to be a problem with XSIDiff - it doesn't work as advertised.
Should we return the license?

Offline

#6 2019-07-18 15:30:12

admin
Administrator
Registered: 2017-04-21
Posts: 1,362

Re: XSIDiff: Error writing struct offset_out position xxx

XSIDiff is not sold alone, it is a binary bundled with XSIBACKUP-PRO, it works when called from XSIBACKUP-PRO. You didn't buy XSIDiff as we don't sell it. Use XSIBACKUP-PRO as described in the program's documentation.

Offline

#7 2019-07-18 16:47:31

tom_22
Member
Registered: 2019-07-15
Posts: 18

Re: XSIDiff: Error writing struct offset_out position xxx

Correct. I had to purchase XSIBACKUP-PRO in order to get the XSIDiff license. As it is not a product which you can split, I have to return the entirety.

We do follow the documentation in terms of intended use and syntax:

"https://33hops.com/xsibackup-using-backup-programs.html" (Intention)

... (c) XSIDiff is a standalone binary that requires its own license key and that may be used separately to copy files manually between datastores, different ESXi hosts or from an ESXi host to a Linux system. ...

"https://33hops.com/xsidiff-man-page.html" (Syntax)

IP Server :
./xsidiff --source=10.0.0.1:33 --target=[file2]
IP Client :
./xsidiff --source=[file1] --target=10.0.0.1:33

Unfortunately, this returns a "Segmentation fault"

If there is no way to fix it, what else would you suggest than reimbursement?

Offline

#8 2019-07-19 19:06:08

admin
Administrator
Registered: 2017-04-21
Posts: 1,362

Re: XSIDiff: Error writing struct offset_out position xxx

We have tried your command in the very same scenario with big files: 50GB and 100GB and we don't get any write error. Debug your network and make sure that it is working properly. XSIDiff will throw those errors when it can't write data to the socket for some reason, the most common one is some network problem.

Offline

#9 2019-07-28 14:45:45

tom_22
Member
Registered: 2019-07-15
Posts: 18

Re: XSIDiff: Error writing struct offset_out position xxx

Copied 120GB file through terminal (cp) to NFS share, from multiple host. Network works properly. The Switch doesn't report any lost packet or other issues. No issues with the network whatsoever.

'Segmentation fault' is a memory 'access violation'.
'Error writing struct info to SOCKET' is related to the socket, which was created by XSIDiff.

Yes, looks like an issue while data written into the _local_ socket. What are the socket options, while creation?
The offset positions don't look random. What are buffer-/cache-size, XSIDiff is working with?

A variable overflow would makes sense, as it explains as well the 'Segmentation Fault' error.

Offline

#10 2019-07-29 08:28:46

admin
Administrator
Registered: 2017-04-21
Posts: 1,362

Re: XSIDiff: Error writing struct offset_out position xxx

You are duplicating your posts, that won't get you more support.
Since the moment that you consider that using an Atom processor should not hurt SSH performance and you are not even open to reconsider your position, I'm afraid there's not much we can do for you.

Offline

#11 2019-07-29 15:20:05

tom_22
Member
Registered: 2019-07-15
Posts: 18

Re: XSIDiff: Error writing struct offset_out position xxx

Yes, there shouldn't be any cross-posting. This wasn't meant to happen.

We do consider your suggestions seriously, therefore we only ask question about the errors which we observe.
In the end, we are happy, when you can provide support in order to make XSIDiff work in client/server mode.

Offline

Board footer