Posts by lievenmoors

lievenmoors · General matters

Well, it's not a different place, but another name for the same place.
The xxx was used to hide the scratch volume id.

lievenmoors · General matters

I have the problem that that the xsibackup tmp folder is not being cleaned up
correctly after a backup.

This is my job command:

/vmfs/volumes/xxx/xsi/xsibackup \
--backup \
"VMs(VM1, VM2, VM3)" \
"root@my_ip:22:/srv/xsibackup_repo/" \
--remote-path=/my_home/xsi/xsibackup \
--rotate=25 \
--compression=yes \
--mail-to=backups@jes.be \
>> /vmfs/volumes/xxx/xsi/var/log/xsibackup.log 2>&1

Xsibackup is installed in the /scratch directory. Xsibackup is running as root.
When the job has finished, it leaves stuff in the /scratch/xsi/tmp directory,
and after a while the backups fail because it is full.

I emptied the directory three days ago, and now it looks like this:

# ls -ld /scratch/xsi/tmp/*
drwxr-xr-x    1 root     root             8 Apr 28 10:15 /scratch/xsi/tmp/2923915
drwxr-xr-x    1 root     root             8 Apr 29 10:14 /scratch/xsi/tmp/2981378

du -sh /scratch/xsi/tmp/*
322.8M  /scratch/xsi/tmp/2923915
325.1M  /scratch/xsi/tmp/2981378

This is the from the log file. It shows the end of the backup that left behind
the folder 2923915

Backup end date: 2020-04-28 11:36:43
-----------------------------------------------------------------------------------------------------------
Time taken:     4849 sec.
-----------------------------------------------------------------------------------------------------------
Total time:    45274 sec.
-----------------------------------------------------------------------------------------------------------
Full file speed:                                                                            35.06 mb/s
-----------------------------------------------------------------------------------------------------------
Real data speed:                                                                            21.10 mb/s
-----------------------------------------------------------------------------------------------------------
Item backup completed
-----------------------------------------------------------------------------------------------------------
Differential blocks were added to the .blocklog database
-----------------------------------------------------------------------------------------------------------
Data processing completed successfully
-----------------------------------------------------------------------------------------------------------
Could not find SMTP -1, falling back to SMTP 1
-----------------------------------------------------------------------------------------------------------
Sending e-mail to my_email...
-----------------------------------------------------------------------------------------------------------
E-mail was sent          OK
-----------------------------------------------------------------------------------------------------------
--rotate option was detected, retrieving backups to prune...
-----------------------------------------------------------------------------------------------------------
Nothing to prune as per the rotation figure
-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir        OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup          OK
-----------------------------------------------------------------------------------------------------------
SSH session was closed   OK
-----------------------------------------------------------------------------------------------------------
Removed PID              OK

Thanks for your help,

Greetings,

lieven

lievenmoors · General matters

When trying to use --prune instead of --rotate, I get a little more output:

./xsibackup --prune /vmfs/volumes/xsibackup/xsibackup_repo/20200218122926
/
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.1.1.0: Backup & Replication Software                    |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 0 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: unlicensed trial version
-----------------------------------------------------------------------------------------------------------
PID: 1479730, Running job as: root
-----------------------------------------------------------------------------------------------------------
Finding blocks to prune, please wait...
-----------------------------------------------------------------------------------------------------------
Getting map files from repo...
-----------------------------------------------------------------------------------------------------------
Retrieving general block data 100.00%
-----------------------------------------------------------------------------------------------------------
Please wait while we order the data at: /vmfs/volumes/59397c00-97c01908-d68f-0cc47a91a3c8/xsi/tmp/0U.xsi
-----------------------------------------------------------------------------------------------------------
sort: out of memory
2020-04-01T12:23:27 | Error code 287 at file prune.c, line 287
Error description: something went wrong when ordering: /vmfs/volumes/59397c00-97c01908-d68f-0cc47a91a3c8/xsi/tmp/0U.xsi, count was: 0

-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir        OK
-----------------------------------------------------------------------------------------------------------
Removed PID              OK
-----------------------------------------------------------------------------------------------------------

lievenmoors · General matters

Xsibackup is running on VMware ESXi, and repository is on NFS 4.1 share,
provided by CentOS 8 system. The size of the repository is about 2TB.

lievenmoors · General matters

It also seems that everytime this happens, new files are left in /scratch/xsi/tmp.
This is probably the reason my /tmp directory was getting full after a while.

After deleting everything in /scratch/xsi/tmp, I tried to --prune one of the directories,
but I still get the same error.

Because pruning has been failing for quite some time now, I have 24 snapshots in
my xsibackup repository.

lievenmoors · General matters

Hi,

I'm always getting this error, when Xsibackup tries to prune my backups.
Any idea what's going wrong here?

Also, Xsibackup doesn't report this failure in the email that is sent after finishing the backup.

Thanks for your help.

Greetings,

lieven

Please wait while we order the data at: /vmfs/volumes/59397c00-97c01908-d68f-0cc47a91a3c8/xsi/tmp/0U.xsi
-----------------------------------------------------------------------------------------------------------
sort: out of memory
2020-03-24T10:41:33 | Error code 287 at file prune.c, line 287
Error description: something went wrong when ordering: /vmfs/volumes/59397c00-97c01908-d68f-0cc47a91a3c8/xsi/tmp/0U.xsi, count was: 0

lievenmoors · General matters

Ok I will do that...

Let me try to clarify things anyway.

I have two VMFS-5 filesystems

/vmfs/volumes/A
/vmfs/volumes/B

The VM I'm trying to backup has its own subfolder on A:

/vmfs/volumes/A/my_vm/

But that VM has second disk located on B.

/vmfs/volumes/B/disk2.vmdk

As you can see, it doesn't have a parent directory with the name of the VM.

Now I try to backup that VM with:

xsibackup /vmfs/volumes/A/my_vm /vmfs/volumes/B/my_backup_folder/

What happens is that xsibackup starts copying the WHOLE of /vmfs/volumes/B
to /vmfs/volumes/B/my_backup_folder.

And using the VMs() function does the same thing.

lievenmoors · General matters

I don't understand, but when I look in the backup folder I see this:

ls /vmfs/volumes/Backup/xsibackup-repo/20200203090947/

0                                    4                                    7                                    aqua                               e
1                                    5                                    8                                    b                                    f
2                                    593f...                            9                                     c                                    terra
3                                    6                                    a                                     d                                    xsibackup-repo

It seems that the subfolders from the data folder are being backed up to seperate folders.

I think there might be some recursion going on.

lievenmoors · General matters

This is the backup command I'm using:

/vmfs/volumes/5939.../xsi/xsibackup \
--backup \
"/vmfs/volumes/5939.../aqua" \
"/vmfs/volumes/593f.../xsibackup-repo/" \
--exclude="aqua_[12]-flat.vmdk" \
--block-size=50M \
--compression=yes \
--rotate=5 \
>> /vmfs/volumes/5939.../xsi/var/log/xsibackup.log 2>&1

One of the disks of the VM called "aqua" resides on the 593f... VMFS:

/vmfs/volumes/Backup/aqua_1.vmdk

I was expecting that Xsibackup would try to backup the VM, and all related disks.
But now I see that it is trying to backup EVERY folder on 593f.. VMFS.

And that includes the whole backup itself: /vmfs/volumes/593f.../xsibackup-repo/

Is this supposed to happen, or not?

Thanks for your help.

lieven

lievenmoors · General matters

Thanks for clarifying.

Greetings, lieven

lievenmoors · General matters

And now i missed your post again

lievenmoors · General matters

Sorry, I just missed your post.

Yes, I've read everything about the new default block size.
But I couldn't find anything on your website about compatibility between
xsibackup-dc and xsi-tools repositories.

I had used the same block size as was used in the xsitools repository, 50M,
but it seems xsibackup-dc is not reusing the data that was generated by xsitools,
and that's why I'm hitting the inode boundary. Otherwise I would be OK.

So my main question is now if xsibackup is supposed to be able to reuse those blocks, or not.

lievenmoors · General matters

My problem is now that I can't put my old xsi-tools repository aside, because of the inode problem.

Is there a way to make xsibackup-dc reuse the old repository?

I don't want to delete the old xsi-tools repository, before xsibackup-dc has made it's first backup.

lievenmoors · General matters

I hit the max number of inodes of vmfs-5.

But it seems that the data of the original xsi repository which was used by
the pro version with xsitools has not been reused somehow. The size of data
was 2TB, and after running the datacenter version, it nearly doubled in size.

I wish you had provided more information on your site about upgrading from pro to dc,
and if reusing the same repo is supposed to work.

Is it?

Thanks again,

lieven

lievenmoors · General matters

Now I removed the 20200129090426 directory, and tried again, with the same error:

2020-01-29T09:39:00 | Error code 2786 at file common.c, line 2786
Error description: can't open lock file: /vmfs/volumes/593f.../xsit-repo/20200129093853/terra/.locked
-----------------------------------------------------------------------------------------------------------
2020-01-29T09:39:00 | Error code 2397 at file xsibackup.c, line 2397
Error description: failed to lock the repo /vmfs/volumes/593f.../xsit-repo/20200129093853

lievenmoors · General matters

Hi,

I'm trying DC version for the first time but I'm experiencing a number of hiccups.

I started of with a failed backup because I had --compression, instead of --compression=yes :

2020-01-28T08:32:25 | Error code 1293 at file xsibackup.c, line 1293
Error description: compression value cannot be evaluated as true or false, you wrote: --block-size=50m

That was easily solved.

The next backup ran for a very long time, but then experienced another error,
probably due to a leftover from the previous failed backup:

(Note: I must admit I stopped a few backups with Ctrl-C while sorting out the license)

2020-01-29T06:46:05 | Error code 333 at file dedup-in.c, line 333
Error description: can't open temp block file: /vmfs/volumes/Backup/xsit-repo/data/3745876087575088.tmp

I didn't understand why it couldn't open that file, since there shouldn't be any permission problems running as
root, but I saw that there was another *.tmp file in the same directory.

I moved that other file out of the way, and tried again. And now I got this error:

2020-01-29T09:04:34 | Error code 2786 at file common.c, line 2786
Error description: can't open lock file: /vmfs/volumes/593f.../xsit-repo/20200129090426/terra/.locked
-----------------------------------------------------------------------------------------------------------
2020-01-29T09:04:35 | Error code 2397 at file xsibackup.c, line 2397
Error description: failed to lock the repo /vmfs/volumes/593f.../xsit-repo/20200129090426
Remove snapshot failed

In the end, the snapshot got removed anyhow...

So this is getting more messy every time.
Can you help me clean this up?

Thank you!

Greetings,

lieven

lievenmoors · © XSITools

Hi,

That was indeed the case. This was the output:

ps -c | grep xsi
2054591  2054591  sh                             /bin/sh /vmfs/volumes/datastore1/xsi-dir/xsibackup --backup-prog=xsitools:z --backup-point=/vmfs/volumes/Backup/xsit-repo --backup-type=Custom --backup-vms=xxx --backup-room=1900 --mail-to=x --use-smtp=1 --backup-how=Hot --backup-id=01 --description=backup all vms --exec=yes
2054592  2054592  sh                             /bin/sh /vmfs/volumes/datastore1/xsi-dir/xsibackup --backup-prog=xsitools:z --backup-point=/vmfs/volumes/Backup/xsit-repo --backup-type=Custom --backup-vms=xxx --backup-room=1900 --mail-to=x --use-smtp=1 --backup-how=Hot --backup-id=01 --description=backup all vms --exec=yes
2054598  2054598  grep                           grep xsi
1862327  1862327  sh                             /bin/sh -c "/vmfs/volumes/datastore1/xsi-dir/jobs/01"
1862330  1862330  busybox                        ash /vmfs/volumes/datastore1/xsi-dir/jobs/01
1862336  1862336  sh                             /bin/sh /vmfs/volumes/datastore1/xsi-dir/xsibackup --backup-prog=xsitools:z --backup-point=/vmfs/volumes/Backup/xsit-repo --backup-type=Custom --backup-vms=xxx --backup-room=1900 --mail-to=x --use-smtp=1 --backup-how=Hot --backup-id=01 --description=backup all vms --exec=yes
1965372  1965372  sh                             /bin/sh /vmfs/volumes/datastore1/xsi-dir/xsibackup --backup-prog=xsitools:z --backup-point=/vmfs/volumes/Backup/xsit-repo --backup-type=Custom --backup-vms=xxx --backup-room=1900 --mail-to=x --use-smtp=1 --backup-how=Hot --backup-id=01 --description=backup all vms --exec=yes

I have killed all those processes.
But I still don't know how they were spawned.
Do you have an idea?

Here is my cron configuration

cat conf/root-crontab
00 23 * * * "/vmfs/volumes/datastore1/xsi-dir/jobs/01"

ls /var/spool/cron/crontabs/
root

cat /var/spool/cron/crontabs/root 
#min hour day mon dow command
... (stuff not related to xsibackup)
00 23 * * * "/vmfs/volumes/datastore1/xsi-dir/jobs/01" # Added by XSIBackup

cat /etc/rc.local.d/local.sh 
#!/bin/sh

"/vmfs/volumes/datastore1/xsi-dir/src/cron-init" root
exit 0

lievenmoors · © XSITools

I get an error after which the backup of one in three VM's gets aborted with:

ERROR: cannot locate .ERR file at [/vmfs/volumes/datastore1/xsi-dir/.ERR-83d64b47e3d6103d7e158cc8b48a1c43c7097e1d]
2019-12-16T23:01:00|  [terra] CLXSITO1 error: Killed
2019-12-16T23:01:00|  sh: bad number

Below, you will find the xsibackup log output until that error.

Do you have an idea of what could have gone wrong here?

Thanks for your help,

lieven

---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:20|  ###############################################################################
2019-12-16T23:00:20|     XSIBACKUP-PRO 11.2.2: new execution request                     
2019-12-16T23:00:20|  ###############################################################################
2019-12-16T23:00:20|  
2019-12-16T23:00:28|  NOTICE: (c) XSIBackup kills any user launched jobs, make sure you don't overlap manual jobs
---------------------------------------------------------------------------------------------------------------------------------
XSIBackup PID:          1575299                                                 SocialServerJes
Mon, 16 Dec 2019 23:00:26 +0000                                    IPv4: X.X.X.X/255.255.0.0
VMware ESXi 6.0.0 build-5050593                              (c) Rsync 3.1.0 as opt. dependency
---------------------------------------------------------------------------------------------------------------------------------
Backup Id:                   01                       Intel(R) Xeon(R) CPU E3-1245 v5 @ 3.50GHz 
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:33|  Backup description: backup all vms
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:34|  Message: crontab is installed for user root
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:35|  Backup user is: root
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:35|  Backup program is: xsitools
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:36|  /vmfs/volumes/Backup/xsit-repo is an XSITools repo
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:46|  --backup-point found at /vmfs/volumes/Backup/xsit-repo (filesystem: VMFS-5)
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:50|  The backup room has been limited to 1900 gb.
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:53|  Getting list of all VMs...
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:53|  12     VM1                [datastore1] VM1/VM1.vmx                             centos64Guest     vmx-11              
2019-12-16T23:00:54|  3      Mikrotik Firewall   [datastore1] Mikrotik Firewall/Mikrotik Firewall.vmx   otherLinuxGuest   vmx-08              
2019-12-16T23:00:55|  8      VM2              [datastore1] VM2/VM2.vmx                           centos64Guest     vmx-11              
2019-12-16T23:00:56|  9      VM3               [datastore1] VM3/VM3.vmx                           centos64Guest     vmx-11              
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:58|  VMs to backup:
---------------------------------------------------------------------------------------------------------------------------------
2019-12-16T23:00:59|  12     VM1                [datastore1] VM1/VM1.vmx                             centos64Guest     vmx-11              
---------------------------------------------------------------------------------------------------------------------------------
ERROR: cannot locate .ERR file at [/vmfs/volumes/datastore1/xsi-dir/.ERR-83d64b47e3d6103d7e158cc8b48a1c43c7097e1d]
2019-12-16T23:01:00|  [VM3] CLXSITO1 error: Killed
2019-12-16T23:01:00|  sh: bad number
2019-12-16T23:01:02|  Excluded disks removed from backup .vmx file

...

lievenmoors · © XSITools

Ok, sorry for the fuzz. Everything went back to normal. Probably a hiccup in the datacenter...
Thanks for your quick support!

lievenmoors · © XSITools

Hi,

My Xsitools backups have been working flawlessly for months now.
Suddenly today, it's failing with the errors shown below.
Do you have any idea what is going wrong here?

Thanks for your help,

lieven

----

df -h

VMFS-5 3.9T 3.6T 347.7G 91% /vmfs/volumes/Backup
VMFS-5 1.9T 1.4T 448.2G 76% /vmfs/volumes/datastore1

----

cat jobs/01

"/vmfs/volumes/datastore1/xsi-dir/xsibackup" \
--backup-prog=xsitools:z \
--backup-point=/vmfs/volumes/Backup/xsit-repo \
--backup-type=Custom \
--backup-vms="terra!terra_1.vmdk,aqua,moles" \
--backup-room=2000 \
--mail-to=backups@jes.be \
--use-smtp=1 \
--backup-how=Hot \
--backup-id=01 \
--description="backup all vms" \
--exec=yes >> "/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log"

----

from var/logs/xsibackup.log

2019-08-19T10:31:40| [terra] CLXSITO1 error: /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: can't create /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
2019-08-19T10:31:40| mkdir: can't create directory '/vmfs/volumes/Backup/': Input/output error
2019-08-19T10:31:40| sed: /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
2019-08-19T10:31:40| /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: can't create /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
2019-08-19T10:31:40| mkdir: can't create directory '/vmfs/volumes/Backup/': Input/output error
2019-08-19T10:31:40| sed: /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
2019-08-19T10:31:40| /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: can't create /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
2019-08-19T10:31:40| mkdir: can't create directory '/vmfs/volumes/Backup/': Input/output error
2019-08-19T10:31:40| sed: /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
2019-08-19T10:31:40| /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: can't create /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
...And many more of these...

----

In the email sent xsibackup, there is more info:

Last warning for the above VM:
[terra] Info: disks [terra_1.vmdk] have been excluded from the backup
Available space in device /vmfs/volumes/Backup/xsit-repo after backup: 347 Gb.
Complete backup elapsed time: 698 min
The ESXi configuration was saved to /vmfs/volumes/Backup/xsit-repo/20190818230003
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: 28090: not found
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: 28091: not found
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: 28092: not found
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: 28093: not found
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: 28094: not found
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: 28095: not found
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: 28096: not found

...And many more of these...

• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: can't create /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
• /vmfs/volumes/datastore1/xsi-dir/xsibackup: line 155: can't create /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error

• [ Mon Aug 19 10:31:40 UTC 2019 ] ERROR (CLXSITO1), details [terra] error: backup error, details: /vmfs/ volumes/datastore1/xsi-dir/xsibackup: line 155: can't create /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
• [ Sun Aug 18 23:01:05 UTC 2019 ] ERROR (MKROOM01), details Error: cannot make 219G of room, only 178G can be made available

• mkdir: can't create directory '/vmfs/volumes/Backup/': Input/output error
• sed: /vmfs/volumes/Backup/xsit-repo/20190818230003/terra/terra_0-flat.vmdk: Input/output error
• [ Mon Aug 19 02:15:57 UTC 2019 ] WARNING (XSIPRU00), details The (c)XSITools repo was pruned, 87571 Mb were made available. The size of the repo was reduced to 2067832

• [ Mon Aug 19 02:27:55 UTC 2019 ] WARNING (XSIPRU00), details The (c)XSITools repo was pruned, 228348 Mb were made available. The size of the repo was reduced to 1839484

• [ Mon Aug 19 02:39:25 UTC 2019 ] WARNING (VMBDEXC1), details [terra] Info: disks [terra_1.vmdk] have been excluded from the backup

• [ Mon Aug 19 09:20:16 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:18 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:19 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:21 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:23 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/ volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:25 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:27 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:29 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:31 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: / vmfs/volumes/datastore1/xsi-dir

• [ Mon Aug 19 09:20:32 UTC 2019 ] WARNING (XSITBPWC), details Cannot process block 9429cca18262d9c1e5f210ce23e30b3c14cec059 (28091), this is just a warning, will try again. Warning: /vmfs/volumes/datastore1/xsi-dir

lievenmoors · © XSITools

Just to be sure I understand how the new version works...

So if I run this xsitools job repeatedly:

"/vmfs/volumes/datastore1/xsi-dir/xsibackup" --backup-prog=xsitools:z --backup-point=/vmfs/volumes/Backup/xsit-repo --backup-type=Custom --backup-vms="..." --backup-room=2000 --mail-to=... --use-smtp=1 --backup-how=Hot --backup-id=01 --description="..." --exec=yes >> "/vmfs/volumes/datastore1/xsi-dir/var/logs/xsibackup.log"

it will make room as needed within the --backup-point folder, by deleting the eldest folders (named like 20190102002335, as determined by mask), and by pruning the repository afterwards.

Do I understand this right?

lievenmoors · © XSITools

Could you explain if --backup-room will be able to make room within one repository, so the repository doesn't grow bigger than that. If I understood right, --backup-room used to delete older repositories when you had more than one xsitools repository. In other words, do I still need more than one repository, in order to make use of this feature?

lievenmoors · © XSITools

admin wrote:

If you only have room for one backup, then what you need is more room.
You can try to overcome it in any possible way, but facing the real issue is something that you can't escape.

Do you mean that there is a high enough risk that this backup isn't sane?
Does this have to do with possible hash collisions?

lievenmoors · © XSITools

I made a couple of changes to the script above.
The main changes are:

- Use -name '*-flat.vmdk' when looking for vmdk files containing hashes.
On the Xsibackup website, it is stated that only these vmdk files are deduplicated.

- Use sort -u, instead of sort + awk. I don't think awk is needed because we sort the input anyway.

- Use: find with -regex looking for blocks, instead of ls+grep,
and be stricter on the filename (don't use word boundaries).

- use `basename $0` instead of $0

Note: make sure you create the directory "var/logs/$name" or adapt it to your taste...

#!/bin/sh

# Check for inconsistencies in xsitools-repository
# Find and delete unused files
# Prune old backups

name=`basename $0`

{
echo "Begin: `date`"

# usage
if [ -e $1/.xsitools ]
  then
  echo "$1 seems to be an xsitools-Repository, using it."
  else
  echo "$1 doesn't seem to be an xsitools-Repository."
  echo "Use \"$name [xsitools-repo-directory]\""
  exit 1
fi

if [ "$2" != "--delete" ]
  then
  echo "Use \"$name [xsitools-repo-directory] [--delete]\" to remove unused files (be careful)."
  else
  echo "\"--delete\" is set, will remove unused files."
fi

if echo $3 | egrep -q '^[0-9]+$';
  then
  echo "Searching for backup-folders older than $3 days."
  bkpfolders=`find $1 -type d -maxdepth 1 -regex ".*/[0-9\-]\{14\}" -mtime +$3`
  if [ ! -z "$bkpfolders" ]
    then
    echo "$bkpfolders found, deleting"
    rm -rf $bkpfolders
    else
    echo "No backup-folders found."
  fi
  else
  echo "3rd option can be a number: Delete backup-folders older than ... days."
  echo "You can use this to prune older backups (be careful)."
fi

# Temporary files and variables
temp_dir=`mktemp -d -t`
hashes="$temp_dir/hashes"
hashes_sorted="$temp_dir/hashes_sorted"
files="$temp_dir/files"
files_sorted="$temp_dir/files_sorted"
delete_candidates="$temp_dir/delete_candidates"
missing_files="$temp_dir/missing_files"
diff_output="$temp_dir/diff_output"
hashes_count=0
files_count=0

echo "Collecting hashes of all .vmdk files."
# my old version to exclude delta files:
# find $1/ -path data -prune -o -name *.vmdk -maxdepth 3 | grep -v '\delta.vmdk$' | grep -v '\sesparse.vmdk$' | while read line; do cat "$line" ; done | grep -o '\b[0-9a-f]\{40\}\+\b' > $hashes
# wile-loop inserted for handling filenames with spaces, exclude delta files (snapshots), faster search (thanks to wowbagger)
# find $1/ -path $1/data -prune -o -name *.vmdk | grep -v '\delta.vmdk$' | grep -v '\sesparse.vmdk$' | grep -v $1/data | while read LINE; do cat "$LINE" ; done | grep -o '^\b[0-9a-f]\{40\}\+\b' > $hashes
find $1/ -path $1/data -prune -o -name '*-flat.vmdk' -exec cat {} \; > $hashes
echo "Sorting hashes and removing duplicates."
sort -u $hashes > $hashes_sorted
hashes_count=`cat $hashes_sorted | wc -l`
echo "Hashes in vmdks: $hashes_count"

echo "Generating list of files in ./data."
# find $1/data -type f -exec basename {} \; > $files
# ls -1R $1/data | grep -o '\b[0-9a-f]\{40\}\+\b' > $files

find $1/data -type f -regex '.*/[0-9a-f]\{40\}\+$' -exec basename {} \; > $files
echo "Sorting list of files."
sort $files > $files_sorted
files_count=`cat $files_sorted | wc -l`

echo "Files: $files_count"

# some checks if everything is valid
echo "Using diff for comparing .vmdk-hashes with filenames in ./data."
diff $hashes_sorted $files_sorted -U 0 > $diff_output

if [ $? -eq 0 ];
  then
  echo "No unused files found. Every hash in the .vmdk files"
  echo "has a proper file in data-directory. Good."
  echo "Removing temporary files."
  rm -rf "$temp_dir"
  echo "End: `date`"
  exit 0
  else
  echo "Checking if hashes in .vmdk files have a file in the data-directory."
  grep "^-[a-f0-9]" $diff_output | sed 's/^.//' > $missing_files
  if [ `cat $missing_files | wc -l` -eq 0 ];
    then
    echo "Every hash contained in the .vmdk files has a proper file in data-directory. Good."
    grep "^+[a-f0-9]" $diff_output | sed 's/^.//' > $delete_candidates
    unused_count=`cat $delete_candidates | wc -l`
    echo "There are $unused_count unused files in ./data:"
    if [ "$2" != "--delete" ];
      then
      cat $delete_candidates
    fi
    else
    echo "The following `cat $missing_files | wc -l` data files are missing:"
    cat $missing_files
    echo "Repository is damaged. Leaving everything untouched. Exiting."
    echo "Removing temporary files."
    rm -rf "$temp_dir"
    echo "End: `date`"
    exit 1
  fi
fi

if [ "$2" == "--delete" ]
  then
  echo "Counting space used of $1/data."
  echo "Repo-size before pruning: `du $1/data/ -h -s | awk '{print $1;}'`"
  cat $delete_candidates | while read file
    do
    rmpath="$1/data/`echo $file | cut -c1-1`/`echo $file | cut -c2-1`/`echo $file |cut -c3-1`/$file"
    echo "Deleting $rmpath"
    rm -rf $rmpath
    done;
  echo "Counting space used of $1/data."
  echo "Repo-size after pruning: `du $1/data/ -h -s | awk '{print $1;}'`"
  echo "Removing empty directories."
  # Busybox find doesnt know -empty.
  find $1/data -type d -depth -exec rmdir -p --ignore-fail-on-non-empty {} \;
  echo "Counting files in data-directory again."
  # no sort needed here
  # find $1/data -type f -exec basename {} \; > $files
  # ls -1R $1/data | grep -o '\b[0-9a-f]\{40\}\+\b' > $files
  find $1/data -type f -regex '.*/[0-9a-f]\{40\}\+$' -exec basename {} \; > $files
  files_count=`cat $files | wc -l` 
  if [ $files_count == $hashes_count ]
    then
    echo "Number of files and hashes ($files_count) are same, everything went right."
    else
    echo "Number of files ($files_count) and hashes ($hashes_count) are different."
    echo "Perhaps not every file could be deleted. Check it using the logfile."
    echo "End: `date`"
    echo "Removing temporary files."
    rm -rf "$temp_dir"
    exit 1
  fi
  echo "Updating Bcnt in .xsitools-file:"
  bcnt=`grep Bcnt $1/.xsitools | awk -F ': ' '{print $2}'`
  echo "Old value of Bcnt: $bcnt."
  echo "Setting actual number of files ($files_count) as new value of Bcnt."
  sed -i -e "s/Bcnt: $bcnt/Bcnt: $files_count/" $1/.xsitools
fi

echo "Removing temporary files."
rm -rf "$temp_dir"
echo "End: `date`"
} 2>&1 | tee -a var/logs/$name/$name-`date +"%d"`.log

exit 0

lievenmoors · © XSITools

> Keeping some snapshots in your production VM is a good technique to achieve versioning while still using regular backup methods.

Just to clear this up. When I was talking of snapshots, I meant the different versions in the Xsitools repository.
Technically, they provide something very similar to VM snapshots, because of the data deduplication.

Of course you can keep VM snapshots, and then those can be backed up in the Xsitools repository.
And I think this would be the best way of doing things.

But then there is still one problem left unsolved for me:

I only have room for one backup, and some extra space to keep a number of "versions" (or snapshots, or whatever you call them)
in Xsitools. As I am not prepared to live without any backup for about 8 hours, I am obliged to stay within one Xsitools repository,
and to delete old versions to make space. And that is why this script is indispensable for me.

It would be really nice if this functionality would be part of Xsibackup Pro.

Thanks,

lieven

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

#1 Re: General matters » xsibackup doesn't cleanup tmp folder » 2020-04-30 10:29:14

#2 General matters » xsibackup doesn't cleanup tmp folder » 2020-04-29 13:57:09

#3 Re: General matters » error using --rotate » 2020-04-01 12:36:05

#4 Re: General matters » error using --rotate » 2020-03-30 13:37:30

#5 Re: General matters » error using --rotate » 2020-03-25 11:00:38

#6 General matters » error using --rotate » 2020-03-25 10:26:55

#7 Re: General matters » xsibackup is trying to backup every folder in the same dir as *.vmdk » 2020-02-04 08:35:17

#8 Re: General matters » xsibackup is trying to backup every folder in the same dir as *.vmdk » 2020-02-03 12:58:52

#9 General matters » xsibackup is trying to backup every folder in the same dir as *.vmdk » 2020-02-03 12:45:41

#10 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 14:31:11

#11 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 13:47:29

#12 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 13:46:12

#13 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 13:35:30

#14 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 11:11:19

#15 Re: General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 09:44:54

#16 General matters » trouble with XSIBackup/ESXi backups » 2020-01-29 09:29:20

#17 Re: © XSITools » backup of VM failed with "ERROR: cannot locate .ERR file" » 2019-12-18 09:10:21

#18 © XSITools » backup of VM failed with "ERROR: cannot locate .ERR file" » 2019-12-17 10:42:02

#19 Re: © XSITools » xsitools backup failing » 2019-08-26 08:18:58

#20 © XSITools » xsitools backup failing » 2019-08-19 12:20:30

#21 Re: © XSITools » Space usage with xsitools » 2019-01-07 11:31:58

#22 Re: © XSITools » Space usage with xsitools » 2019-01-07 09:57:42

#23 Re: © XSITools » Space usage with xsitools » 2018-12-18 10:02:57

#24 Re: © XSITools » Space usage with xsitools » 2018-12-18 09:45:01

#25 Re: © XSITools » Space usage with xsitools » 2018-12-18 08:54:21

Board footer