©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#1 2021-04-07 14:04:17

isombrinia
Member
Registered: 2019-06-25
Posts: 14

sort: out of memory

I'm trying to prune some old backups, and I'm running into this error:

[root@vmware-77:/vmfs/volumes/5c42674c-fd8386fc-c5bd-d4ae5292e64f/XSIBackup-DC] ./xsibackup --prune /vmfs/volumes/6tb-backups/xsibackup/20200909194328
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.4.0.0: Backup & Replication Software                    |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
License: [REDACTED]
-----------------------------------------------------------------------------------------------------------
PID: 5606813, Running job as: root
-----------------------------------------------------------------------------------------------------------
Finding blocks to prune, please wait...
-----------------------------------------------------------------------------------------------------------
Getting map files from repo...
-----------------------------------------------------------------------------------------------------------
Retrieving general block data 100.00%
-----------------------------------------------------------------------------------------------------------
Please wait while we order the data at: /vmfs/volumes/5c42674c-fd8386fc-c5bd-d4ae5292e64f/XSIBackup-DC/tmp/0U.xsi
-----------------------------------------------------------------------------------------------------------
sort: out of memory
2021-04-07T13:59:00 | Error code 287 at file prune.c, line 287 | Error description: something went wrong when ordering: /vmfs/volumes/5c42674c-fd8386fc-c5bd-d4ae5292e64f/XSIBackup-DC/tmp/0U.xsi, count was: 0

-----------------------------------------------------------------------------------------------------------
Removed <tmp> dir        OK

EDIT:  I realized I was running an old version.  I've updated, and here is the error now:

[root@vmware-77:/vmfs/volumes/5c42674c-fd8386fc-c5bd-d4ae5292e64f/XSIBackup-DC] ./xsibackup --prune /vmfs/volumes/6tb-backups/xsibackup/20200922000002/
License file must be 88 bytes long
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-Free 1.4.3.17: Backup & Replication Software                 |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 6 Major 7 Minor 0 Patch 0
-----------------------------------------------------------------------------------------------------------
PID: 5607286, Running job as: root
-----------------------------------------------------------------------------------------------------------
Finding blocks to prune, please wait...
-----------------------------------------------------------------------------------------------------------
Getting map files from repo, please wait...
-----------------------------------------------------------------------------------------------------------
2021-04-07T14:17:01 | Error code 202 at file prune.c, line 202 | Error description: failed to malloc memory for all unordered blocks
-----------------------------------------------------------------------------------------------------------
2021-04-07T14:17:01 | Error code 3663 at file xsibackup.c, line 3663 | Error description: --prune process returned an error
-----------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-----------------------------------------------------------------------------------------------------------
Removed prog <tmp> dir        OK
-----------------------------------------------------------------------------------------------------------
Removed PID                   OK
-----------------------------------------------------------------------------------------------------------
[root@vmware-77:/vmfs/volumes/5c42674c-fd8386fc-c5bd-d4ae5292e64f/XSIBackup-DC]

EDIT 2:
After looking around the forum a bit more, I've mounted the xsibackup root onto another RHEL 8 Linux system, unzipped the XSIBackup-DC program there, and ran from there instead of the ESXI host.  It appears to be working much better, however I'm still waiting for it to complete.  I figured I would let anybody else know who is running into this issue.

Last edited by isombrinia (2021-04-07 14:55:58)

Offline

#2 2021-04-07 17:17:18

admin
Administrator
Registered: 2017-04-21
Posts: 2,057

Re: sort: out of memory

Thank you for your detailed explanation and pointing to the solution.

If you accumulated a huge repository (you didn't say what its size is) and you want to perform the most intensive operation on it, namely: --prune, the (c)ESXi shell resources might not be enough.

It's not only he amount of memory you may need to perform the pruning, its also that doing so from the (c)ESXi shell on, let's say an NFS DS, will add the network latency, which is not very recommendable as will just contribute to make things much slower.

The best way to use (c)XSIBackup is to prepare some compatible Linux backup server, to be used as a local DS or as a pure IP backup device, the second is the best option as things will work faster due to part of the CPU load being assumed by the server side.

Then use the backup server to prune, mount the .vmdk files or to perform granular restores. You will have the whole system at your finger tips, while in the (c)ESXi shell you are in a propietary cut down environment which has its limits.

Important considerations:

Think corporate. What we mean by that is: why would you want to prune some repository?

What you should do is to, first of all archive your old repositories to keep a historic set of your VMs. And never prune a repository whithout having kept a copy of it. Pruning is a destructive operation: to delete each redundant block, (c)XSIBackup has to traverse the whole set looking for it, then physically delete it from the underlying disk.

Should something go wrong, like memory corruption or a disk fail, you may delete some block that you do not want to delete. That would yield the VMs sharing that block unusable.

From a usage perspective it's easy to recover from that loss, you just have to --repair the repo and perform an additional backup on it, thus the next round will skip all the preexisting blocks and add the new, so you will still keep the bulk of the data.

Still, from a recovery perspective, you may put yourself in a situation in which you can't recover some good number of VMs in the past.

So, be conservative with your backups and don't get us wrong, we do prune our repos, we always keep redundant data though.

Offline

Board footer