Registered users
Linkedin Twitter Facebook Google+

In order to improve user's experience and to enable some functionalities by tracking the user accross the website, this website uses its own cookies and from third parties, like Google Analytics and other similar activity tracking software. Read the Privacy Policy
33HOPS, IT Consultants Download XSIBackup
33HOPS ::: Proveedores de Soluciones Informáticas :: Madrid :+34 91 930 98 66Avda. Castilla la Mancha, 95 - local posterior - 28700 S.S. de los Reyes - MADRID33HOPS, Sistemas de Informacion y Redes, S.L.Info

XSIBackup Datacenter.

Change log

    • Improvement: you don't need to manually copy the xsibackup binary to the remote host. Now the binary is copied if it does not exist or is updated in case a different version is detected on the remote end. It is copied to /bin in case of Linux systems or /scratch/XSI/XSIBackup-DC in case of ©ESXi hosts. You may still use some binary at some arbitrary remote path by using the --remote-path argument.
    • Bug fix: some issues related to license file recognition were fixed in additional builds.
    • Bug fix: fixed problem with repo lock file on rotation.
    • Bug fix: command line --add-key argument wasn't working well. The equivalent option was nonetheless working in the GUI.
    • Improvement: added support for ©ESXi virtual machines in suspended status.
    • Improvement: improved SMTP protocol messages, errors are captured and shown on screen.
    • Improvement: improved block seek time on big repos (> 1TB), now this time remains stable as the repos grow and allows bigger repos on 1MB block size.
    • Bug fix: fixed problem detecting license files.
    • Bug fix: fixed problem with repo lock file on rotation.
    • Bug fix: fixed issue deleting /tmp/pid folders.
    • Bug fix: /scratch/downloads folder clean up after --config-backup will prevent .tgz config backup files from accumulating there in between reboots.
    • Improvement: subfolder cfgbak has been added to repository to host config backups.
    • Bug fix: sending report to more than one e-mail address was not working due to the e-mail validation being applied to the whole comma separated string.
    • Bug fix: backup of ESXi config over IP was failing due to the server side shutting down before receiving the backup file.
    • Improvement: helper routine to help set up accounts from the command line.
    • Improvement: SIGTERM(2), namely execution stopped as per user request, is treated differently and is not trapped as an error, but just displayed and informed as such.
    • Improvement: faster locking mechanism for pruning operation.
    • Improvement: improved installer now takes care to keep user config files: smtpsrvs.conf, xsibackup.conf & crontabs.
    • Bug fix: VMs with similar names (name of one VM contained within another), can cause the wrong VM to be selected for backup or cause issues during the backup. This has been fixed in this release.
    • Improvement: ghost entries in the VM inventory, namely: VMs that have been deleted from disk but have not been unregistered from the ESXi inventory, would trigger a generic error, now it is more verbose and the cause is explained.
    • Bug fix: fixed issue affecting last block in backed up virtual disks with the potential to write that last block wrongly.
    • Bug fix: revised and fixed issues relative to backup of disks hosted in NFS datastores.
    • Bug fix: the client controls the server side behaviour. Up until now the client would just stop writting to the server when done, this would cause an error on the server side in the form of a SIGPIPE(13), as the SSH tunnel would just get disconnected. Since this version the client writes signals to the server at the application layer, so that the server side is notified when the client is done in such way that the server binary ends execution in a nice way.
    • Improvement: final checksum on over IP backups compares the number of bytes sent with the number of bytes effectively written on the server side, this is done on a per job basis and prints a green notice in case everything matched or a red one in case something went wrong and figures don't match. This check is done on addition to pre-existing partial checksums on every individual block.
    • Bug fix: fixed bug in block detection. Would not affect backup integrity but could halt a job.
    • Improvement: optimized memory usage in --prune operation.
    • Bug fix: recursive pruning could raise a segfault. Not critical, subsequent --prune operations would continue the pruning process.
    • Improvement: search in the block manifest is performed by means of a new detection algorithm which includes native C sorting and a dynamic index, which drastically reduces the amount of CPU cycles needed to detect a block.
    • Improvement: ©ESXi 7.0 blocks all files in a VM for reading, existing snapshots can't be backed up. Now the backup/ replica process deletes any previously existing snapshot previous to performing the backup, so that all your VM data is backed up. This is done only when ©ESXi 7.0 or above is detected.
    • Improvement: all pruning operations have been redesigned to be faster and to support a bigger volume of data.
    • Improvement: all block indexing related to pruning is now performed via C native functions and dynamic indexing of data is employed vs an old indexing fixed depth system that used little memory but was slower and had hardcoded limits, as it depended on the sort binary.
    • Bug fix: bug in cron launched job detection would cause jobs not to be executed from the crontab
    • Bug fix: restore module now re-truncates disks to their nominal size after restore process, just to make sure disks with eventual empty ends are rebuilt correctly
    • Bug fix: a 300 GB limit was wrongly imposed on licensed versions since previous version
    • Improvement: New argument --config-backup will copy the ESXi Config to the remote repository or replica folder.
    • Improvement: Execute permissions are now set for each backup job everytime they are saved in the GUI.
    • Improvement: Pruning alone as an action or per the --rotate argument after a backup is now fully supported in the GUI and a progressbar in two phases (collecting blocks and deleting them) is shown.
    • Improvement: Little routine to avoid interpreting malformed remote paths as if they were local.
    • Improvement: Improved error detection from the GUI layer..
    • Bug fix: a bug prevented from generating backups when using a block size over 1MB (default). You can again use bigger block sizes
    • Bug fix: extremely verbose progress on cron launched --replica jobs. Fixed in this version, only user launched jobs produce a verbose progress.
    • Compatibility: this version is fully compatible with ©ESXi 7
    • Improvement: VMs can be restored from a remote folder on any SSH server. This makes ©XSIBackup-DC a fully capable cloud backup solution
    • Improvement: matching of client and server ciphers is now more user friendly. Only if no matching cipher is found the tunnel negotiation will not be stablished.
    • Improvement: since this version you can run multiple concurrent jobs in a single host. Nonetheless we highly discourage such practice and recommend running jobs sequentially.
    • Improvement: a full improved GUI has been added to this version. It is a dialog/ nCurses User Interface which allows to manage all main functioning aspects of ©XSIBackup-DC
    • Bug fix: a bug was producing wrong backup chunks since version only on backups performed on Virtual Machines residing in NFS volumes. It has been fixed in this version. If you host production VMs on NFS shares, you should update immediately and discards backups made with versions and
    • Now allows sending e-mail from ESXi 7.0 by disabling the firewall during the e-mail submission lapse
    • Compatible with ESXi 7.0.0. If you run ESXi 6.7 or below you should use ©XSIBackup-DC
    • An installer has been added to the downloadable package
    • Some routines have been improved
    • Bug fixes: fixed detection of duplicate known_hosts file entries when managing keys to avoid unhandled errors or blocking of command line.
    • Improvements: temp dirs split into two locations: /tmp/xsi and [install root]/tmp. Temp files requiring speed are placed into the first one, while storage intensive operations are moved to the second. This will allow to improve backup speed by 30% in some cases where (c)XSIBackup-DC is installed to a HD.
    • Improvements: two new arguments have been added: --smtp-add and --smtp-test. They allow by means of a command line easy to use UI to respectively add an SMTP server to the list of SMTP servers in the etc/smtpsrvs.conf file and to test any of the existing servers by sending a test e-mail to any e-mail address.

      You can just upgrade by overwritting the xsibackup binary.
    • Improvements: improved resiliency to fatal errors in scenarios where you use folder backup/ replica source on ESXi VMs which aren't correctly registered or have missing components.
    • Improvement: general block log is recounted to make sure that we consider all blocks that must be kept after pruning
    • Improvement: (c)XSIBackup-DC uses a TMP dir at <install root>/tmp in ESXi, which will tipically be /scratch/XSI/XSIBackup-DC/tmp, or the /tmp dir in different Linux flavours when working as a server, that comprises © Synology DSM OS too. The scratch volume in ESXi generally is 4.00 GB in size while the /tmp folder or filesystem ranges from one Linux distro to the other and goes from 500 MB to some GBs in size.

      The above will be more than enough when using big block sizes for deduplication. Nevertheless, should you use some backup repository with a block size of 1MB and let it grow to some hundreds of GBs, the prune process can become quite resource hungry in terms of space. In those cases you will need to grow the TMP file system or move it somewhere else where more room can be allocated to store temporary logs.

      In this version we have introduced the alt_tmp_dir variable in the <install root>/etc/xsibackup.conf file which allows to pass a list of additional TMP directories to use when the default dir is not enough.

      (c)XSIBackup-DC will detect whether switching to an alternative TMP dir is required for pruning. Nonetheless you should always try to grow your /tmp dir in your Linux server to some tenths of GBs in these cases, thus moving the TMP is not necessary.
    • Bug fix: .vmsd file would need some manual tweaking if some snapshots were saved along with the VM to be able to switch it on with the exact same configuration as in the source VM. Now the .vmsd file is automatically arranged to match the existing snapshot structure no matter the number of snapshots.
    • Bug fix: exit code was always 0, that is a bug, we fixed this in this version. Now the exit code is the number of accumulated errors at the moment the program exits.
    • Bug fix: empty lines in the .vmx file could result in a truncated target .vmx file. Nevertheless your .vmx file should not contain blank lines at all, especially in the middle of the file, blank lines at the end would not matter except for the different outcome in size. As ESXi allows (we should probably say, it tolerates and ignores them as well) this empty lines XSIBackup-DC will just do the same from now on
    • Improvement: now (c)XSIBackup-DC finds its remote counterpart more easily. From this version xsibackup binary is searched for on remote's /usr/bin folder first, if it can't be found there a which command is run and, if positive, its returned path is used as the remote xsibackup bin location. If both of the previous methods fail, the binary will be searched for in the same path as in the client side. If all of the previous locations do not return a match you must set the --remote-path argument to explicitly set the remote binary location.
    • Bug fix: empty lines in the ./etc/xsibackup.conf file would cause a SEGFAULT, now they are ignored.
    • Improvement: annoying 'FIPS mode initialized' message can interleave in the STDERR FD in the OpenSSH tunnel raising errors. We have tried to make the inner OpenSSH subprocess quiet there where a single value is expected. Please OpenSSH developers, stop spitting irrelevant information into STDERR
    • Bug fix: visual bug when showing user defined block size units, bytes printed, but MB was appended to amount
    • Improvements: now the xsibackup.conf entry ssh_ciphers supports the 'auto' keyword as a value. Thus you have multiple options to control what cipher to use for data transfer: auto, default in .conf file, per job via the --ssh-ciphers argument and as per the l option (Less Secure Ciphers) in the --options argument.

      This update requires that you update the ./etc/xsibackup.conf file as well as the xsibackup main binary.
    • Bug fix: interstitial spaces would still cause trouble in some systems, we fixed this and now spaces should work seamlessly locally or over IP. Nevertheless you should not forget to wrap any path containing spaces into double quotes.
    • Bug fix: interstitial spaces in VM name or path would raise a SEGFAULT in transfers over IP --replica or --backup
    • Improvement: license is checked and in case it doesn't match a clear message is printed out along with the reason
    • Improvement: time and speed info added to the replica --ckeck action
    • Improvements: additional file buffering is performed on the server side so that the update of virtual disks and hash tables is turned into an atomic operation, namely an inode re-assignment. This makes --replica a lot more resilient, as any file transfer that is interrupted will be re-taken from a 0 offset This guarantees file and hashmap consistency.
    • Improvements: .vmdk files are checked against its stored hashes before starting a --replica job. When some change is detected the hash tables for all disks are rebuilt to match the underlying data. This allows to switch on the --replica VMs without having to delete the replicated folder and start from scratch, and: what is more important, you don't have to remind to keep --replica consistency by hand.

      Since this version ( the server installation will require the xsilib binary, thus you should get used to install the full package in the server side as well as in the client.
    • Improvements: reinforced error handling in --prune action
    • We have refined the way remote files are synchronized during a replica. Some tasks like keeping its size (truncating) and simple cheksums are now performed from the server role and do not depend on Linux components
    • Bug fixes: VM configuration files would not be truncated when their size is reduced from one backup cycle to the next, i.e.: .vmsd files may present this behaviour. This would only happen with some config files, would not affect disk files.

      All users interested in the --replica feature should upgrade to this version. And remember, replicating is not backing up. The --backup feature is much more robust by its own nature and can overcome almost any circumstance, replicating depends on a stable enough connection.

      GENERAL INFO WHEN UPGRADING: (c)XSIBackup-DC is even easier to upgrade and maintain than (c)XSIBackup-Pro, as it is constituted by a main binary (xsibackup) and just an additional library xsilib which performs some additional tasks just in ESXi (you don't need it in Linux). So all you need to do is overwrite the main binary. We have performed only one upgrade on the accompanying library xsilib in the last months, in any case you can overwrite it too, just in case.
    • Improvements: block files are transferred to a buffer, tested then renamed with test. This adds an additional layer of tests ensuring block map integrity and retaking partial replicas is now more resilient.
    • Improvements: license is now only checked when using the --backup or --replica actions, rest of actions including server role are free.
    • Improvements: as you should know, modifying a replica will corrupt the replica data. You must leave replicas alone until you need them. Once you switch a replicated VM on, you can't continue to replicate to it.

      Now you can --check replicas too, not only backups. Checking a replica ensures that the data is the same as the original without the need to switch the VM on.

      If you still want to switch the replica VM on and continue to replicate to it, there are ways to circumvent this inconvenience, like creating a snapshot on the replica from a stopped state, switching the replica on and discarding the snapshot when you are done.

      ATTENTION: please be aware that discarding a snapshot is not the same as deleting it. In virtualization language deleting a snapshot is integrating it with the base disks and that would also break your replica.
    • Bug fixes: SSH subprocess wasn't conveniently killed in some circumstances
    • Bug fixes: Some keys send different signals to the main process if you hit them while the program is running in user mode. We try to control everything going on, but some signals can't just be discarded, so be careful not to strike keys randomly while running jobs in user mode ;-)
  • -- All (c)XSIBackup-DC users should upgrade to this version, specially if using pruning --
    • Data chunks sent over the SSH tunnel are counted and a simple application level checksum mechanism is set on top of the TCP layer and SSH tunnel signalling.
  • -- All (c)XSIBackup-DC users should upgrade to this version, speciall if using pruning --
    • Bug fixes: we have fixed some bugs in the --check and --info actions that would make them slow finding .map files in big repositories with small block sizes.
    • Bug fix: pruning is a delicate process, running out of space in the /tmp dir could lead to data loss. We have moved the main tmp dir used for pruning to the installation root in ESXi. Recommended location is /scratch/xsi, default size is 4GB, more than enough to handle multi-TB multi restore point repositories. You may move /scratch to a bigger volume in case you need more room. Read this post to delve into pruning details.
    • Improvement: --prune module has been completely reprogrammed to be faster. It was enough to prune 10MB block sizes and above, but would take long and require tweaking in the OS to prune big 1MB repositories. Read this post to delve into pruning details.
    • Improvement: fprintf() function has been replaced by fwrite() in some parts where recursiveness would make fwrite() be faster than fprintf() this improves performance a bit.
    • Fixed: issue with --update-cron command in ESXi 6.7.0 U3
    • Fixed: minor bugs
    • Fixed: since --compression option was set as default, only editing the etc/xsibackup.conf file would allow to disable compression. Now it can be set through the --compression=[0|1|true|false|yes|no] option again.
    • Improvement: default block size for backups has been changed to 1M in alignment with the ESXi block size. 10M, 20M and 50M are still available to use. Please, note that although 1M block size is a must to backup over IP and it is indeed faster when used over SSH/IP, you might get better speed results with bigger block sizes when backing up to a local datastore. Enabling jumbo frames can be of great help too at the time to increase throughput.

      Bigger block sizes allow also to use VMFS volumes as the target of deduplicated repositories. VMFS 5.0 declared to be able to host around 132,000 inodes which is a very limited number to host lots of tiny chunks of data, nonetheless VMWare says that VMFS 6.0 is able to host millions of files, which is more than enough for (c)XSIBackup-DC to be able to host many terabytes of data even at 1M block size.

      In any case, being able to use NFS to backup to underlying XFS or EXT4 file systems makes VMFS not that interesting or needed for your backups.
    • Improvement: the file has been moved to /tmp/ so that using any Linux distro as an IP target is eased in regards to permissions required when compared to the /var/run folder previously holding that file
    • Improvement: we have added extended debug information to some critical error messages to help narrow down NFS connectivity/ permission issues
    • Fixed: small bug when there are no blocks to prune this fact was previously informed as an error
    • Added: clean-up of var/spool/mail and var/log/prog directories on load. Latest e-mail templates and progress information available in these directories will remain available in between backup cycles for you to inspect. Until this version they would accumulate in those dirs requiring manual clean-up
    • Fixed: e-mail subject would not inform of errors during the backup process. They were informed in the e-mail body though.
    • Fixed: partial error count per VM was not logged properly.
    • Added: accepting EULA on first run is now mandatory.
    • Fixed: minor bugs
    • Fixed: the xsibackup.log was wiped on every new execution, now new output is appended.
    • Fixed: progress info would stuff the xsibackup.log file with every progress update in a new line, resulting in a lot of useless information being logged.
    • Improvement: request.key file is not generated unless a real command is run. We changed this behaviour as some users might prefer to install their license first. To generate a request.key file just run: ./xsibackup --request-key
    • Bug fixes: final release
    • Compatible with Synology devices as a native binary. Requires to configure root access to the Synology appliance.
      The --link-srv action will work out of the box for user root. All other users different than root store their configuration files at custom paths in Synology NAS devices, thus by now you need to perform the key exchange manually.
    • Small bug fixes and output texts retouch
    • Unified log subsystem for client and server.
    • Improved --check routine with more accurate progress and some bugs fixed. Before this version it would report small files as K.O. due to being smaller than the block size, which was a gross bug.
    • Accumulated time added to the output and the e-mail report.
    • Last errors data added to the e-mail report footer. It can handle up to 8KB of errors, beyond that point you must consult the error.log file.
    • Rotating data can now be achieved from an IP client, before the --rotate option was only available in local backups.
    • Bug fix: signal handling in the IP server side would send a 13 signal to the client when triggering jobs from the cron, which would in turn record it in the log. It caused no additionalissues, apart from overpopulating the xsibackup.log file.
    • The way to use ciphers has been improved and normalized. (c)XSIBackup-DC will pick the ciphers defined in the ssh_ciphers variable inside the etc/xsibackup.conf file. You may supersede the default list of ciphers by using the --ssh-ciphers argument in the command line. By doing so the default list of ciphers is replaced by your own custom one.

      To set a custom list of ciphers, you need to find out which ciphers are available in your OpenSSH build in order to not get a cipher negotiation error. It is arguable whether OpenSSH should simply ignore an unknown cipher, but the truth is that you must be careful as a single mismatch will cause an error.

      You may also use the --options=L switch, which is equivalent to set a light encryption deprecated list of ciphers or "Less Secure Ciphers" like:
    • --backup-how argument is introduced. It works the same way as in the Classic XSIBackup edition, it can take one of 3 values: hot (default), cold and warm.
    • etc/xsibackup.conf file is now functional, allowing the user to tweak some general scope variables
    • Trial Version notice appeared on every execution even if the server had been just restarted. This was a bug, the Trial Version limits have been restored.
    • Improved progress information, now offering simultaneous per file and global percentages in real time
    • Fixed bug in open port detection prior to performing action, non standard ports could not be used in over IP transfers
    • Support for ALL and RUNNING has been added in the virtual machine selection syntax, i.e: "VMs(ALL)" and "VMs(RUNNING)"
    • Automatic removal of backup snapshot when the backup process is aborted by a Ctrl+C condition
    • We have redesigned PID management to make it more usable and avoid loops
    • Bug fixes mainly
    • HTML report may pick templates from the /var/html folder. Templates must be named 000 to 999 and placed in this folder as: 000.html, 001.html, 002.html and so on.

      You can design whatever HTML layout you wish and then include the <!-- PLACEHOLDER REPORT --> comment in its own line. You may then invoke the template by using the option --html-template=NNN
    • Extended HTML report functionality, fall back options and SMTP servers panel at /etc/smtpsrvs.conf.
    • Basic HTML report functionality.
    • Fixed the "Permanently added 'a.b.c.d' (RSA) to the list of known hosts" error on reboot.
    • Added --exclude argument. Allows to exclude disks by parsing a REGEXP pattern. If you want to exclude multiple disks use the --or-- operator in your REGEXP
    • Added --update-cron=[user] action. Inserts the user crontab contents into its corresponding ESXi user crontab
    • Added --install-cron=[user] action. Adds the user crontab to the ESXi /etc/rc.local.d/ file, so that crontabs are persistent across reboots
    • Added --uninstall-cron=[user] action. Removes the user's crontab from the /etc/rc.local.d/ You still must comment or remove the cron lines and --update-cron=[user] to completely disable cron jobs for that user
    • --check=full action redesigned.
This page was last modified on 2021-01-12

Website Map
Resources & help
Index of Docs
33HOPS Forum

©33HOPS site relies on the following technologies and technology partners:
SSL Protocol PayPal Payment Gateway Stripe Payment Gateway

©33HOPS Sistemas de Información y Redes, S.L. | VAT No: ESB83583716 | Avda. Castilla la Mancha, 95, local posterior, 28701 San Sebastián e los Reyes (Madrid) Spain

Fill in to download
The download link will be sent to your e-mail.

            Read our Privacy Policy

(*) DC & Pro users, please login to your user area to download