Registered users
Linkedin Twitter Facebook Google+

In order to improve user's experience and to enable some functionalities by tracking the user accross the website, this website uses its own cookies and from third parties, like Google Analytics and other similar activity tracking software. Read the Privacy Policy
33HOPS, IT Consultants
33HOPS ::: Proveedores de Soluciones Informáticas :: Madrid :+34 91 930 98 66Avda. Castilla la Mancha, 95 - local posterior - 28700 S.S. de los Reyes - MADRID33HOPS, Sistemas de Informacion y Redes, S.L.Info

XSIBackup Datacenter.

Change log

    • Improvement: New argument --config-backup will copy the ESXi Config to the remote repository or replica folder.
    • Improvement: Execute permissions are now set for each backup job everytime they are saved in the GUI.
    • Improvement: Pruning alone as an action or per the --rotate argument after a backup is now fully supported in the GUI and a progressbar in two phases (collecting blocks and deleting them) is shown.
    • Improvement: Little routine to avoid interpreting malformed remote paths as if they were local.
    • Improvement: Improved error detection from the GUI layer..
    • Bug fix: a bug prevented from generating backups when using a block size over 1MB (default). You can again use bigger block sizes
    • Bug fix: extremely verbose progress on cron launched --replica jobs. Fixed in this version, only user launched jobs produce a verbose progress.
    • Compatibility: this version is fully compatible with ©ESXi 7
    • Improvement: VMs can be restored from a remote folder on any SSH server. This makes ©XSIBackup-DC a fully capable cloud backup solution
    • Improvement: matching of client and server ciphers is now more user friendly. Only if no matching cipher is found the tunnel negotiation will not be stablished.
    • Improvement: since this version you can run multiple concurrent jobs in a single host. Nonetheless we highly discourage such practice and recommend running jobs sequentially.
    • Improvement: a full improved GUI has been added to this version. It is a dialog/ nCurses User Interface which allows to manage all main functioning aspects of ©XSIBackup-DC
    • Bug fix: a bug was producing wrong backup chunks since version only on backups performed on Virtual Machines residing in NFS volumes. It has been fixed in this version. If you host production VMs on NFS shares, you should update immediately and discards backups made with versions and
    • Now allows sending e-mail from ESXi 7.0 by disabling the firewall during the e-mail submission lapse
    • Compatible with ESXi 7.0.0. If you run ESXi 6.7 or below you should use ©XSIBackup-DC
    • An installer has been added to the downloadable package
    • Some routines have been improved
    • Bug fixes: fixed detection of duplicate known_hosts file entries when managing keys to avoid unhandled errors or blocking of command line.
    • Improvements: temp dirs split into two locations: /tmp/xsi and [install root]/tmp. Temp files requiring speed are placed into the first one, while storage intensive operations are moved to the second. This will allow to improve backup speed by 30% in some cases where (c)XSIBackup-DC is installed to a HD.
    • Improvements: two new arguments have been added: --smtp-add and --smtp-test. They allow by means of a command line easy to use UI to respectively add an SMTP server to the list of SMTP servers in the etc/smtpsrvs.conf file and to test any of the existing servers by sending a test e-mail to any e-mail address.

      You can just upgrade by overwritting the xsibackup binary.
    • Improvements: improved resiliency to fatal errors in scenarios where you use folder backup/ replica source on ESXi VMs which aren't correctly registered or have missing components.
    • Improvement: general block log is recounted to make sure that we consider all blocks that must be kept after pruning
    • Improvement: (c)XSIBackup-DC uses a TMP dir at <install root>/tmp in ESXi, which will tipically be /scratch/XSI/XSIBackup-DC/tmp, or the /tmp dir in different Linux flavours when working as a server, that comprises © Synology DSM OS too. The scratch volume in ESXi generally is 4.00 GB in size while the /tmp folder or filesystem ranges from one Linux distro to the other and goes from 500 MB to some GBs in size.

      The above will be more than enough when using big block sizes for deduplication. Nevertheless, should you use some backup repository with a block size of 1MB and let it grow to some hundreds of GBs, the prune process can become quite resource hungry in terms of space. In those cases you will need to grow the TMP file system or move it somewhere else where more room can be allocated to store temporary logs.

      In this version we have introduced the alt_tmp_dir variable in the <install root>/etc/xsibackup.conf file which allows to pass a list of additional TMP directories to use when the default dir is not enough.

      (c)XSIBackup-DC will detect whether switching to an alternative TMP dir is required for pruning. Nonetheless you should always try to grow your /tmp dir in your Linux server to some tenths of GBs in these cases, thus moving the TMP is not necessary.
    • Bug fix: .vmsd file would need some manual tweaking if some snapshots were saved along with the VM to be able to switch it on with the exact same configuration as in the source VM. Now the .vmsd file is automatically arranged to match the existing snapshot structure no matter the number of snapshots.
    • Bug fix: exit code was always 0, that is a bug, we fixed this in this version. Now the exit code is the number of accumulated errors at the moment the program exits.
    • Bug fix: empty lines in the .vmx file could result in a truncated target .vmx file. Nevertheless your .vmx file should not contain blank lines at all, especially in the middle of the file, blank lines at the end would not matter except for the different outcome in size. As ESXi allows (we should probably say, it tolerates and ignores them as well) this empty lines XSIBackup-DC will just do the same from now on
    • Improvement: now (c)XSIBackup-DC finds its remote counterpart more easily. From this version xsibackup binary is searched for on remote's /usr/bin folder first, if it can't be found there a which command is run and, if positive, its returned path is used as the remote xsibackup bin location. If both of the previous methods fail, the binary will be searched for in the same path as in the client side. If all of the previous locations do not return a match you must set the --remote-path argument to explicitly set the remote binary location.
    • Bug fix: empty lines in the ./etc/xsibackup.conf file would cause a SEGFAULT, now they are ignored.
    • Improvement: annoying 'FIPS mode initialized' message can interleave in the STDERR FD in the OpenSSH tunnel raising errors. We have tried to make the inner OpenSSH subprocess quiet there where a single value is expected. Please OpenSSH developers, stop spitting irrelevant information into STDERR
    • Bug fix: visual bug when showing user defined block size units, bytes printed, but MB was appended to amount
    • Improvements: now the xsibackup.conf entry ssh_ciphers supports the 'auto' keyword as a value. Thus you have multiple options to control what cipher to use for data transfer: auto, default in .conf file, per job via the --ssh-ciphers argument and as per the l option (Less Secure Ciphers) in the --options argument.

      This update requires that you update the ./etc/xsibackup.conf file as well as the xsibackup main binary.
    • Bug fix: interstitial spaces would still cause trouble in some systems, we fixed this and now spaces should work seamlessly locally or over IP. Nevertheless you should not forget to wrap any path containing spaces into double quotes.
    • Bug fix: interstitial spaces in VM name or path would raise a SEGFAULT in transfers over IP --replica or --backup
    • Improvement: license is checked and in case it doesn't match a clear message is printed out along with the reason
    • Improvement: time and speed info added to the replica --ckeck action
    • Improvements: additional file buffering is performed on the server side so that the update of virtual disks and hash tables is turned into an atomic operation, namely an inode re-assignment. This makes --replica a lot more resilient, as any file transfer that is interrupted will be re-taken from a 0 offset This guarantees file and hashmap consistency.
    • Improvements: .vmdk files are checked against its stored hashes before starting a --replica job. When some change is detected the hash tables for all disks are rebuilt to match the underlying data. This allows to switch on the --replica VMs without having to delete the replicated folder and start from scratch, and: what is more important, you don't have to remind to keep --replica consistency by hand.

      Since this version ( the server installation will require the xsilib binary, thus you should get used to install the full package in the server side as well as in the client.
    • Improvements: reinforced error handling in --prune action
    • We have refined the way remote files are synchronized during a replica. Some tasks like keeping its size (truncating) and simple cheksums are now performed from the server role and do not depend on Linux components
    • Bug fixes: VM configuration files would not be truncated when their size is reduced from one backup cycle to the next, i.e.: .vmsd files may present this behaviour. This would only happen with some config files, would not affect disk files.

      All users interested in the --replica feature should upgrade to this version. And remember, replicating is not backing up. The --backup feature is much more robust by its own nature and can overcome almost any circumstance, replicating depends on a stable enough connection.

      GENERAL INFO WHEN UPGRADING: (c)XSIBackup-DC is even easier to upgrade and maintain than (c)XSIBackup-Pro, as it is constituted by a main binary (xsibackup) and just an additional library xsilib which performs some additional tasks just in ESXi (you don't need it in Linux). So all you need to do is overwrite the main binary. We have performed only one upgrade on the accompanying library xsilib in the last months, in any case you can overwrite it too, just in case.
    • Improvements: block files are transferred to a buffer, tested then renamed with test. This adds an additional layer of tests ensuring block map integrity and retaking partial replicas is now more resilient.
    • Improvements: license is now only checked when using the --backup or --replica actions, rest of actions including server role are free.
    • Improvements: as you should know, modifying a replica will corrupt the replica data. You must leave replicas alone until you need them. Once you switch a replicated VM on, you can't continue to replicate to it.

      Now you can --check replicas too, not only backups. Checking a replica ensures that the data is the same as the original without the need to switch the VM on.

      If you still want to switch the replica VM on and continue to replicate to it, there are ways to circumvent this inconvenience, like creating a snapshot on the replica from a stopped state, switching the replica on and discarding the snapshot when you are done.

      ATTENTION: please be aware that discarding a snapshot is not the same as deleting it. In virtualization language deleting a snapshot is integrating it with the base disks and that would also break your replica.
    • Bug fixes: SSH subprocess wasn't conveniently killed in some circumstances
    • Bug fixes: Some keys send different signals to the main process if you hit them while the program is running in user mode. We try to control everything going on, but some signals can't just be discarded, so be careful not to strike keys randomly while running jobs in user mode ;-)
  22. -- All (c)XSIBackup-DC users should upgrade to this version, specially if using pruning --
    • Data chunks sent over the SSH tunnel are counted and a simple application level checksum mechanism is set on top of the TCP layer and SSH tunnel signalling.
  23. -- All (c)XSIBackup-DC users should upgrade to this version, speciall if using pruning --
    • Bug fixes: we have fixed some bugs in the --check and --info actions that would make them slow finding .map files in big repositories with small block sizes.
    • Bug fix: pruning is a delicate process, running out of space in the /tmp dir could lead to data loss. We have moved the main tmp dir used for pruning to the installation root in ESXi. Recommended location is /scratch/xsi, default size is 4GB, more than enough to handle multi-TB multi restore point repositories. You may move /scratch to a bigger volume in case you need more room. Read this post to delve into pruning details.
    • Improvement: --prune module has been completely reprogrammed to be faster. It was enough to prune 10MB block sizes and above, but would take long and require tweaking in the OS to prune big 1MB repositories. Read this post to delve into pruning details.
    • Improvement: fprintf() function has been replaced by fwrite() in some parts where recursiveness would make fwrite() be faster than fprintf() this improves performance a bit.
    • Fixed: issue with --update-cron command in ESXi 6.7.0 U3
    • Fixed: minor bugs
    • Fixed: since --compression option was set as default, only editing the etc/xsibackup.conf file would allow to disable compression. Now it can be set through the --compression=[0|1|true|false|yes|no] option again.
    • Improvement: default block size for backups has been changed to 1M in alignment with the ESXi block size. 10M, 20M and 50M are still available to use. Please, note that although 1M block size is a must to backup over IP and it is indeed faster when used over SSH/IP, you might get better speed results with bigger block sizes when backing up to a local datastore. Enabling jumbo frames can be of great help too at the time to increase throughput.

      Bigger block sizes allow also to use VMFS volumes as the target of deduplicated repositories. VMFS 5.0 declared to be able to host around 132,000 inodes which is a very limited number to host lots of tiny chunks of data, nonetheless VMWare says that VMFS 6.0 is able to host millions of files, which is more than enough for (c)XSIBackup-DC to be able to host many terabytes of data even at 1M block size.

      In any case, being able to use NFS to backup to underlying XFS or EXT4 file systems makes VMFS not that interesting or needed for your backups.
    • Improvement: the file has been moved to /tmp/ so that using any Linux distro as an IP target is eased in regards to permissions required when compared to the /var/run folder previously holding that file
    • Improvement: we have added extended debug information to some critical error messages to help narrow down NFS connectivity/ permission issues
    • Fixed: small bug when there are no blocks to prune this fact was previously informed as an error
    • Added: clean-up of var/spool/mail and var/log/prog directories on load. Latest e-mail templates and progress information available in these directories will remain available in between backup cycles for you to inspect. Until this version they would accumulate in those dirs requiring manual clean-up
    • Fixed: e-mail subject would not inform of errors during the backup process. They were informed in the e-mail body though.
    • Fixed: partial error count per VM was not logged properly.
    • Added: accepting EULA on first run is now mandatory.
    • Fixed: minor bugs
    • Fixed: the xsibackup.log was wiped on every new execution, now new output is appended.
    • Fixed: progress info would stuff the xsibackup.log file with every progress update in a new line, resulting in a lot of useless information being logged.
    • Improvement: request.key file is not generated unless a real command is run. We changed this behaviour as some users might prefer to install their license first. To generate a request.key file just run: ./xsibackup --request-key
    • Bug fixes: final release
    • Compatible with Synology devices as a native binary. Requires to configure root access to the Synology appliance.
    • The --link-srv action will work out of the box for user root. All other users different than root store their configuration files at custom paths in Synology NAS devices, thus by now you need to perform the key exchange manually.
    • Small bug fixes and output texts retouch
    • Unified log subsystem for client and server.
    • Improved --check routine with more accurate progress and some bugs fixed. Before this version it would report small files as K.O. due to being smaller than the block size, which was a gross bug.
    • Accumulated time added to the output and the e-mail report.
    • Last errors data added to the e-mail report footer. It can handle up to 8KB of errors, beyond that point you must consult the error.log file.
    • Rotating data can now be achieved from an IP client, before the --rotate option was only available in local backups.
    • Bug fix: signal handling in the IP server side would send a 13 signal to the client when triggering jobs from the cron, which would in turn record it in the log. It caused no additionalissues, apart from overpopulating the xsibackup.log file.
    • The way to use ciphers has been improved and normalized. (c)XSIBackup-DC will pick the ciphers defined in the ssh_ciphers variable inside the etc/xsibackup.conf file. You may supersede the default list of ciphers by using the --ssh-ciphers argument in the command line. By doing so the default list of ciphers is replaced by your own custom one.

      To set a custom list of ciphers, you need to find out which ciphers are available in your OpenSSH build in order to not get a cipher negotiation error. It is arguable whether OpenSSH should simply ignore an unknown cipher, but the truth is that you must be careful as a single mismatch will cause an error.

      You may also use the --options=L switch, which is equivalent to set a light encryption deprecated list of ciphers or "Less Secure Ciphers" like:
    • --backup-how argument is introduced. It works the same way as in the Classic XSIBackup edition, it can take one of 3 values: hot (default), cold and warm.
    • etc/xsibackup.conf file is now functional, allowing the user to tweak some general scope variables
    • Trial Version notice appeared on every execution even if the server had been just restarted. This was a bug, the Trial Version limits have been restored.
    • Improved progress information, now offering simultaneous per file and global percentages in real time
    • Fixed bug in open port detection prior to performing action, non standard ports could not be used in over IP transfers
    • Support for ALL and RUNNING has been added in the virtual machine selection syntax, i.e: "VMs(ALL)" and "VMs(RUNNING)"
    • Automatic removal of backup snapshot when the backup process is aborted by a Ctrl+C condition
    • We have redesigned PID management to make it more usable and avoid loops
    • Bug fixes mainly
    • HTML report may pick templates from the /var/html folder. Templates must be named 000 to 999 and placed in this folder as: 000.html, 001.html, 002.html and so on.

      You can design whatever HTML layout you wish and then include the <!-- PLACEHOLDER REPORT --> comment in its own line. You may then invoke the template by using the option --html-template=NNN
    • Extended HTML report functionality, fall back options and SMTP servers panel at /etc/smtpsrvs.conf.
    • Basic HTML report functionality.
    • Fixed the "Permanently added 'a.b.c.d' (RSA) to the list of known hosts" error on reboot.
    • Added --exclude argument. Allows to exclude disks by parsing a REGEXP pattern. If you want to exclude multiple disks use the --or-- operator in your REGEXP
    • Added --update-cron=[user] action. Inserts the user crontab contents into its corresponding ESXi user crontab
    • Added --install-cron=[user] action. Adds the user crontab to the ESXi /etc/rc.local.d/ file, so that crontabs are persistent across reboots
    • Added --uninstall-cron=[user] action. Removes the user's crontab from the /etc/rc.local.d/ You still must comment or remove the cron lines and --update-cron=[user] to completely disable cron jobs for that user
    • --check=full action redesigned.
This page was las modified on 2020-09-30

Website Map
Resources & help
Index of Docs
33HOPS Forum

©33HOPS site relies on the following technologies and technology partners:
SSL Protocol PayPal Payment Gateway Stripe Payment Gateway

©33HOPS Sistemas de Información y Redes, S.L. | VAT No: ESB83583716 | Avda. Castilla la Mancha, 95, local posterior, 28701 San Sebastián e los Reyes (Madrid) Spain

Fill in to download
The download link will be sent to your e-mail.

            Read our Privacy Policy