Last updated on Tuesday 16th of August 2022 03:14:37 PM

©XSIBackup-DC

Change log

  • 1.5.2.1 (current)
    • Improvement: added --remove-all-snapshots argument to allow forcing deletion of any eventual pre-existing snapshot in the VMs to backup/replicate.
    • Improvement: new mechanism to prevent some job from being fired more than once. This can happen under some circumstances, like with multiple cron instances or some human error. A job with same action + source is considered to be a duplicate job.

  • 1.5.2.0 Major Improvements
    • Bug fix: disks of type sata are also considered when using CBT feature, until now only SCSI and IDE would work with CBT.
    • Bug fix: when trying to perform a CBT backup on a set of VMs where one of the VMs would raise an error due to a missing -ctk.vmdk file the rest of VMs in the set would fail to be CBT backed up even if their CBT scheme was OK.
    • Improvement: ©XSIBackup now also works as a command line tool (no GUI by now) in Linux servers (experimental) for regular file content with almost the same features as for ©ESXi VM backup. It's aim was always to become a general purpose tool also in Linux command line scenarios. Symlinks are ignored, only files and dirs are backed up/replicated.
    • Improvement: the binary bin/base64 has been renamed to bin/xsib64 to avoid conflicts with Linux systems already having a binary by that name (base64). This makes it mandatory to upgrade using the installer following the instructions in the README file, just updating the main xsibackup binary file IS NOT enough. Otherwise you will receive errors when the new xsib64 binary is not found. The client installation will take care to automatically update the file at the remote backup/replica hosts.
    • Improvement: we have taken on account your feedback and extended the functionality to more easily support other users than root in remote backup hosts. As keeping the binaries, logs, config files, etc... in a strict Unix File Hierarchy poses unavoidable permission issues, we have moved all ©XSIBackup related files in the Linux backup server to the folder /home/$USER/.xsi in such way that the remote user has full default user rights on all the files required to operate. Thus, now you only have to add the user to the Linux backup host, set a password and use ©XSIBackup as you would do with the root user. This allows to use ©XSIBackup on backup hosts on which you don't have root access but you have full /home/$USER access.
    • Improvement: added integrated memory management both: manually through the --memory-size[=MB] argument and automatic. This will help to manage edge cases with huge VMs bigger than 6-8TB and also --prune & --repair of big repos from ©ESXi, although as you should already know big repos should be hosted in Linux/XFS servers and the --prune/--repair should be accomplished from the server OS.
    • Improvement: improved formatting of errors in interactive mode.
    • Improvement: added random rotation to the list of dynamic target code snippets in the GUI.
  • 1.5.1.17
    • Bug fix: fixed incompatibility of cbt with VMs containing spaces in local backups.
    • Improvement: revised remote command execution routines.
  • 1.5.1.16
    • Bug fix: granular restore program bin/XSIGR/xsigr now supports spaces in paths of VMs and virtual disks.
    • Bug fix: when the username crontab at /var/spool/cron/crontabs/ would not exist an error would be returned on --update-cron=username. The bug could easily be worked around by issuing the command: touch /var/spool/cron/crontabs/username
    • Improvement: check for /tmp folder size. The job is halted and a message displayed inviting to increase the temp folder size in case you reach a critical threshold.
    • Improvement: the use of non-root users has been eased by loosening checks and somehow ignoring what username you are passing. This on the other side will move the eventual errors produced by any lack of permissions to the remote end. ©XSIBackup has been conceived as a service, although it allows interactive sessions, it is strongly recommended that you run it as root to avoid falling into permission issues.
  • 1.5.1.15
    • Improvement: --options are now editable from the GUI. Read the full option description in the manual before using them.
    • Improvement: the GUI now displays the version. The GUI tmp folder has been coalesced with the binary main tmp at /tmp/xsi/gui.
    • Improvement: new version of XSIGR available. Improved automatic unmount routines.
  • 1.5.1.14
    • Bug fix: resizing some terminal windows like Putty causes a SIGWINCH signal being sent by the TTY. This odd behaviour has been removed by ignoring the SIGWINCH signal, you can now safely resize your terminal window while in an interactive session.
    • Bug fix: some ©ESXi versions lack the %j modifier in the date command (so weird). To work that around we have replaced the expression $(date +%j) or Julian day (ordinal day of the year) by $(( $(date +%s)/86400 )) in the gui/dat/.dyntargets file, which will produce the same result.
    • Improvement: added [l] flag to --options to allow to test Regular Expressions in the VMs() argument without actually running the job.
  • 1.5.1.13
    • Bug fix: CBT feature was not fully compatible with exclusions, an error could be raised in some cases like when having RDM disks attached to the VM.
    • Bug fix: adding more than 9 arguments from the GUI would cause them to be recorded in the wrong order in the job file, requiring manual fix.
  • 1.5.1.12
    • Bug fix: spaces in local datastore names are now supported. Before you would receive the error can't open input file when attempting to hot backup a VM hosted in a volume with some space in its name. Spaces in VM names or intermediate paths were handled correctly.
  • 1.5.1.11
    • Bug fix: pruning restore points inside a nested folder via the --subfolder option would raise a SIGSEV at the end of the pruning process and not remove the restore point folder. It has now been fixed.
  • 1.5.1.10
    • Bug fix: adding servers from the command line would always result in a new SMTP Id of 1, causing smtpsrvs.conf files with more than one SMTP line to fail and requiring a manual fix. All other methods: GUI and manual editing of file would just work fine.
  • 1.5.1.9
    • Improvement: the exit status of the backup job is now always printed out in the e-mail report subject, even when some custom subject is set. The custom subject will be prepended to the exit status..
  • 1.5.1.8
    • Improvement: .vmsn files accumulated in remote replica folders are now cleaned up by default. Only the .vmsn files referenced by the .vmsd file are kept.
    • Improvement: added new argument --enable-vsyscall. Some newer Linux OSs (like latest Debian 10 & 11) have deprecated vsyscall in favour of vdso, which causes (c)XSIBackup to return a SEGFAULT when accessing the system clock as a client. Running ./xsibackup --enable-vsyscall will add vsyscall emulation to your Linux host.
      /!\ This command will reboot your host automatically after enabling vsyscall emulation.
    • Bug fix: the --smtp-add argument now disables the firewall to probe the SMTP server and to avoid having to switch it off manually. ©ESXi FW is becoming more limited with each release, making necessary to halt the FW service during some fractions of a second to reach some outer ports. The FW is also disabled while the e-mail is sent in ©ESXi > 6.7.0 U3, in builds below 6.7.0U3 a custom rule is added instead and enabled/ disabled as needed. This should not pose any real security issue, as the e-mail submission just takes some fractions of a second, still you must know this fact.
    • Bug fix: --description, --subject and --exclude now are properly double quoted when saved using the --save-job argument.
  • 1.5.1.7
    • Bug fix: the --rotate argument could fail under some circumstances when the backup volume would fill up.
    • Improvement: added new option (f) to force provision of space even on rotation by number.
    • Improvement: added new option (S) to consider VM size based on -size on disk- or non zero data in sparse files.
    • Improvement: added new argument --fsblocksize to set the underlying FS block size on NFS shares. The default size is 512 bytes and should work fine most of the times. You will rarely use this argument.
  • 1.5.1.6
    • Improvement: added option to check and install licenses from the GUI.
  • 1.5.1.5
    • Bug fix: combining REGEXP expressios in VM selection with _XSIREP VMs would not work as expected, it has been fixed in this version.
    • Improvement: additional info and tip associated to missing CTK files when enabling CBT in a stopped VM and not starting it to generate them
  • 1.5.1.4
    • Bug fix: added redumdant check to make sure the /tmp/xsi folder is created even when a non-root user is employed to perform a --prune operation.
  • 1.5.1.3
    • Bug fix: non-critical bug would cause temp manifests to be accumulated in the root dir "/". In mixed environments (jobs with =cbt and non cbt enabled VMs) a non CBT enabled VM could cause that the subsequent ones were considered normal backups/ replicas. This would force the user to split jobs into CBT and non-CBT, which BTW is a good practice in any case.
  • 1.5.1.2
    • Improvement: added option to override _XSIREP filter --options=O
    • Improvement: --enable-cbt and --reset-cbt arguments don't start a VM if it's in a stopped state
  • 1.5.1.1
    • Bug fix: fixed small spurious error complaining about a missing .blocklog file when creating a new repository.
  • 1.5.1.0
    • Improvement: now CBT is also available for --backup action (only 1M block size).
    • Improvement: improved management of remote dependencies. New binary base64 added to bin folder /!\ Perform a full install.
    • Improvement: now CBT works also for disks distributed in multiple datastores or VM folders (absolute paths).
    • Improvement: .vswp files used for memory overcommitment are now excluded from the backups if they are present.
    • Improvement: files with .skip extension are now skipped, this allows to rename unused files (i.e. old disks) to be excluded from the backups but still be kept in the VM folders.
  • 1.5.0.14
    • Improvement: improved detection of SCSI disks.
  • 1.5.0.13
    • Bug fix: fixed bug in replica rotation.
  • 1.5.0.12
    • Bug fix: fixed bug selecting remote path in GUI.
    • Bug fix: workaround for some faulty OpenSSL implementations in older ©ESXi releases from the 5 & early 6 branches..
  • 1.5.0.11
      This is a minor release from a feature point of view, still we have made rather profound changes to allow this new feature improvements and the overall performance and reliability has been improved.

    • Improvement: full REGEXP support for VM selection, not only to exclude VMs up until now.
    • Improvement: the --rotate argument now supports two additional modes and their corresponding flags: size (GB | TB) and max. It also auto detects the last folder format and in case it is 'YYYMMDDhhmmss' and you have set the rotate option for a --replica, it rotates the replicas at that root point. We have also added the --rotate-at argument, which allows to set the rotation root from part of the user.

      This set of improvements is aimed at allowing users to rotate replicas, as we have detected that many of you like to keep a set of replicas instead of creating deduplicated repositories with many restore points. It makes sense as the replicas are ready to use, it's not very efficient from a storage utilization point of view, still it's a way to go under some circumstances.
    • Improvement: pruning can now be performed over IP. In reality, although it will still be performed locally on the server side, you will receive real time output on the client side.
  • 1.5.0.10
    • Bug fix: removed spurious errors.
    • Bug fix: snapshots are now kept when running jobs in ©ESXi 7.0 when using the --backup-how=cold flag.
  • 1.5.0.9
    • Bug fix: removed spurious notice on 'data' folder creation.
    • Bug fix: limitation to copying NFS based disks in latest versions has been waived.
  • 1.5.0.8
    • Improvement: error reporting has been extended to all functions, only unhandled errors will produce no e-mail report.
  • 1.5.0.7
    • Bug fix: the recently added SMT server regexp checker would fail due to the misplacement of a hyphen character in a character class, this would affect other characters too. The port of the SMTP server has been extended to allow using up to 5 characters for those of you willing to use a custom non-standard port
    • Bug fix: when combining --replica=cbt action type plus --options=R to register the replicated VM in a local target datastore with VM selection modes RUNNING or ALL, the resulting replica VM_XSIREP would again be selected as a running VM or as part of the ALL group. After this bug fix VMs ending in _XSIREP will be skipped by the RUNNING and ALL VM selection keywords.
    • Improvement: now when any error signal is trapped, the errors are reported via SMTP, even if the job is halted and can't be run. The only circumstance in which you won't receive a report would be a totally unhandled exception, which are rare.
    • Improvement: when combining CBT arguments and flags with VMs which aren't correctly configured for CBT (CBT disabled, missing -ctk files, etc...), some errors would be raised. We have added some extra logic so that when such conditions are met a warning is raised and the CBT arguments are ignored to ease hearding ©ESXi hosts in sets and also making output nicer to the user. Still, you must properly configure your hosts and comprehend how ©XSIBackup works before using it.
  • 1.5.0.6
    • Bug fix: wrong post processing of hashes in CBT replicas could cause the CBT scheme to be reinitialized after every CBT cycle
  • 1.5.0.5
    • Bug fix: minor bug fixes
  • 1.5.0.4
    • Improvement: added 'mail-from' keyword to SMTP client as a valid e-mail address. It autosends the report to the same 'mail-from' address configured in the SMTP server entry
    • Improvement: new checks on SMTP file etc/smtpsrvs.conf, now each line is matched agains a REGEXP to prevent empty variables from bubbling up.
  • 1.5.0.3
    • Bug fix: regexp filter simulator would not work in the GUI when selecting the --exclude argument.
  • 1.5.0.2
    • Bug fix: missing file 'regexfilt', a helper to build regular expressions from the GUI was added to the bin/ folder.
  • 1.5.0.1
    • Bug fix: fixed spurious bug when not finding a remote blocklog file in --replica mode.
  • 1.5.0.0
    • Bug fix: multiple minor bug fixes and refinements.
    • Improvement: CBT Changed Block Tracking compatibility for the --replica action with target multitenacy, allows to replicate a VM via CBT to multiple target systems.
    • Improvement: non-interactive mode available for the install script via the -y switch, allos to automate installation.
  • 1.4.3.18
    • Bug fix: under some unfrequent circumstances, the .vmx file would not be arranged for the remote disk paths to match the resulting --replica or --backup layout.
  • 1.4.3.17
    • Bug fix: wrong disk detected when combining multiple sources and SCSI or iSCSI would precede NFS virtual disks.
  • 1.4.3.16
    • Bug fix: spurious error complaining on SCSI disk not found in case of NFS disks was removed.
  • 1.4.3.15
    • Improvement: filter out of annoying messages in the STDERR channel of OpenSSH like "FIPS mode initialized", which in turn trigger errors in ©XSIBackup
  • 1.4.3.14
    • Bug fix: disks in secondary locations, that is: other folders than the main VM folder, were not being backed up
    • Improvement: improved restore menu in the GUI
  • 1.4.3.13
    • Bug fix: false positive in SMTP error detection could be raised when some line not in the very protocol conversation would start by an SMTP error number, like some encryption key line. The e-mail would still be sent but an error would be registered in the e-mail submission phase.
    • Improvement: additional check on remote restore to make sure you are passing a VM backup folder
    • Improvement: when --remote-path is used ©XSIBackup will not try to update the remote server binary, even if you use the default path. This allows to use --remote-path as a way to bypass the remote binary check/ update.
  • 1.4.3.12
    • Bug fix: --add-key was halted from the command line due to an spurious check. The exchange key function was working fine from the GUI, as it's based on a different method.
    • Improvement: server role has been revised to limit the xsibackup binary to read data from the SSH tunnel and write to the backup repository or replica folder when using a different user than 'root'. Setting hyerarchical backup topologies up with multiple users can be difficult due to the fact that xsibackup has been conceived as a service and not so much as user software. Thus it requires permissions to create folders and write files on parts of the FHS (Filesystem Hierarchy Standard) where other users than root normally can't, as /etc and /var/log apart from temp folders.

      When xsibackup is running as a server and some user other than root is detected, it will not try to create folders or files in restricted parts of the FHS. You should then first run some command as root to create those files to avoid errors on subsequent runs with a non privileged user.

      In case some error is raised in the server role, the user with which you invoke the server user@a.b.c.d:22:/some/repo must have rights to create and write logs to /var/log in case of Linux servers. The more privileged the user is, the less permission problems you will find.

      In ©ESXi systems running as a server, the situation is more controlled, as the etc and var folders are stored under the installation root, thus you can assign permissions to the bin file and also the etc and var folders as needed.
  • 1.4.3.11
    • Improvement: added size of differential data to the e-mail report
  • 1.4.3.10
    • Improvement: warnings related to cron usage removed in the GUI for Free edition
    • Improvement: added cron summary where each cron line can be viewed in a easier to understand language than classic cron syntax
  • 1.4.3.9
    • Bug fix: fixed bug in restore of VMs over IP
    • Improvement: enabled control of cron in the GUI for the Free edition
    • Improvement: added detection of GUI window size with alert when it is sized under the working threshold
  • 1.4.3.8
    • LIMITATION WAIVED: now replicas are fully enabled to local repositories up to the Free version VM size limit of 60GB.
  • 1.4.3.7
    • Bug fix: empty folders were not being removed from repositories when performing prune rotation.
  • 1.4.3.6
    • Improvement: repository folders are again created on local backups if they don't exist.
  • 1.4.3.5
    • Bug fix: duplicate check on files when running --check on a replica could be caused by a missing string NULL terminator.
    • Improvement: optimized buffering on --check commands increases speed on HDs checks.
  • 1.4.3.4
    • Bug fix: when the first six hex chars of the last block to prune would coincide with the index's last entry, a segfault would be raised at the last phase of the pruning process, after having indeed pruned the blocks themselves. This had no other consequence than leaving orphan blocks in the .blocklog file. This has been fixed in this release.
  • 1.4.3.3
    • Bug fix: garbage collected from server side component in first over IP backup when the xsibackup binary would be automatically updated could make the final job phase to raise a false failed checksum.
    • Bug fix: minor bug fixed in adding SMTP server --smtp-add.
    • Improvement: xsilib binary is also automatically updated in case of an ©ESXi host detected as the backend in an over IP backup.
    • Improvement: reorganization of --smtp-* functions.
  • 1.4.3.2
    • Bug fix: ugly but irrelevant bug, ©XSIBackup-DC tries to copy the xsibackup binary when running --add-key
    • Bug fix: final checksum verification could fail in some circumstances on Linux appliance guests.
  • 1.4.3.1
    • Improvement: you don't need to manually copy the xsibackup binary to the remote host. Now the binary is copied if it does not exist or is updated in case a different version is detected on the remote end. It is copied to /bin in case of Linux systems or /scratch/XSI/XSIBackup-DC in case of ©ESXi hosts. You may still use some binary at some arbitrary remote path by using the --remote-path argument.
  • 1.4.3.0
    • Bug fix: some issues related to license file recognition were fixed in additional builds.
    • Bug fix: fixed problem with repo lock file on rotation.
    • Bug fix: command line --add-key argument wasn't working well. The equivalent option was nonetheless working in the GUI.
    • Improvement: added support for ©ESXi virtual machines in suspended status.
    • Improvement: improved SMTP protocol messages, errors are captured and shown on screen.
    • Improvement: improved block seek time on big repos (> 1TB), now this time remains stable as the repos grow and allows bigger repos on 1MB block size.
  • 1.4.2.9
    • Bug fix: fixed problem detecting license files.
  • 1.4.2.8
    • Bug fix: fixed problem with repo lock file on rotation.
    • Bug fix: fixed issue deleting /tmp/pid folders.
    • Bug fix: /scratch/downloads folder clean up after --config-backup will prevent .tgz config backup files from accumulating there in between reboots.
    • Improvement: subfolder cfgbak has been added to repository to host config backups.
  • 1.4.2.7
    • Bug fix: sending report to more than one e-mail address was not working due to the e-mail validation being applied to the whole comma separated string.
    • Bug fix: backup of ESXi config over IP was failing due to the server side shutting down before receiving the backup file.
  • 1.4.2.6
    • Improvement: helper routine to help set up smtp.gmail.com accounts from the command line.
    • Improvement: SIGTERM(2), namely execution stopped as per user request, is treated differently and is not trapped as an error, but just displayed and informed as such.
    • Improvement: faster locking mechanism for pruning operation.
    • Improvement: improved installer now takes care to keep user config files: smtpsrvs.conf, xsibackup.conf & crontabs.
  • 1.4.2.5
    • Bug fix: VMs with similar names (name of one VM contained within another), can cause the wrong VM to be selected for backup or cause issues during the backup. This has been fixed in this release.
    • Improvement: ghost entries in the VM inventory, namely: VMs that have been deleted from disk but have not been unregistered from the ESXi inventory, would trigger a generic error, now it is more verbose and the cause is explained.
  • 1.4.2.4
    • Bug fix: fixed issue affecting last block in backed up virtual disks with the potential to write that last block wrongly.
    • Bug fix: revised and fixed issues relative to backup of disks hosted in NFS datastores.
    • Bug fix: the client controls the server side behaviour. Up until now the client would just stop writting to the server when done, this would cause an error on the server side in the form of a SIGPIPE(13), as the SSH tunnel would just get disconnected. Since this version the client writes signals to the server at the application layer, so that the server side is notified when the client is done in such way that the server binary ends execution in a nice way.
    • Improvement: final checksum on over IP backups compares the number of bytes sent with the number of bytes effectively written on the server side, this is done on a per job basis and prints a green notice in case everything matched or a red one in case something went wrong and figures don't match. This check is done on addition to pre-existing partial checksums on every individual block.
  • 1.4.2.3
    • Bug fix: fixed bug in block detection. Would not affect backup integrity but could halt a job.
    • Improvement: optimized memory usage in --prune operation.
  • 1.4.2.2
    • Bug fix: recursive pruning could raise a segfault. Not critical, subsequent --prune operations would continue the pruning process.
  • 1.4.2.1
    • Improvement: search in the block manifest is performed by means of a new detection algorithm which includes native C sorting and a dynamic index, which drastically reduces the amount of CPU cycles needed to detect a block.
    • Improvement: ©ESXi 7.0 blocks all files in a VM for reading, existing snapshots can't be backed up. Now the backup/ replica process deletes any previously existing snapshot previous to performing the backup, so that all your VM data is backed up. This is done only when ©ESXi 7.0 or above is detected.
  • 1.4.2.0
    • Improvement: all pruning operations have been redesigned to be faster and to support a bigger volume of data.
    • Improvement: all block indexing related to pruning is now performed via C native functions and dynamic indexing of data is employed vs an old indexing fixed depth system that used little memory but was slower and had hardcoded limits, as it depended on the sort binary.
  • 1.4.1.2
    • Bug fix: bug in cron launched job detection would cause jobs not to be executed from the crontab
  • 1.4.1.1
    • Bug fix: restore module now re-truncates disks to their nominal size after restore process, just to make sure disks with eventual empty ends are rebuilt correctly
    • Bug fix: a 300 GB limit was wrongly imposed on licensed versions since previous version 1.4.1.0
  • 1.4.1.0
    • Improvement: New argument --config-backup will copy the ESXi Config to the remote repository or replica folder.
    • Improvement: Execute permissions are now set for each backup job everytime they are saved in the GUI.
    • Improvement: Pruning alone as an action or per the --rotate argument after a backup is now fully supported in the GUI and a progressbar in two phases (collecting blocks and deleting them) is shown.
    • Improvement: Little routine to avoid interpreting malformed remote paths as if they were local.
    • Improvement: Improved error detection from the GUI layer..
  • 1.4.0.0
    • Bug fix: a bug prevented 1.3.0.2 from generating backups when using a block size over 1MB (default). You can again use bigger block sizes
    • Bug fix: extremely verbose progress on cron launched --replica jobs. Fixed in this version, only user launched jobs produce a verbose progress.
    • Compatibility: this version is fully compatible with ©ESXi 7
    • Improvement: VMs can be restored from a remote folder on any SSH server. This makes ©XSIBackup-DC a fully capable cloud backup solution
    • Improvement: matching of client and server ciphers is now more user friendly. Only if no matching cipher is found the tunnel negotiation will not be stablished.
    • Improvement: since this version you can run multiple concurrent jobs in a single host. Nonetheless we highly discourage such practice and recommend running jobs sequentially.
    • Improvement: a full improved GUI has been added to this version. It is a dialog/ nCurses User Interface which allows to manage all main functioning aspects of ©XSIBackup-DC
  • 1.3.0.2
    • Bug fix: a bug was producing wrong backup chunks since version 1.3.0.0 only on backups performed on Virtual Machines residing in NFS volumes. It has been fixed in this version. If you host production VMs on NFS shares, you should update immediately and discards backups made with versions 1.3.0.0 and 1.3.0.1
  • 1.3.0.1
    • Now allows sending e-mail from ESXi 7.0 by disabling the firewall during the e-mail submission lapse
  • 1.3.0.0
    • Compatible with ESXi 7.0.0. If you run ESXi 6.7 or below you should use ©XSIBackup-DC 1.1.1.2
    • An installer has been added to the downloadable package
    • Some routines have been improved
  • 1.1.1.2
    • Bug fixes: fixed detection of duplicate known_hosts file entries when managing keys to avoid unhandled errors or blocking of command line.
  • 1.1.1.1
    • Improvements: temp dirs split into two locations: /tmp/xsi and [install root]/tmp. Temp files requiring speed are placed into the first one, while storage intensive operations are moved to the second. This will allow to improve backup speed by 30% in some cases where (c)XSIBackup-DC is installed to a HD.
  • 1.1.1.0
    • Improvements: two new arguments have been added: --smtp-add and --smtp-test. They allow by means of a command line easy to use UI to respectively add an SMTP server to the list of SMTP servers in the etc/smtpsrvs.conf file and to test any of the existing servers by sending a test e-mail to any e-mail address.

      You can just upgrade by overwritting the xsibackup binary.
  • 1.1.0.9
    • Improvements: improved resiliency to fatal errors in scenarios where you use folder backup/ replica source on ESXi VMs which aren't correctly registered or have missing components.
  • 1.1.0.8
    • Improvement: general block log is recounted to make sure that we consider all blocks that must be kept after pruning
    • Improvement: (c)XSIBackup-DC uses a TMP dir at <install root>/tmp in ESXi, which will tipically be /scratch/XSI/XSIBackup-DC/tmp, or the /tmp dir in different Linux flavours when working as a server, that comprises  Synology DSM OS too. The scratch volume in ESXi generally is 4.00 GB in size while the /tmp folder or filesystem ranges from one Linux distro to the other and goes from 500 MB to some GBs in size.

      The above will be more than enough when using big block sizes for deduplication. Nevertheless, should you use some backup repository with a block size of 1MB and let it grow to some hundreds of GBs, the prune process can become quite resource hungry in terms of space. In those cases you will need to grow the TMP file system or move it somewhere else where more room can be allocated to store temporary logs.

      In this version we have introduced the alt_tmp_dir variable in the <install root>/etc/xsibackup.conf file which allows to pass a list of additional TMP directories to use when the default dir is not enough.

      (c)XSIBackup-DC will detect whether switching to an alternative TMP dir is required for pruning. Nonetheless you should always try to grow your /tmp dir in your Linux server to some tenths of GBs in these cases, thus moving the TMP is not necessary.
  • 1.1.0.7
    • Bug fix: .vmsd file would need some manual tweaking if some snapshots were saved along with the VM to be able to switch it on with the exact same configuration as in the source VM. Now the .vmsd file is automatically arranged to match the existing snapshot structure no matter the number of snapshots.
    • Bug fix: exit code was always 0, that is a bug, we fixed this in this version. Now the exit code is the number of accumulated errors at the moment the program exits.
  • 1.1.0.6
    • Bug fix: empty lines in the .vmx file could result in a truncated target .vmx file. Nevertheless your .vmx file should not contain blank lines at all, especially in the middle of the file, blank lines at the end would not matter except for the different outcome in size. As ESXi allows (we should probably say, it tolerates and ignores them as well) this empty lines XSIBackup-DC will just do the same from now on
  • 1.1.0.5
    • Improvement: now (c)XSIBackup-DC finds its remote counterpart more easily. From this version xsibackup binary is searched for on remote's /usr/bin folder first, if it can't be found there a which command is run and, if positive, its returned path is used as the remote xsibackup bin location. If both of the previous methods fail, the binary will be searched for in the same path as in the client side. If all of the previous locations do not return a match you must set the --remote-path argument to explicitly set the remote binary location.
  • 1.1.0.4
    • Bug fix: empty lines in the ./etc/xsibackup.conf file would cause a SEGFAULT, now they are ignored.
    • Improvement: annoying 'FIPS mode initialized' message can interleave in the STDERR FD in the OpenSSH tunnel raising errors. We have tried to make the inner OpenSSH subprocess quiet there where a single value is expected. Please OpenSSH developers, stop spitting irrelevant information into STDERR
  • 1.1.0.3
    • Bug fix: visual bug when showing user defined block size units, bytes printed, but MB was appended to amount
    • Improvements: now the xsibackup.conf entry ssh_ciphers supports the 'auto' keyword as a value. Thus you have multiple options to control what cipher to use for data transfer: auto, default in .conf file, per job via the --ssh-ciphers argument and as per the l option (Less Secure Ciphers) in the --options argument.

      This update requires that you update the ./etc/xsibackup.conf file as well as the xsibackup main binary.
  • 1.1.0.2
    • Bug fix: interstitial spaces would still cause trouble in some systems, we fixed this and now spaces should work seamlessly locally or over IP. Nevertheless you should not forget to wrap any path containing spaces into double quotes.
  • 1.1.0.1
    • Bug fix: interstitial spaces in VM name or path would raise a SEGFAULT in transfers over IP --replica or --backup
    • Improvement: license is checked and in case it doesn't match a clear message is printed out along with the reason
    • Improvement: time and speed info added to the replica --ckeck action
  • 1.1.0.0
    • Improvements: additional file buffering is performed on the server side so that the update of virtual disks and hash tables is turned into an atomic operation, namely an inode re-assignment. This makes --replica a lot more resilient, as any file transfer that is interrupted will be re-taken from a 0 offset This guarantees file and hashmap consistency.
    • Improvements: .vmdk files are checked against its stored hashes before starting a --replica job. When some change is detected the hash tables for all disks are rebuilt to match the underlying data. This allows to switch on the --replica VMs without having to delete the replicated folder and start from scratch, and: what is more important, you don't have to remind to keep --replica consistency by hand.

      Since this version (1.1.0.0) the server installation will require the xsilib binary, thus you should get used to install the full package in the server side as well as in the client.
    • Improvements: reinforced error handling in --prune action
  • 1.0.1.4
    • We have refined the way remote files are synchronized during a replica. Some tasks like keeping its size (truncating) and simple cheksums are now performed from the server role and do not depend on Linux components
  • 1.0.1.3
    • Bug fixes: VM configuration files would not be truncated when their size is reduced from one backup cycle to the next, i.e.: .vmsd files may present this behaviour. This would only happen with some config files, would not affect disk files.

      All users interested in the --replica feature should upgrade to this version. And remember, replicating is not backing up. The --backup feature is much more robust by its own nature and can overcome almost any circumstance, replicating depends on a stable enough connection.

      GENERAL INFO WHEN UPGRADING: (c)XSIBackup-DC is even easier to upgrade and maintain than (c)XSIBackup-Pro, as it is constituted by a main binary (xsibackup) and just an additional library xsilib which performs some additional tasks just in ESXi (you don't need it in Linux). So all you need to do is overwrite the main binary. We have performed only one upgrade on the accompanying library xsilib in the last months, in any case you can overwrite it too, just in case.
  • 1.0.1.2
    • Improvements: block files are transferred to a buffer, tested then renamed with test. This adds an additional layer of tests ensuring block map integrity and retaking partial replicas is now more resilient.
    • Improvements: license is now only checked when using the --backup or --replica actions, rest of actions including server role are free.
    • Improvements: as you should know, modifying a replica will corrupt the replica data. You must leave replicas alone until you need them. Once you switch a replicated VM on, you can't continue to replicate to it.

      Now you can --check replicas too, not only backups. Checking a replica ensures that the data is the same as the original without the need to switch the VM on.

      If you still want to switch the replica VM on and continue to replicate to it, there are ways to circumvent this inconvenience, like creating a snapshot on the replica from a stopped state, switching the replica on and discarding the snapshot when you are done.

      ATTENTION: please be aware that discarding a snapshot is not the same as deleting it. In virtualization language deleting a snapshot is integrating it with the base disks and that would also break your replica.
    • Bug fixes: SSH subprocess wasn't conveniently killed in some circumstances
    • Bug fixes: Some keys send different signals to the main process if you hit them while the program is running in user mode. We try to control everything going on, but some signals can't just be discarded, so be careful not to strike keys randomly while running jobs in user mode ;-)
  • 1.0.1.1 -- All (c)XSIBackup-DC users should upgrade to this version, specially if using pruning --
    • Data chunks sent over the SSH tunnel are counted and a simple application level checksum mechanism is set on top of the TCP layer and SSH tunnel signalling.
  • 1.0.1.0 -- All (c)XSIBackup-DC users should upgrade to this version, speciall if using pruning --
    • Bug fixes: we have fixed some bugs in the --check and --info actions that would make them slow finding .map files in big repositories with small block sizes.
    • Bug fix: pruning is a delicate process, running out of space in the /tmp dir could lead to data loss. We have moved the main tmp dir used for pruning to the installation root in ESXi. Recommended location is /scratch/xsi, default size is 4GB, more than enough to handle multi-TB multi restore point repositories. You may move /scratch to a bigger volume in case you need more room. Read this post to delve into pruning details.
    • Improvement: --prune module has been completely reprogrammed to be faster. It was enough to prune 10MB block sizes and above, but would take long and require tweaking in the OS to prune big 1MB repositories. Read this post to delve into pruning details.
    • Improvement: fprintf() function has been replaced by fwrite() in some parts where recursiveness would make fwrite() be faster than fprintf() this improves performance a bit.
  • 1.0.0.8
    • Fixed: issue with --update-cron command in ESXi 6.7.0 U3
    • Fixed: minor bugs
  • 1.0.0.7
    • Fixed: since --compression option was set as default, only editing the etc/xsibackup.conf file would allow to disable compression. Now it can be set through the --compression=[0|1|true|false|yes|no] option again.
    • Improvement: default block size for backups has been changed to 1M in alignment with the ESXi block size. 10M, 20M and 50M are still available to use. Please, note that although 1M block size is a must to backup over IP and it is indeed faster when used over SSH/IP, you might get better speed results with bigger block sizes when backing up to a local datastore. Enabling jumbo frames can be of great help too at the time to increase throughput.

      Bigger block sizes allow also to use VMFS volumes as the target of deduplicated repositories. VMFS 5.0 declared to be able to host around 132,000 inodes which is a very limited number to host lots of tiny chunks of data, nonetheless VMWare says that VMFS 6.0 is able to host millions of files, which is more than enough for (c)XSIBackup-DC to be able to host many terabytes of data even at 1M block size.

      In any case, being able to use NFS to backup to underlying XFS or EXT4 file systems makes VMFS not that interesting or needed for your backups.
  • 1.0.0.6
    • Improvement: the xsibackup.pid file has been moved to /tmp/xsibackup.pid so that using any Linux distro as an IP target is eased in regards to permissions required when compared to the /var/run folder previously holding that file
    • Improvement: we have added extended debug information to some critical error messages to help narrow down NFS connectivity/ permission issues
    • Fixed: small bug when there are no blocks to prune this fact was previously informed as an error
  • 1.0.0.5
    • Added: clean-up of var/spool/mail and var/log/prog directories on load. Latest e-mail templates and progress information available in these directories will remain available in between backup cycles for you to inspect. Until this version they would accumulate in those dirs requiring manual clean-up
  • 1.0.0.4
    • Fixed: e-mail subject would not inform of errors during the backup process. They were informed in the e-mail body though.
    • Fixed: partial error count per VM was not logged properly.
  • 1.0.0.3
    • Added: accepting EULA on first run is now mandatory.
    • Fixed: minor bugs
  • 1.0.0.2
    • Fixed: the xsibackup.log was wiped on every new execution, now new output is appended.
    • Fixed: progress info would stuff the xsibackup.log file with every progress update in a new line, resulting in a lot of useless information being logged.
  • 1.0.0.1
    • Improvement: request.key file is not generated unless a real command is run. We changed this behaviour as some users might prefer to install their license first. To generate a request.key file just run: ./xsibackup --request-key
  • 1.0.0.0
    • Bug fixes: final release
  • 0.9.8.2
    • Compatible with Synology devices as a native binary. Requires to configure root access to the Synology appliance.
      The --link-srv action will work out of the box for user root. All other users different than root store their configuration files at custom paths in Synology NAS devices, thus by now you need to perform the key exchange manually.
  • 0.9.8.1
    • Small bug fixes and output texts retouch
  • 0.9.8.0
    • Unified log subsystem for client and server.
    • Improved --check routine with more accurate progress and some bugs fixed. Before this version it would report small files as K.O. due to being smaller than the block size, which was a gross bug.
  • 0.9.7.2
    • Accumulated time added to the output and the e-mail report.
  • 0.9.7.1
    • Last errors data added to the e-mail report footer. It can handle up to 8KB of errors, beyond that point you must consult the error.log file.
  • 0.9.7.0
    • Rotating data can now be achieved from an IP client, before the --rotate option was only available in local backups.
  • 0.9.6.2
    • Bug fix: signal handling in the IP server side would send a 13 signal to the client when triggering jobs from the cron, which would in turn record it in the log. It caused no additionalissues, apart from overpopulating the xsibackup.log file.
  • 0.9.6.1
    • The way to use ciphers has been improved and normalized. (c)XSIBackup-DC will pick the ciphers defined in the ssh_ciphers variable inside the etc/xsibackup.conf file. You may supersede the default list of ciphers by using the --ssh-ciphers argument in the command line. By doing so the default list of ciphers is replaced by your own custom one.

      To set a custom list of ciphers, you need to find out which ciphers are available in your OpenSSH build in order to not get a cipher negotiation error. It is arguable whether OpenSSH should simply ignore an unknown cipher, but the truth is that you must be careful as a single mismatch will cause an error.

      You may also use the --options=L switch, which is equivalent to set a light encryption deprecated list of ciphers or "Less Secure Ciphers" like:
      --ssh-ciphers=arcfour,blowfish-cbc,aes128-ctr
  • 0.9.6.0
    • --backup-how argument is introduced. It works the same way as in the Classic XSIBackup edition, it can take one of 3 values: hot (default), cold and warm.
    • etc/xsibackup.conf file is now functional, allowing the user to tweak some general scope variables
  • 0.9.5.3
    • Trial Version notice appeared on every execution even if the server had been just restarted. This was a bug, the Trial Version limits have been restored.
  • 0.9.5.2
    • Improved progress information, now offering simultaneous per file and global percentages in real time
    • Fixed bug in open port detection prior to performing action, non standard ports could not be used in over IP transfers
  • 0.9.5.1
    • Support for ALL and RUNNING has been added in the virtual machine selection syntax, i.e: "VMs(ALL)" and "VMs(RUNNING)"
    • Automatic removal of backup snapshot when the backup process is aborted by a Ctrl+C condition
  • 0.9.5.0
    • We have redesigned PID management to make it more usable and avoid loops
  • 0.9.4.6
    • Bug fixes mainly
  • 0.9.4.2
    • HTML report may pick templates from the /var/html folder. Templates must be named 000 to 999 and placed in this folder as: 000.html, 001.html, 002.html and so on.

      You can design whatever HTML layout you wish and then include the <!-- PLACEHOLDER REPORT --> comment in its own line. You may then invoke the template by using the option --html-template=NNN
  • 0.9.4.1
    • Extended HTML report functionality, fall back options and SMTP servers panel at /etc/smtpsrvs.conf.
  • 0.9.4.0
    • Basic HTML report functionality.
  • 0.9.3.9
    • Fixed the "Permanently added 'a.b.c.d' (RSA) to the list of known hosts" error on reboot.
  • 0.9.3.8
    • Added --exclude argument. Allows to exclude disks by parsing a REGEXP pattern. If you want to exclude multiple disks use the --or-- operator in your REGEXP
    • Added --update-cron=[user] action. Inserts the user crontab contents into its corresponding ESXi user crontab
    • Added --install-cron=[user] action. Adds the user crontab to the ESXi /etc/rc.local.d/local.sh file, so that crontabs are persistent across reboots
    • Added --uninstall-cron=[user] action. Removes the user's crontab from the /etc/rc.local.d/local.sh. You still must comment or remove the cron lines and --update-cron=[user] to completely disable cron jobs for that user
  • 0.9.3.7
    • --check=full action redesigned.