Registered users
Linkedin Twitter Facebook Google+
  This website uses its own cookies or from third parties Close
33HOPS, IT Consultants
33HOPS ::: Proveedores de Soluciones Informáticas :: Madrid :+34 91 663 6085Avda. Castilla la Mancha, 95 - local posterior - 28700 S.S. de los Reyes - MADRID33HOPS, Sistemas de Informacion y Redes, S.L.Info

<< Return to index

XSIBACKUP  Automated Backups for VMWare ESXi

Copyright (C) 2013-2015 33HOPS, Sistemas de Información y Redes, S.L.
Developer: Daniel Jesús García Fidalgo
ALL RIGHTS RESERVED.

You are allowed to use this software for personal or commercial use. You are allowed to redistribute it without any modification on a one to one base. You can modify it's source code freely just as long as you do not redistribute the modified source code.

It is expressly forbidden to distribute derivative works without the written consent of the author.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY. USE AT YOUR OWN RISK.


CHANGE LOG:

11.1.5 (PRO) BUG FIXES:
• Argument --update-cron was not working from the command line, the GUI was O.K., we took the chance to improve the output.

• The text "# Added by XSIBackup" was being appended from the command line to cron entries, but not from the GUI. The GUI would not use it to detect XSIBackup cron entries, so if you would use the command line and the GUI at the same time, that could lead to duplicate cron schedules in /var/spool/cron/crontabs/root. This behaviour has now been made uniform.

• Some other minor bug fixes have been included in this release. The most important is launching a warning when some VMs in a backup job are no longer present in the host when editing the job.
11.1.4 (PRO) BUG FIXES:
• Virtual Machines hosted in non VMFS volumes would not be backed up, we have fixed this bug. Still, it is highly recommendable to host virtual machines in VFMS volumes.

• XSITools backups over IP have been enabled in ESXi 6.7. There were some minor bugs affecting this kind of functionality in VMWare ESXi 6.7
11.1.3 (PRO) BUG FIXES & IMPROVEMENTS:
• SSH tunneling in XSITools over IP deprecated in favour of direct individual SSH key authentication. SSH tunnelling sometimes requires deep tweaking of the SSH client and server, whereas direct use of SSH is more convenient. The time saved is not worth the hassle in case of using a tunnel.

• Block copies are retried up to 10 times. Once that maximum amount of retries has been reached (c) XSITools will return an error and will give up that file. This can help when copying across slightly unstable connections.
11.1.2 (PRO) BUG FIXES:
• Corrected little bugs in the install script that raise informative errors when the jobs and conf folders don't exist (older XSIBACKUP-PRO versions).

• Improved --save-job argument functioning. It would not handle an inexistent jobs folder, now it's automatically created if it does not exist.
11.1.1 (PRO) BUG FIXES:
• The file src/pro/xsitoolschk was not being copied to the remote server when using the --host argument.

• GUI: some fixes have been applied to prevent jobs saved via the --save-log argument from not being properly read by the GUI. All argument values are normalized to lower case to be read and avoid mismatches and derived errors.
11.1.0 (PRO) BUG FIXES & IMPROVEMENTS:
• New argument --save-job allows to save a backup job previously tested in the command line. Just add the --save-job string to the command line job.

• GUI: functionality extended to cover all arguments.

• (c)XSITools features deduplication over IP. 10M, 20M and 50M (default) block sizes are supported.

• Improvements in XSITools deduplication engine algorithm offers improved resiliency. Chunks are moved to a temp file which is renamed after a successful transfer, should something go wrong while the transfer is taking place, the chunk is resent on subsequent backup cycles, this prevents data corruption due to power outages or unstable TCP/IP communications.
11.0.3 (PRO) BUG FIXES & IMPROVEMENTS:
• GUI: --backup-room argument added to GUI.

• GUI: --del-dir argument added to GUI.

• GUI: bug in --subject argument setting removes double quote wrap, now fixed.

• Fixed bug when detecting backup folder in unstable NFS datastores.

• Fixed bug in GUI: unquoted --on-error and --on-success arguments created with the GUI pointing to scripts would fail.

• GUI: fixed bug that would prompt users to select VMs at the Disk Exclusion screen, when All or Running had previously been chosen. To exclude disks, always set --backup-type=custom.

• Line esxcli storage filesystem rescan has been added to xsibackup main file. It will make sure all connected datastores are refreshed when running the program. It may take some time in systems with a lot of connected storage devices. This line, which is now the first in the script can be safely removed, it will be useful in very rare occasions.

• Added Uninstall option to GUI tools.
11.0.2 (PRO) BUG FIXES:
• Fixed bug in src/pro/custimgs module that would cause custom images not to be set.
11.0.1 (PRO & FREE) Improvements and bug fixes:
• The free version incorporates all changes related to free features and is now compatible with ESXi 6.7.0

• Log rotation added, configurable size in conf/xsiopts, defaults to 50M.

• Additional control logic added to remote job execution. Now a UID (Unique Id) is parsed in the remote command and is returned to the master server once a job completes, to tighten backup chains logic and give more consistency to the process.

• Default quiescing behavior is to not quiesce, you must now explicitly add the --snapshot=doquiesce argument to quiesce your guests when performing backups. Most users run nightly backups on idle VMs that don't need quiescing. Configuring all services in your guests to ensure all services are properly quiesced is something that may require some time and will from your part. Those of you that require quiescing know exactly what they want, those that just want to perform a quick backup and don't care much about those picky details, most probably don't need to quiesce their VMs. So it makes more sense to explicitly say you want to quiesce than trying to always do by default.

• Text comment "# Added by XSIBackup" added at the end of every crond entry. Do not remove, this text allows XSIBackup to identify its own lines in the conf/[username]-crontab and /var/spool/cron/crontabs/[username] files.

• Fixed bug that would break HTML lines in the report causing flawed rendering in some e-mail clients.

• When a (c)OneDiff differential backup fails, the .vmx file is renamed to .vmx.bak and the OneDiff cycle is reinitialized on next backup round. Now backups made on VMs which are in this state, look for the .vmx.bak file and use it in place of the .vmx file, which allows to back them up.

• Disk exclusions now available in the GUI. It allows to exclude local and remote disks, when using the --host argument.
11.0.0 (PRO) FULL RENOVATION:
• XSIBACKUP-PRO 11.0.0 is the most ground breaking version since the program exists. It has been fully renovated in most important aspects of its functioning, like, cron, remote execution, backup chaining, etc..., start here.

• XSIBackup's crontabs are now fully integrated into ESXi's cron daemon and controlled via the GUI.

• Remote jobs are base64 encoded before being sent to the remote host and decoded there previous to being executed, this ensures no escaping issues with especial characters.

• Centralized backup job chain management. Now execution of remote jobs is truly controlled from a single installation point. Up until v. 11.0.0, when your parsed a remote job via the --host argument, once the remote job would end at the remote host, the execution chain would be delegated to the remote host, thus, the subsequent backup job would be sought in the remote host's crontab and not the master host where the chain originated. Now all jobs in a chain must just be present in the /jobs folder of the master host.

• Core functionality passed on to binaries in detriment of bash code loops.

When the project was started, we intentionally kept most of the routines in plain bash (dash to be more precise), to avoid use of busybox's command line utilities, like sed, grep or awk, as they demonstrated to not be very reliable. As newer ESXi versions were released, we started to trust those utilities a bit more and we have increasingly passed some functionality to them. This obviously makes the program be faster in regards to inner processing tasks.

• Some minor HTML errors have been fixed in the report layout.

• Fixed bug https://33hops.com/forum/viewtopic.php?id=240 It would prevent -sesparse.vmdk files from being backed up on piled snapshots.
---------------------------

10.3.4 (PRO) BUG FIXES:
• Compression flags removed from the SCP remote xsi-dir syncronization routine. Some ESXi builds may contain an SCP binary (busybox) that would not accept compression flags.

• Fixed escaping issue when parsing an --img-list to a remote job executed via --host argument. If you were not escaping the | (pipeline) character inside the --img-list argument it would break execution of remote job.

• Fixed bug that would turn --remote-xsipath variable to lower case if it contained upper case letters.

• Fixed bug that would prevent ESXi config backup.

• Added logic to handle delays in Async NFS. Some Async NFS systems take too long to create the backup folder, so that when the backup starts the folder is not available yet. This fix checks the existence of the folder up to 10 times before giving up. This is only a seldom problem in async storage, which is still not recommended, always use sync storage protocols.
10.3.3 (PRO) IMPROVEMENTS:
• Now the xsibackup-cron file accepts time arguments in the form "10t14:29", where the first two digit number represents the day of the month and the part after the t letter is a time string in hh:mm (hours and minutes separated by a colon).

• Added new xsbackup-cron.sam file in the conf directory with extended functionality to allow execution of any script or command line executable.

• Trapping of KEX messages to avoid errors being raised when OpenSSL tries to use deprecated KEX algorithms
10.3.2 (PRO) BUG FIXES & IMPROVEMENTS:
• Some minor bugs have been addressed and individual VM size (real data and disk size) are reported in the header of each VM backup log.

• XSIDiff binary upgraded to version 1.1.1.3

• Remote dir would not be deleted when using --backup-prog=vmkfstools without the --date-dir option, in this case the new backup must just replace the pre-existing one. This is probably not used by many users, so we had not had complaints so far.

• We have finally fixed issues with disk exclusions, we know there's been previous inaccurate announces about this, so we have to apologize.
10.3.1 (PRO) BUG FIXES:
• Wrong (c) XSIDiff version was packed with v. 10.3.0, this newer version solves it.
10.3.0 (PRO) BUG FIXES & IMPROVEMENTS:
• We have rewritten the PID management logic. It had been simplified in the latest versions, but this could affect users with complex or recursive backup topologies, as ongoing processes could be wiped by newer starting ones in some circumstances. Read this post to have an overview of the matter.

• XSIBACKUP-PRO and XSIBACKUP-FREE converge into a single package. Pro options are modules which are simply not present in the free version, some other features are activated by means of the (c) XSIDiff keys. The Free version will keep on offering the same characteristics as the old 9.1.11, plus (c) OneDiff (only 7 rounds, after which the OneDiff mirror is reinitialized) and (c) XSITools de-duplication, although the restore feature will only be available in the Pro version.
10.2.10 (PRO) BUG FIXES:
• Fixed small bug that would cause an empty list of dirs to be reported as deleted when using the :F flag at the end of the --backup-point argument..
10.2.9 (PRO) BUG FIXES & IMPROVEMENTS:
• Added --del-dirs argument. It accepts values following the format [+-][N][d]

• Main program file xsibackup's code has been wrapped between {} to prevent script caching, so that upgrades can be done without a reboot. This measure is experimental, although it will for sure improve mem caching issues on upgrades, there's no guarantee that it will eliminate all mem caching issues. In these cases a reboot will always be the best solution.
10.2.8 (PRO) BUG FIXES & IMPROVEMENTS:
• XSITools --check-repo=yes argument will only check the VMs just backed up, --check-repo=full will check the whole repository.

• Escape sequences filtered in log file when running job from the cronfile (this had been erroneously announced for previous version). Not 100% of escape sequences will be removed, we will continue to filter this remaining characters in future releases.

• Introducing --remote-xsipath argument to allow setting the remote xsi-dir path per backup job.

• Fixed issue in disk exclusions.
10.2.7 (PRO) BUG FIXES & IMPROVEMENTS:
• Added variable in conf/xsiopts to disable new version checking.

• :F option on --backup-point deletes the remote VM on --backup-prog=rsync|onediff|xsidiff. It does not make sense when combined with OneDiff, unless you use it only for the first copy.
10.2.6 (PRO) BUG FIXES & IMPROVEMENTS:
• Check for new versions in console and e-mail report.
10.2.5 (PRO) BUG FIXES & IMPROVEMENTS:
• Fixed bug in output redirection when using --host option.

• Added new version availability check in console output and e-mail report.
10.2.4 (PRO) BUG FIXES:
• Fixed bug when creating target directory on full backups.
10.2.3 (PRO) BUG FIXES:
• Fixed SEGFAULT bug when using an installation path shorter than 52 characters.
10.2.2 (PRO) BUG FIXES & IMPROVEMENTS:
• Fixed issue in paths containing spaces.

• Borg module has been updated to support latest releases (Borg 1.1.1), error reporting has been improved.
10.2.1 (PRO) BUG FIXES:
• Fixed bug in (c) XSIDiff, it would detect files as being dirs in some circumstances. XSIDiff has been upgraded to v 1.0.6.1.
10.2.0 (PRO) BUG FIXES & IMPROVEMENTS:
• conf/xsiopts file has been extended with the defremusr variable, which let's you stablish the remote user when connecting to remote systems.

• New argument introduced, --backup-as=user allows to override the defremusr hardcoded value above and set the remote user per backup job.

• Improved logic to backup to Linux systems vía Rsync or XSIDiff. Please, be aware that not any distro or kernel version will be compatible with XSIDiff binary and/ or provided Rsync.

• Trivial check adds the number of used space in megabytes to the first N megabytes check. Now a positive trivial check multiplies the possibilities of having two identical files on both ends, by checking: file size, used bytes in sparse file and first N megabytes checksum. This additional check works only with VMFS FS.

• SSH 1 returns as the default version of SSH to tunnel Rsync data through. SSH 2 has some features, like dynamic compression, that don't work well with Rsync. On addition, resource consumption grows wild when using SSH 2. This considerations are restricted to the environment XSIBackup runs in, namely ESXi Hypervisor.
10.1.5 (PRO) BUG FIXES:
• Fixed bug when running backup from server A in server B to server C. File comparison failed due to function comparing checksums not having access to unique id variable.
10.1.4 (PRO) BUG FIXES:
• Fixed bug preventing (c) XSIDiff to be used as a standalone backup program on its own, and not just as an Rsync helper.

• Fixed bug that would cause ambiguity in (c) OneDiff VM detection when some VM name is contained within another, i.e.: VM1, MyVM1, VM1-Win7, etc...
10.1.3 (PRO) BUG FIXES:
• Fixed bug that would cause .ERR-XXXXXXX... file to be deleted by the xsibackup-cron call causing errors not to be informed in the e-mail report.

• Fixed bug affecting win64 quiesced backups. This bug reappeared due to some changes in the code structure, it's now been finally fixed.
10.1.2 (PRO) IMPROVEMENTS:
• When preparing the VMs to be backed up, XSIBackup takes a snapshot. When it detects that VMWare Tools are installed, it tries to perform a quiesced snapshot to ensure all services finish writing data to disk before the snapshot is created. Since this version, XSIBackup will first try to take a quiesced snapshot; should the operation fail, will then try to perform a non-quiesced snapshot before giving up. Still an error will be raised for you to take action.
10.1.1 (PRO) BUG FIXES & IMPROVEMENTS:
• Improved PID management, now all cron pids are killed when a new backup session starts. Backups must be triggered sequentially via the --on-success & --on-error arguments.

• Improved --certify-backup hash computing when backing up over IP. Since this version, hash calculations are made in parallel, shortening times, when two servers are implied in the process.

• OneDiff, to local data store, backup snapshot copy is now handled by (c) XSIDiff.

• Fixed bug when backing up Windows x64 quiesced VMs by means of OneDiff to local data stores.
10.1.0 (PRO) BUG FIXES & IMPROVEMENTS:
• conf/xsiopts file has been extended with new tweakable variables, like trivialcheck=XX where you can set the number of megabytes that will be quick checked for each -flat.vmdk file after backup.

• Quick fix of bug in OneDiff management of absolute paths, which affected backup of VMs that have .vmdk disks spread across different datastores. Special thanks to Boris Pavlovic from Bosnia Herzegovina from pointing out the issue.

• Support for Borg 1.1.1, all Borg users should upgrade their Borg binary to 1.1.1 or later.

• Especific Windows algorithm created for vmkfstools and OneDiff programs. Some Windows versions, even EFI ones, create fragmented snapshot files which are hard to handle by backup programs. We created a separate logic to handle this type of OSs.

• Fixed bug in XSITools module that would prevent Python arithmetic calculations from working in some Python interpreter versions, throwing a backup error.
10.0.4 (PRO) BUG FIXES & IMPROVEMENTS:
• Fixed bug that prevented OneDiff to work in VMFS 6 formatted volumes.

• We have added the --run-backup argument, which will allow you to run some backup job stored in the xsibackup-cron, or any other alternative file. All lines will be taken into account, including commented out lines. In case some backupId is duplicated, the --run-backup argument will execute the first that it can find.

• Trivial check, after OneDiff cycle, has been raised to 500 megabytes (before was 50 M). It will only take some seconds, but will improve chances to detect files out of sync when not using the --certify-backup option

• xsidefaultpath is determined dynamically on every backup job, only if more than one, or none, remote installation point is detected, the hardcoded value at conf/xsiopts, will be used.

• conf/xsiopts file is introduced, to host hardcoded configuration values, so that you don't need to touch the program's file.

• Now SSH 2.0 is forced for every SSH connection. Compression is set dynamically by the SSH tunnel and :z modifier on OneDiff and Rsync is deprecated.

• Regular expressions are now accepted to select VMs when using the --backup-type=custom option. The regexp must be enclosed in parenthesys and preceded by the word REGEXP in upper case.
10.0.3 (PRO) BUG FIXES & IMPROVEMENTS:
• XSITools module has been reinforced to offer maximum resiliency: redundant checks are applied, each block copy is retried 10 times if an error is detected. In case the 10 retries fail, an additional copy loop is performed for all missing/ faulty blocks. In any case, it is extremely rare that a block cannot be copied, especially since the RAM footprint has been dramatically reduced in this version.

• (*) Improved speed when checking individual VMs backed up with XSITools.

• (*) Bug fixed that would cause block copy retries not to work when using compression with XSITools.

• Bug fixed in progress information when using XSITools individual VM integrity check. Check would exit before 100% and report one block less than total in check info.

• (**) XSITools memory usage optimization. From this version XSITools will use just a 1 mb. buffer, reducing available memory requirements to the minimum. Up until this version, ESXi Busybox dd utility would try to allocate a contiguous block of memory as big as the used block size (default is 50M) when backing up via XSITools. In some systems this amount of memory could not be allocated, this would depend on the level of fragmentation as well as the host configuration and load. By reducing memory requirements to a contiguous megabyte, the chances to receive memory outage errors are almost null.

(*) Special thanks to Bernhard Heidegger from www.hyperwave.comz for taking the time to detect some bugs and offer us qualified feedback.
(**) Thanks again to Bernhard for taking the time to perform empiric tests to determine how ESXi Busybox's dd uses memory and give us feedback.
10.0.2 (PRO) BUG FIXES:
• This is a quick fix version for users that run remote backups via the --host argument. Some essential functions were moved to a new file (src/functions), but this file was not being replicated to the remote xsi-dir.
10.0.1 (PRO) IMPROVEMENTS:
• The restore module now allows to choose the type of restore disk: thin, zeroedthick or eagerzeroedthick when using vmkfstools as the restore program. This is valid just for local regular backups, not for XSITools or Borg.

• We added the --options argument to parse some commands that might be needed in some circumstances.
10.0.0 (PRO) IMPROVEMENTS:
• First data transfer in Rsync and OneDiff backups is overtaken by (c) XSIDiff. It can handle -flat.vmdk files as native ESXi sparse files, making first transfer of whole file much more rapidly than Rsync.

• New argument --disable-vmotion allows to disable VMotion while the backup is taking place to avoid VMs being moved around by vCenter.

• New argument --shell-warning=[yes|no] allows to disable the annoying message that informs that the shell service is activated, run once.

• VM listing time previous to backup drastically reduced.

• OneDiff VMs are unregistered once data consolidation has completed to avoid duplicate MAC errors thrown by vCenter, even when the VM is off.
9.1.9 (PRO) BUG FIXES & IMPROVEMENTS:
• Implemented workaround for 'kex protocol error bug'. Newer curve25519-sha256@libssh.org kex algorithm is set at runtime for Rsync transfers when ESXi 6.5.0 is detected. KEX error, which is thrown as a warning, is trapped to avoid nasty uncontrolled messages and is printed out nicely. MD5 checksum is forced on every -flat.vmdk file pair.

• Now the SMTP panel file is available in the Free version, edit the conf/smtpsrvs file to add a virtually unlimited number of SMTP servers. They can be later used by invoking the --use-smtp=[Id] option, to shorten your backup jobs and make them more manegeable and meaningful.

• Added more control points to debug complex scenarios.
9.0.2 (FREE) BUG FIXES:
• Removed sed filtering of \r in xsibackup-cron-log file to prevent buffering of output in ESXi 6.5.0. This will allow to still be able to observe the log as it is written.
9.1.8 (PRO) BUG FIXES & IMPROVEMENTS:
• Fixed bug in XSITools module when deleting bad blocks in VM backup check,

• Improved speed in collecting of blocks previous to XSITools VM check.

Especial thanks to Bernhard Heidegger from hyperwave.com for pointing out bug and improvement.

• Fixed issue, spaces were not handled in paths when checking XSITools VM backup.

• Fixed issue, spaces were not handled in paths when using OneDiff to local storage if the VM already existed.
9.1.7 (PRO) BUG FIXES & IMPROVEMENTS:
• Improvement of xsibackup-cron processing speed, now lines without a --time argument are skipped.

• OneDiff logic error handling improved. The algorithm now tries to handle errors automatically and reinitializes the OneDiff cycle if necessary, up until this version manual intervention was required to delete the mirror VM if something was changed or moved. File size + first 50 mb. of .vmdk disk are checked. This first chunk will tipically contain the primary superblock, thus, any difference between both FSs (original & mirror) would cause a change in the first 50M block checksum.
9.1.6 (PRO) BUG FIXES & IMPROVEMENTS:
• If user didn't set the hostname, it will be set as "localhost", instead of empty string, so that the SMTP client is able to EHLO something.

• SSH-out port set to link to another server is managed automatically by the --link-srv command and added to the firewall set of rules if needed. This is specially useful with the ESXi 6.5.0 web client, which still needs some tuning and does not always work well.

• Log output is delayed in ESXi 6.5.0 due to sed buffering (new default behaviour). Filtering of \r lines is removed from XSIBackup temporarily until a fixed is added to next main revision.
9.1.5 (PRO) IMPROVEMENTS:
• We have moved time measurement to the start of the script to help reduce latencies a bit.

• Improved detection of blocked files.

• Fixed memory leak that could arise under some conditions and delay backups. Overall speed should improve.

• Improved detection of problematic OSs (Win x64 on MBR boot), now OS detection is made by regexp "^win.*64.*". The string is previously converted to lower case and then is matched to something starting by win and containing a 64 number somewhere else before the end of it. Mac OSX, coded as darwin is not mistaken for a Win OS, as the "win" substring is not at the beggining.
9.1.4 (PRO) BUG FIXES:
• This notice applies only to XSITools program. In extreme conditions, in which a host is short of memory, dd (used only by XSITools) can fail to copy disk slices with a "Cannot allocate memory" error. We have reinforced XSITools to detect such condition, raise a controlled error and retry the slice copy. Up until this version XSITools would not notice this spare bad/missing blocks until a repository check would be performed. Please, discard your previous XSITools backups and wait until 9.1.4 has been released to use XSITools in production.

• Added function to check VM hardware version against ESXi version. In case you are backing up a VM to an ESXi server with a version that cannot handle the VM hardware, a warning will be raised. The VM will be copied anyway, as the data store can host the VM files without problems, it is at the time to add the VM to the inventory when an unhandled error will be thrown and the VM will appear as "unknown" in the GUI inventory. I guess VMWare should address this situation and just output something like "This ESXi N.N.N host cannot handle HW version N".
9.1.3 (PRO) BUG FIXES:
• We have detected a late very seldom and random error that affects only to Windows Server system disks (there's a high correlation although the cause > effect relation is not clear). We have disabled XSITools for such systems until we release a fix or workaround. Sorry about the inconvenience. Please, do not use XSITools for production purposes until we release a solution.
9.1.2 (PRO) NEW FEATURES:
• Added posibility to check XSITools VM backup individually via the --check-xsitoolsrepo argument.

• Error Policy changed, now all errors, even those unrecoverable are informed, but execution continues to favour continuity of a job chain. Take on account that this changes XSIBackup's base working philosophy: before, some very gross errors were considered to need human intervention (like a wrong path) and backup would halt and ask you to fix it. Now, even in the case of very obvious errors, the main priority is to keep the backup chain rolling and try to make the backup by any means. In case of full storage, XSIBackup will look for folders that match the --date-dir mask (YYYYMMDDhhmmss) and will try to delete the eldest, until enough room is gained to make the backup. In case of fatal errors that prevent a backup job from completing, XSIBackup will try to raise a controlled error and jump to the next backup job in the chain off --on-error and --on-success events.
9.1.1 (PRO) NEW FEATURES & IMPROVEMENTS:
• Added --check-xsitoolsrepo argument to check XSITools repositories, in combination with --mail-to argument will e-mail a short report with the result.

• Improvements in some routines.
9.1.0 (PRO) NEW FEATURES:
• Now the block size is settable for XSITools backups, 10, 20 and 50 M are allowed.

• Added option --check-repo=yes to allow post backup consistency check.
9.1.0 (PRO) NEW FEATURES:
• Now the default behaviour when taking a snapshot is not to copy memory. From now on, if you want to have memory included, just set --snapshot=includememory

• New argument --check-xsitoolsrepo added. You can use this command to check the integrity of an XSITools repo.

• An SMTP server database file has been added as /conf/smtpsrvs. You can add your SMTP servers there and call them from your backup jobs like --use-smtp=2. Each SMTP server entry is identified by a unique integer. Servers can be checked with the command --check-smtp=test@email.com --use-smtp=1, this example command would send an e-mail to the address especified in the --check-smtp argument through the SMTP server number #1 set in the /conf/smtpsrvs file. You can parse all SMTP authentication data as before too for backward compatibility.

• E-mail submission code has been reorganized in a separate module in the /src/sendmail file. The name alludes to the well known MTA, but has nothing to do with it.

• Improved syncronization of files with linked hosts.
9.0.1 (FREE & PRO) NEW FEATURES:
• XSIBACKUP-FREE incorporates some of the improvements made to the XSIBACKUP-PRO core in the last moenths and some features not present until now.

• Fixed excessive proliferation of progress indication lines in the cron file. Especial thanks to Lothar Schilling for reminding the request and proposing an elegant and simple solution.
9.0.0 (PRO) NEW FEATURES:
• XSIBACKUP-PRO 9.0.0 introduces a new concept in VMWare backups: XSITools. It is a module with a set of functions that will allow you to store your VMs in deduplicated repositories directly in VMFS volumes, or on any other FS of your choice.
8.1.4 (PRO) IMPROVEMENTS & BUG FIXES:
• Code has been reorganized in modules that are sourced from the main file xsibackup. The code had grew and needed to be split in smaller chunks.

• If when installing/ removing the cron, a preexistent xsibackup-cron file is found, it will be renamed to xsibackup-cron.001.
8.1.3 (PRO) IMPROVEMENTS & BUG FIXES:
• Added --exclude-vms argument to exclude VMs from general --backup-type rules, like all or running.

• Some functions have been reprogrammed to be more consistent.
8.1.2 (PRO) IMPROVEMENTS:
• Added --override argument. By now it only accepts the string xsibakfilter, i.e.: --override=xsibakfilter. It removes the default filter for _XSIBAK terminated VMs generated by the OneDiff program, to allow to back them up under some special circumstances.
8.1.1 (PRO) BUG FIXES:
• Fixed some general bugs.
8.1.0 (PRO) IMPROVEMENTS:
• Added ISO datetime prefix to log lines. Not present in manual console backups.

• Warm & cold backups are now fully compatible with OneDiff backup program. Before this update, there were some limitations in the way some options could be combined, per instance "cold" and "warm" backups, that were thought to offer absolute compatibility with some less common use OSs, could not be used with the OneDiff program.
8.0.6 (PRO) BUG FIXES & IMPROVEMENTS:
• Added option to choose between nominal disk sizes or real data size in reports via the --show-sizes=real(default)|disk.

• Fixed relative path issue when trying to execute XSIBackup from a relative path instead of local or absolute, i.e.: ./xsi-dir/xsibackup. Paths above XSIBackup installation dir are not allowed, i.e.: ./../xsibackup
8.0.5 (PRO) BUG FIXES:
• Revision and improvement of ESXi config backup routine.
8.0.4 (PRO) BUG FIXES:
• Fixed bug, Borg backups would initialize repo in every backup cycle.

• Fixed bug in --type=all argument when doing onediff backups.
8.0.3 (PRO) BUG FIXES:
• Fixed bug in Borg client logic.

• Fixed bug in custom subject and other minor bugs.
8.0.2 (PRO) NEW FEATURES:
• Added support for FreeBSD 11, key linking, Rsync and Borg Backup.
8.0.1 (PRO) BUG FIXES:
• Small bugs fixed.
8.0.0 (PRO) NEW FEATURES:
• Added full restore module and backed up VMs browser.
7.3.0 (PRO) NEW FEATURES:
• XSIBACKUP-PRO now can backup over IP to any Linux system with a compatible rsync binary.

• Added possibility to parse option to --backup-prog argument to force use of xsibackup-rsync instead of remote system's rsync binary.

• Improved logic.
7.2.0 (PRO) NEW FEATURES:
• A new --backup-prog option has been added: borg. Borg backup is one of the fastest backup applications, featuring full block level de-duplication combined with compression and, optionally, encryption. The results in combination with XSIBackup are really impressive.

• Added possibility to parse option to --backup-prog argument to set compression when using rsync or borg. This may help over narrower connections (i.e.: --backup-prog=rsync:z).
7.1.1 (PRO) NEW FEATURES:
• --certify-backup=yes|md5|sha|sha1 argument added. By appending this command XSIBACKUP-PRO will compare checksums for every datafile backed up and will inform about the result.
7.0.5 (PRO) IMPROVEMENTS: • --snapshot=auto argument is substituted by a smarter default behaviour. If an MS Windows Server guest is detected under an MBR boot disk, it is not quiesced by default. To quiesce Windows Server guests, turn them into GTP bootable OSs. 7.0.4 (PRO) BUG FIXES:
• Some small bugs have been corrected.

• Added support for settable Rsync compression. This will only help in very special circumstances, thus the default recommended config is not to use compression at all.
7.0.3 (PRO) BUG FIXES:
• Warnings would cause the e-mail subject to show errors in backup. This was too alarming, as a simple missing .VMXF file (not needed in every case), would result in a backup job marked as bad. After correction, warnings are shown but don't force the e-mail subject as (X) bad.
7.0.2 (PRO) IMPROVEMENTS:
• Error handling logic has been rewritten.
7.0.1 (PRO) NEW FEATURES:
• Over IP transfers now allow --date-dir argument.
7.0.0 (PRO) NEW FEATURES & IMPROVEMENTS:
• OneDiff differential backup over IPv4.

• Output bash layout redesigned.

• More visual alerts in e-mail subject.
6.1.2 (PRO) IMPROVEMENTS:
• Resolvable hostnames are now allowed instead of IPv4 in --backup-point argument.

• Spaces are now allowed in datastores names.
6.1.1 (PRO) NEW FEATURES:
• New --backup-how option, apart from the preexisting ones (cold,hot), we added the warm option, it switches the VM down, takes a snapshot and switches the VM on again. It works with any Operating System, just like the cold option, even with no VMWare Tools or Open VM Tools available, but reduces the downtime to some seconds, or minutes at most.
6.0.11 (PRO) NEW FEATURES:
• The execprog directive has been added to the --on-success and --on-error arguments. It allows to run shell programs (binaries or scripts) with arguments upon backup completion.
6.0.10 (FREE & PRO) IMPROVEMENTS:
• Changed line breaks by inside HTML body. hMailServer requires linebreaks inside HTML content to pledge to the RFC standard as if it was plain text.

• Added MIME-Version: 1.0 to e-mail report header.
6.0.9 (FREE & PRO) NEW FEATURES:
• We have detected that sometimes ESXi 6.0.0 reports more datastores for a given VM than the datastores actually being used, when running:

vim-cmd vmsvc/get.datastores [VMID]
This is an ESXi bug and can confuse XSIBackup, making it not be able to find the .vmx file for the given VM. We have fixed this by restricting the .vmx file search to just the main datastore.

• We have added dontquiesce as a user option to the --snapshot argument, so that you can decide wether to quiesce snapshots or not.
6.0.8 (PRO) NEW FEATURES:
• XSIBACKUP-PRO will load additional code files in its own directory with the ".inc" extension.

• SMTP servers are probed prior to sending the e-mail report and set a failed status and error in case they are not reachable, instead of previous behaviour: skip and continue.

• Trapped errors for the open/close/reload firewall operations previous and subsequent to the e-mail submision, before any failure in performing these operations would result in one simple uncontrolled exception. It's rare that this errors would be thrown though.
6.0.7 (FREE & PRO) IMPROVEMENTS:
• xsibackup-pid turned into a multiprocess file. User and cron processes can coexist without having to manually delete the xsibackup-pid file.
6.0.6 (FREE & PRO) IMPROVEMENTS:
• Improved signal trapping and process management to make XSIBACKUP-PRO more stable.
6.0.5 (FREE & PRO) IMPROVEMENTS:
• Added argument --shutdown-wait to set the time a VM is granted to perform a clean shutdown before power off on cold backups.

• Subject text is not lowered any more.
6.0.4 (FREE & PRO) BUG FIXES:
• Improved disk exclusion matching, now it matches any base disk and any snapshot derived from the base disk.

• Improved backed up VM availability. Now any kind of backup is directly bootable from its backup location.
6.0.3 (PRO) BUG FIXES:
• Backup chaining by backupId issue fixed.

• Improved logic.
6.0.2 (FREE & PRO) IMPROVEMENTS:
• Hostnames can now be used as well as IPv4 addresses with the --link-srv argument.

• EXIT signal is trapped to ensure proper cleanup of the xsibackup-pid file and remaining processes.
6.0.1 (FREE & PRO) NEW FEATURES:
• VMDK paths are totally arranged at the backup destination folder to guarantee backups are runnable by: removing independent disks, arranging paths to point at the backed up .vmdk files.

• Code improvement to ensure maximum reliability, consistency and resilience.
6.0.0 (PRO) NEW FEATURES:
• OneDiff technology has been added as a new feature, real time diff backups.

• .nvram and .vmxf files are backed up with the rest of the files.

• VMs can be run in their backup location, all file paths are arranged for this.

• Now only --smtp-pwd and --backup-vms argument values are case sensitive.

• Color denoting VM state (on|off) lightened to differentiate from the backup state.
5.1.6 BUG FIX:
• Bug would prevent datastores matching a certain pattern from being used as backup origin and/or target. The bug has been fixed.
5.1.5 IMPROVEMENTS:
• Added trapping of signals to allow a cleaner shutdown when [Control+C] is pressend during manual execution.

• Added optional debug info to the e-mail report.

• More solid routines.
5.1.0 FIXES:
• When backing up a simple VM layout (all .vmdk files in a unique VM folder) over IP by means of Rsync. A reference to the backup snapshot is kept in the VMX file, preventing the VM from directly starting in the backup location. We have fixed this to avoid having to tweak the VMX file manually to switch the backed VM on.

• The error handling routines have been improved to show last error per VM and a global error list, containing all the errors at the end of the e-mail report. Finer error handling.
5.0.0 LIST OF NEW FEATURES:
• You can keep your pre-existing snapshots and have your VMs backed up from the last one. All information will be backed up, including data present in the last snapshot and all its chain down to the base disk. Read this post for more details.

• Compatibility with GMail. Now you can send e-mail reports through your gmail.com account. Thanks to Jeff Kaminski for pointing some clues.

• Now XSIBackup key pair is RSA to maintain compatibility with ESXi 6.0 update 2 and onwards.

• Improved core routines.
-----------------------------------------

4.7.3 Added colors in bash output to denote relevant information. Certain messages are highlighted to claim your attention. Blue means relevant plain info, green means something was completed O.K., red means something went wrong.

4.7.2 Fixed bug on same regexp again, some valid addresses were rejected. Added support to exclude memory from snapshots --snapshot=excludeMemory, this is useful when you want to hot backup a VM that contains an independent .vmdk disk, as excluding it may not be enough.

4.7.1 Fixed bug on regexp to match IPs in the --backup-point validation.

4.7.0 Added --on-error and --on-success arguments that allow transmitting the XSIBackup PID, argument list and errors to any HTTP URL for them to be proccessed or logged.

4.6.5 Multiple e-mail addresses accepted as a comma delimited list. Backups to same folder (no --date-dir) will be always overwritten to avoid vmkfstools error: "The file already exists (39)". Added colors to denote success, warnings and errors in a more visual friendly way.

4.6.4 SMART report is now optional, new option --smart-info has been added to set/unset this feature. Hole punching will only be done when the --backup-how flag has been set to "cold" (--backup-how=cold) to avoid a lock message being displayed causing some confusion.

4.6.3 Modifications made to Rsync to allow bigger buffer and block sizes in delta checksums. VMDK data file sizes are now compared previous to making an IP xsibackup-rsync copy on a .vmdk or -delta.vmdk virtual disk. If they use the same number of blocks, then MD5 checksum in both end files is performed. If the result of the MD5 hash matches there is no need to copy the .vmdk virtual disk. This avoids inner checksums when both files are identical. Rsync can't use size and timestamp because the latter is not preserved in VMFS and it can only know about apparent sizes and not real used blocks in thin disks. In any case MD5 is better than a simple, size plus timestamp comparison. Exception trapped and informed as a controlled error when backing up through IP and a .vmdk disk is missing. Some minor bug fixes have also been acomplished.

4.6.2 Added support to backup VMs located in datastore subfolders.

4.6.1 Fixed bug that would return incorrect VM size when using disk exclusions.

4.6.0 Guest system will be quiesced if VMWare Tools are detected to be installed in the guest OS. S.M.A.R.T. report for every disk in your ESXi system. A banner will appear in the e-mail report encouraging to make a donation to get full docs on how to use XSIBackup report as a marketing tool. NOTE!!: we have changed the way the file logo1.gif is downloaded in the e-mail report ?f=logo1.gif so that we can at least have a bare idea about the base of XSIBackup users. Of course you can get rid of it at any time.

4.5.0 Now XSIBackup uses its own DSA key pair instead of using the default. This allows persistance of this key pair across reboots and thus maintain links between servers avoiding to re issue --link-srv=[IP of linked server] after every reboot.

A PID file is created to allow follow up of the process state. Only one XSIBackup process is allowed at a time. If a second process is triggered manually the user will be given the option to delete the current PID file and launch a second process. Some small bugs have also been fixed.

4.4.0 Now it is possible to use vmkfstools or rsync for local backups by using the --backup-prog switch. Over the network transfers will only be possible with rsync. By combining the --backup-prog switch with the --date-dir argument you can mirror or build backup catalogs to a local datastore or over the network. A new argument --backup-room is also introduced that will allow to set the ammount of disk space that XSIBackup will use for its date subfolder backups. BUGFIX: added the -f flag to HOSTNAME=$(hostname -f) to get the fully qualified domain name under ESXi 6.0 (thanks to Jayce for reporting the bug).

4.3.1 Fixed bug, local rsync differential backups now allow spaces in paths.

4.3.0 Support for TLS e-mail communications. Added Rsync option for local backups between datastores that will allow to do differential mirroring of disks locally. Improved error reporting and minor bugs fixed.

4.2.5 Added support for multiple times in each cron line. You don't need to set a different line for each backup to be executed. You can separate different times with the format "Day hh:mm" by using the pipeline character (i.e. --time="Mon 10:30|Mon 21:30|Tue 11:31|Sat 03:21") This version is backward compatible, so you can keep your old multiple line cron file. Cron info is completely wiped now once you remove with --install-cron and added again if you run this command again and choose to reinstall.

4.2.4 Added full support for no athentication. Allows to use open SMTP relays inside a LAN. See the help section for details. Opening an open SMTP relay even inside a LAN requires you to be very confident you have a controlled environment. Use at your own risk.



Website Map
Consultancy
IT Manager
In Site
Resources & help
Index of Docs
33HOPS Forum

Fill in to download
The download link will be sent to your e-mail.
Name
Lastname
E-mail


            Read our Privacy Policy