Forum ©XSIBackup: ©VMWare ©ESXi Backup Software
MenuYou are not logged in.
- Topics: Active | Unanswered
#1 Re: General matters » How to delete all XSITool backups of a VM ? » 2019-06-14 11:11:20
Well, as said, I need to free up some space, so open a new set is not an option, since there is not enough space 
Conclusion:
I cannot run any further backups unless I delete all (instead just the ones I don't need).
Feature request: Pruning by VM name from repository or an option to cleanup a repository by search and delete orphaned blocks.
#2 Re: General matters » How to delete all XSITool backups of a VM ? » 2019-06-13 13:52:38
That's a pity. So orphaned blocks will never be removed? Isn't there a script for that at least?
#3 General matters » How to delete all XSITool backups of a VM ? » 2019-06-08 12:01:41
- NextLevel
- Replies: 6
I need to free up some space on the backup target repository. It is already limited with xsibackup-prune to 6Gb.
There are backups of certain VMs that are not needed any more. How can I remove all backups of a VM (including the blocks) from the repository?
Greetings
NextLevel
#4 Re: General matters » How to exclude VM's rather then include » 2019-04-03 14:36:54
Just say how to implement the necessary -v option of grep into the --backup-vms=REGEXP 
Because something like --backup-vms="REGEXP('^(?!.*temp).*$')"
simply does not work
#5 Re: General matters » No need to prune the repo with 3 Petabyte :) » 2019-04-03 14:31:21
If you pass 3 072 000 GB (3 298 534 883 328 000 bytes), you are playing russian roulette with the integer limits supported by the system, as some inner calculations use bytes as the base unit. That does not mean that it won't succeed though.
The log reads 3089855Mb = ~3Tb. Emmm, you don't tell me, that I can't use a target/repository with more than 3TB (3145728 MB) ? Do you?
How can I send you a log file (it is a bit to much to post here)?
#6 Re: General matters » How to exclude VM's rather then include » 2019-04-03 14:09:24
How to exclude .*temp test ^clone.* ?
#7 Re: General matters » How to exclude VM's rather then include » 2019-04-03 14:02:38
I've tried, but as you say you can't use it in combination with --backup-type=Running, so I had no choice
But even if I ignore the power state, I do not have a clue what a REGEXP () pattern should look like to achieve the same exclusions as described.
Any suggestion is welcome
#8 Re: General matters » How to exclude VM's rather then include » 2019-04-03 01:06:00
Since there is yet no built-in way to backup all running VMs while excluding some, I made this script:
OLDIFS="$IFS"
IFS=$'\n'
VMList=$(vim-cmd vmsvc/getallvms | sed '1d' | awk '{if ($1 > 0) print $1" "$2}')
for VM in $VMList; do
VMName=$(echo $VM | awk '{print $2}')
VMID=$(echo $VM | awk '{print $1}')
VMMatch=false
for pattern in $@; do
if (echo $VMName | grep -q $pattern)
then VMMatch=true; break
fi
done
if $VMMatch; then continue; fi
if (vim-cmd vmsvc/power.getstate $VMID | grep -q "Powered on")
then VMsToBackup="$VMsToBackup,$VMName"
fi
done
IFS=$OLDIFS
echo ${VMsToBackup#?}Save it with the name AllRunningButNot then you can use it in the --backup-vms option with one or more exclude patterns as parameter:
--backup-vms=$(./{path_where_you_store_it}/AllRunningButNot .*temp test ^clone.*)In the example above, all running VMs are selected, unless the name ends with "temp", contains the string "test" or starts with "clone"
I only use underscore, hyphen and period as special characters in my VM names. I don't know if it works with others.
The bad thing of this workaround is, you cannot use the PROs GUI to manage the job.
#9 General matters » Select to backup running VM by REGEXP » 2019-03-31 20:50:07
- NextLevel
- Replies: 1
I try to backup all running VMs that fits the REGEXP(^V)
But
--backup-type=Running \
--backup-vms="REGEXP(^V) \
Ignores the REGEXP finally.
Not possible?
#10 Re: General matters » No need to prune the repo with 3 Petabyte :) » 2019-03-31 19:55:40
Now I'm completely lost. This happens after backup up the first VM of a set, and the job stops
---------------------------------------------------------------------------------------------------------------------------------
2019-03-31T19:43:15| Checking the size of the (c)XSITools repository...
---------------------------------------------------------------------------------------------------------------------------------
2019-03-31T19:45:00| The (c)XSITools repository is 3089855Mb in size
---------------------------------------------------------------------------------------------------------------------------------
2019-03-31T19:45:00| Prune round 1: (c)XSITools repository size: 3089855 exceeds --backup-room=3072000
---------------------------------------------------------------------------------------------------------------------------------
-- Pruning exclusive blocks of backup </vmfs/volumes/vSphere_Backups/XSI/20190318222640>
-- Nothing to prune: there's probably just one backup in the repoI think I'll better start a new repository :S
#11 Re: General matters » No need to prune the repo with 3 Petabyte :) » 2019-03-30 17:29:54
No I have just the latest version and never installed any other.
But, within the SAME job it is sometimes right and one VM later it is wrong..
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:14:11| Checking the size of the (c)XSITools repository...
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:16:05| The (c)XSITools repository is 3083810Mb in size
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:16:05| There's still 1012190 Mb available, no need to prune the repo
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:16:22| [VM-001-TS01] Starting backup (size is 28672M on 28672M file)
---------------------------------------------------------------------------------------------------------------------------------
....
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:26:09| Checking the size of the (c)XSITools repository...
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:27:38| The (c)XSITools repository is 3119474Mb in size
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:27:39| There's still 4191184526 Mb available, no need to prune the repo
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:27:55| [VM-001-DC01] Starting backup (size is 25421M on 26624M file)
---------------------------------------------------------------------------------------------------------------------------------I don't understand WHEN a check is triggered at all, because there are 11 VM this job includes, but only 2 checks. Should it check once or after each VM?
#12 Re: General matters » --del-dirs never deletes » 2019-03-30 17:09:05
Does that mean with XSITools I cannot automatically clean up the repository distinguish by dates and VM?
i.e keeping the last 7 days of backups and related blocks of VM-One.
and keeping the last 30 days of backups and related blocks of VM-Two.
Do I need to make a repo per VM to solve that?
--prune-xsitoolsrepo ; This is a standalone option, not a part of a backup job, right? Since it cannot be used as a part of the backup job, it would not help.
--backup-room ; This does always checks and prune against the whole repo no matter which VMs the backup job includes, right?
So a backup of a VM can be deleted even it is the last one that exists of it?
Also I cannot run --backup-room option as a standalone, it must be combined with some (and successful) backup of a VM..., or not?
Confused ...
#13 General matters » --del-dirs never deletes » 2019-03-30 01:48:00
- NextLevel
- Replies: 2
No matter what I define on the --del-dirs option, it never deletes something
./xsibackup --del-dirs=+1d --backup-point=/vmfs/volumes/vSphere_Backups/XSI --backup-type=custom --backup-vms="TestW2k3" --backup-prog=xsitools:zI always get: info: no directories to delete as per the --del-dirs argument
Backing up via XSITools
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
File name: TestW2k3-flat.vmdk, File size: 4294967296
Block size: 52428800, Block count: 82
Disk usage: 4294967296
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
82/82 blocks | Processed 101%
Previous block count: 78838
Current block count: 78838
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Time taken: 22 seconds
Avg speed: 186 mb/s
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[TestW2k3] info: no directories to delete as per the --del-dirs argument
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The ESXi configuration was saved to "/vmfs/volumes/vSphere_Backups/XSI/20190330011240"
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
No errors detected in backup
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Backup finished
Tip: no chained backups scheduled, set --on-success and/or --on-error arguments to chain a backup
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
KilledBut there are olders to delete:
[root@ESX01:/vmfs/volumes/2542dc62-3824324c/xsi-dir] find /vmfs/volumes/vSphere_Backups/XSI/20* -type d -name "TestW2k3"
/vmfs/volumes/vSphere_Backups/XSI/20190319135506/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190319145209/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190319153811/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190319162314/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190319200006/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190320200006/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190321200006/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190322200006/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190323003352/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190330001834/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190330002911/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190330005724/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190330010257/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190330011058/TestW2k3
/vmfs/volumes/vSphere_Backups/XSI/20190330011240/TestW2k3Also the datedirmask=20[1-3][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]
is still original
Where is my fault?
#14 Re: General matters » No need to prune the repo with 3 Petabyte :) » 2019-03-29 20:42:08
Today it is even 4 Petabyte .
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:26:09| Checking the size of the (c)XSITools repository...
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:27:38| The (c)XSITools repository is 3119474Mb in size
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T20:27:39| There's still 4191184526 Mb available, no need to prune the repo
---------------------------------------------------------------------------------------------------------------------------------
The only thing I have changed is --backup-room=4000 (3000 before). So there seems to be a relation.
#15 Re: General matters » No need to prune the repo with 3 Petabyte :) » 2019-03-29 19:46:44
"/vmfs/volumes/vSphere_Backups/xsi-dir/xsibackup" \
--backup-prog=XSITools:z \
--backup-point=/vmfs/volumes/vSphere_Backups/XSI \
--backup-type=Custom \
--backup-how=Hot \
--snapshot=includememory,doquiesce \
--use-smtp=1 \
--mail-to=mymail@nowhere.com \
--backup-id=902 \
--description="Run" \
--del-dirs=+30d \
--backup-room=3000 \
--backup-vms="VM-009-DBS" \
--exec=yes >> "/vmfs/volumes/vSphere_Backups/xsi-dir/var/logs/xsibackup.log"
Target is a NFS datastore on a NAS with 10 Terabyte
#16 General matters » No need to prune the repo with 3 Petabyte :) » 2019-03-29 00:49:38
- NextLevel
- Replies: 9
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T00:31:17| Checking the size of the (c)XSITools repository...
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T00:32:33| The (c)XSITools repository is 2921587 Mb in size
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T00:32:33| There's still 3142806413 Mb available, no need to prune the repo
---------------------------------------------------------------------------------------------------------------------------------
2019-03-29T00:32:48| [VM-009-DBS] Starting backup (size is 131020M on 139264M file)
---------------------------------------------------------------------------------------------------------------------------------
Emmm, I'm pretty sure I do not have 3 petabytes of storage ... must be some bug.
#17 Re: General matters » How to exclude VM's rather then include » 2019-03-27 22:56:58
Sure: Why easy, if complicated. 
RegExp always will often fry your brain ...
Just putting a pattern as REGEXP() in the --backup-vms option ... OK. But you can not define omission in this way. Isn't it?
But "greping" around... with some command constructions where you can do more wrong than right?
Well, we have to live with it. Anyway, please put it on the feature request list.
#18 Re: General matters » How to exclude VM's rather then include » 2019-03-26 21:18:09
Still no VM exclude option on the road map ? Like "All running, but not "*_temp" ? Would still be really helpful ...
#19 Re: General matters » A few questions about XSIBackup ... » 2019-03-20 12:27:39
Thank you for the answers
Q: what is the difference of --check-repo and -check-xsitoolsrepo?
I'm aware of the fast and full option of --check-xsitoolsrepo. The question is, what is the difference to --check-repo ?
A: no, always use 000-999, we'll fix that in the GUI
Oh, I already use job ids like BRXTC because this allows me to set meaningful, brain compatible, abbreviations. Makes it much easier to define chains. And it seem to work w/o problems. Hope you don't change that .
#20 General matters » A few questions about XSIBackup ... » 2019-03-19 16:28:17
- NextLevel
- Replies: 2
While evaluate the fantastic XSIBackup, a few questions came up, that are not covered fully in the man page and docs (or not found).
-----------------------------------------------
Q: Supress Proceeding lines in log
While logging a backup job, progress is written for each block. If you watch (tail) the log this is nice to have, but if you just locking at logs later, you just got thousands of these lines. Is there an option to stop this verbosity?
A: not currently, we will include this as a feature request
-----------------------------------------------
Q: The man pages reads: only 3 digit allowed for job-id.
But the GUI allows up to 5 chars. So is it save to use an Id like DAILY ?
A: no, always use 000-999, we'll fix that in the GUI
-----------------------------------------------
Q: what is the difference of --check-repo and -check-xsitoolsrepo?
The man page is not entirely clear: --check-repo is used in a backup while -check-xsitoolsrepo is a standalone call? And is job --check-repo=full the same as -check-xsitoolsrepo?
A: none, there are two kinds of check: full | fast.
Fast just checks the block existence and size. Full will recalculate the hash for the block's data
Default check is full.
-----------------------------------------------
Q: does a error found in a standalone repository check trigger the --on-error event?
If you run a --check-xsitoolsrepo, does the on-error event fire if errors are found in the repo?
A: no, check of XSITools repositories are performed out of the scope of the backup job
-----------------------------------------------
Q: whenever I want use the numpad to enter numbers, the GUI quits. Why?
While the numpad works fine in the shell, within the GUI it kick me out immediately (or acts as F1) when pressing a numpad key.
Can this behavior be changed?
A: we'll study this.
The GUI is based on dialog, the nCurses Linux binary (untouched), so we might not have control on this
-----------------------------------------------
Q: the GUI complains that I have to define both --on-error AND --on-success.
If I left on empty: Error: you must set both event handlers.
What to enter if there is nothing to do on success? Do I have to create a dummy job?
A: yes, you must define both.
Point it to an empty job, i.e.: 000?
-----------------------------------------------
Q: how to gracefully end a continuous backup
I can create a continuous backup just be using --on--success to recall the same job again and again.
But how do I stop this job gracefully (i.e. end the continuous recall after 19.00h). I can schedule a dummy job at this time, but this will kill the running process in the middle of it (and may leave an inconsistent repository).
Any idea how I can just stop the recall after 19.00h? Can I maybe force the running job to run into on-error?
UPDATE: Sometimes the solution is to simple to see . Simply rename the job file does the job. The current job keeps running to the end and stops the chain because the job file is gone. It does not even throw an error. Fits for me.
A: you must use some scripting to control that.
You can create a custom backup job that uses a counter recorded in a file or that behaves depending on the system date
-----------------------------------------------
Q: can I suppress the creation of ESX backup file
Every job creates a *-config-tgz file. Can this be suppressed?
A: no, those are included in the backup by default.
They weight nothing and the ESXi system backup just takes a couple of seconds
Thank you for your product and support.
#21 General matters » Cronjob only executes the --on-error part of the job » 2019-03-18 22:55:31
- NextLevel
- Replies: 0
I have just one single job as follows:
"/vmfs/volumes/vSphere_Backups/xsi-dir/xsibackup" \
--backup-prog=XSITools:z \
--snapshot=doquiesce \
--backup-point=/vmfs/volumes/vSphere_Backups/XSI \
--backup-type=Running \
--backup-how=Hot \
--use-smtp=2 \
--mail-to=****@****.com \
--backup-id=010 \
--description="All running VMs" \
--del-dirs=+30d \
--img-list="none|none|none|none|none|none" \
--on-error="execProg->/vmfs/volumes/vSphere_Backups/xsi-dir/xsibackup --check-xsitoolsrepo=/vmfs/volumes/vSphere_Backups/XSI:f --mail-to=mymail@mail.com" \
--exec=yes >> "/vmfs/volumes/vSphere_Backups/xsi-dir/xsibackup-$(hostname -s).log"If I start the job manually (Using the GUI Running a backup job) everything works fine.
I put this job in the crontab:
01 22 * * 1-5 "/vmfs/volumes/vSphere_Backups/xsi-dir/jobs/010"It is also correcly copied to the esxi crontab in /var/spool/cron/crontabs
But when it runs from the scheduler it just executes the --on-error part, while there is nothing in the log that indicates an error:
---------------------------------------------------------------------------------------------------------------------------------
2019-03-18T22:01:05| ###############################################################################
2019-03-18T22:01:05| XSIBACKUP-PRO 11.2.3: new execution request
2019-03-18T22:01:05| ###############################################################################
2019-03-18T22:01:05|
Performing fast check on (c)XSITools repo
---------------------------------------------------------------------------------------------------------------------------------
2019-03-18T22:01:05| </vmfs/volumes/vSphere_Backups/XSI> is an (c)XSITools repo
---------------------------------------------------------------------------------------------------------------------------------
Checking XSITools Repo:
---------------------------------------------------------------------------------------------------------------------------------
Physical count 20246 equals 20246 registered blocks at .xsitools file
1/20246 blocks | Processed 0%.
... thousands of processed lines
20246/20246 blocks | Processed 100%.[0K
2019-03-18T22:03:26| No errors detected
---------------------------------------------------------------------------------------------------------------------------------
Using stored SMTP server info...
Found conf/smtpsrvs file...
Using SMTP server 1
Open firewall: 2019-03-18T22:03:27| Opening port 25 for SMTPout-25 service...
...
... the smptp stufff
...
354 Enter mail, end with <CRLF>.<CRLF>
250 Ok, message saved <Message-ID: >
221 See ya in cyberspace
2019-03-18T22:03:35| Firewall rule SMTPout-25 closed.What can I do?
BTW: Is it possible to suppress these thousands of processed lines in the log files?
Update: After adding both --on-error and --on-success, it works now as expected.
#22 Re: General matters » Can I start all backups on all hosts at same time? » 2019-03-16 18:01:07
BTW: your website is using some flash snippets that are quite outdated nowadays...
ROTFL
There's life in the old dog yet.
#23 Re: General matters » OneDiff ends in a registering loop » 2019-03-16 17:58:55
Since ESXi caches as much data as it can in RAM, real usage figures for real situations don't differ that much, in fact, most of the times don't differ at all.
Yes, you are right, except for a Remote Desktop Server. It "feels" the 4k32Q breakdown extremely . When a snapshot is activated, the users begins to complain why is it so slow today....
However, I like XSIBackup more every day.
#24 Re: General matters » OneDiff ends in a registering loop » 2019-03-15 21:27:28
We have never seen this behaviour, but given the fact that you seem to be using a Windows 2003 VM
Do not blame my 2003 machine. It's still the fastest of them all
Maybe your test was too short, we don't get the same results
Not to make me guilty for superficiality, I repeat all the tests again with a freshly installed ESX Server 6.0. With 2012 server, with 2008R2 server, with 2003 server.
All run on an NVME datastore with the latest VMtools.
The result is still the same.
If you say that you do not get the same results, PLEASE enlighten me: What have I missed in the last 10 years?
I would be overjoyed to use OneDiff if that performance loss of snapshots were not so big. Really, I mean it
Because the registering loop does not happen if I use the --options=unreg-xsibak.
#25 Re: General matters » Can I start all backups on all hosts at same time? » 2019-03-15 16:29:35
Now you have the drawback to go through all the backups to find your VM.
IMHO this is the more serious drawback. But other opinions are of course respected ...