When talking about a DataCenter we are not talking about a single Server running with two or three VMs.
We are talking about several servers with dozens or maybe hundreds of VMs.
One Problem in a DataCenter is managing all these servers and in case of XSIbackup: managing all these backups.
XSIbackup is sending you an email and you can see in the emails subject if the backup was successful or failed.
But as soon as you need some more statistics (e.g. how many data was backuped last weekend) its becoming difficult.
I started to create an Excel sheet and entered some data:
Date Weekday Time Session ID Server Esxi Version XSI Version License Method VM-Name Folder (Destination) Size [GB] Diff [GB] State Duration Speed [MB/s] Compress Error 10.06.2021 4 22:50:03 1111111 Server4 6.0.0 Update 2 (Build 3620759) DC 18.104.22.168 replica (CBT) WinADS001 /vmfs/volumes/nas1/backup1 380 9,39 1 00:04:47 1355,82 1 0 10.06.2021 4 23:00:34 1111112 Server5 6.0.0 Update 2 (Build 3620759) DC 22.214.171.124 replica (CBT) WinADS002 /vmfs/volumes/nas2/backup2 480 2,98 1 00:00:21 23406,14 1 0 10.06.2021 4 23:02:07 1111113 Server4 6.0.0 Update 2 (Build 3620759) DC 126.96.36.199 replica (CBT) AppServer1 /vmfs/volumes/nas1/backup1 80 79,5 1 00:31:58 42,75 1 0 11.06.2021 5 01:37:07 1111114 Server9 7.0.0 (Build 16324942) DC 188.8.131.52 replica (CBT) AppServer2 /vmfs/volumes/nas1/backup1 40 39,38 1 00:06:59 97,76 1 0 11.06.2021 5 00:01:24 1111115 Server8 6.0.0 Update 2 (Build 3620759) DC 184.108.40.206 backup Docker001 /vmfs/volumes/nas1/backup1 80 4,39 1 00:01:16 1078,13 1 0
But thats a time-consuming work and therefore I would like to raise the following proposal:
- Send the backup infos to an central logging server
A DataCenter has usually one central logging server who collects e.g. syslogs from router and switches
or event logs from windows servers or application logs from web servers.
Some of these tools used are:
- Logstash, Elasticseach and Kibana
(see https://www.tecmint.com/open-source-cen … nt-tools/)
Integrating XSIBackup in this logging infrastructure is no big deal:
- create a log string
(either using CSV or tab delimited data like the columns in my excel sheet, or a JSON string)
- send it to through VMWare's syslog
- or send it directly to the log server using either TCP or UDP to a specified port
Historically (full image) backups where needed to recover from hardware failures like a disk crash or fire in the server room.
But today it has an additional purpose:
- restore the data after an attack from a crypto trojaner (virus which encrypts all your data)
Using the technology devloped by 33hops (deduplicating data with an SHA-1 checksum algorithm) it is possible to detect such virus activity.
Usually only a small portion of data is changed in a virtual machine.
E.g. our users share is 500 GB and only 1-2 GByte are usually changing over a day.
As soon as the virus is encrypting the data you will see a sharp rise in the amount of differential data backuped by XSI in the night.
( and no, you can't detect this from a normal file backup running in the virtual machine, as the virus will not change the size of the files or change the archive bit or the last modification time )
So, all you have to do is to monitor the Diff size of the backup (see above excel sheet) (or the relation from Total to Diff size)
( and a central logging system will help you to do this job .. )
PS: I'm not claiming that this is my idea. Some other vendors of backup solutions are already using KI technology to detect such abnormal behavior and raise alarms)
What you say does make sense. You can actually add the information generated during a backup cycle to let's say some MySQL DB easily, you just have to exchange key with the DB server and run a postbackup script to add this info to the database.
We will dedicate some time to writing some posts and offer some examples that can be reused by (c)XSIBackup users and also some DB triggers to generate alerts on the situation that you describe.
We are actually finishing to test some new features around --rotate to allow setting virtual user space limits and improving the rotation features.