©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#1 2020-06-15 13:17:40

mbutsko
Member
Registered: 2020-05-16
Posts: 8

Can two XSIbackup instances use the same repository simultaneously?

May be a silly question.

Is it safe to have two hosts running backups at the same time and point them to the same repository but different --subfolder?

It had just occurred to me that there could be some data conflicts if one XSI is not aware of the other, or am I wrong?

Thank you!

Offline

#2 2020-06-15 13:42:23

admin
Administrator
Registered: 2017-04-21
Posts: 2,057

Re: Can two XSIbackup instances use the same repository simultaneously?

Of course it's not a silly question. It is probably the most interesting question to place about (c)XSIBackup-DC, as concurrency allows to use it at a datacenter level in an extremely efficient way. That's what its aim is.

Yes, concurrency is managed through .lock files. Nonetheless, even if there was no lock mechanism, the possibility that the same unique block is being written at the same time from two or more hosts is negligible.

Still obviously some kind of locking mechanism is required. In the future it will be superseeded by semaphores to allow many hosts writting at the same time while still optimizing lock times.

The biggest concern is the .blocklog file, the main manifest where unique blocks are registered, but that may be rebuilt at any time by a --repair argument. Thus, just as long as your local manifests (inside every VM folder) and your blocks are O.K., you can always rebuild the repository.

Offline

#3 2020-06-15 13:50:12

mbutsko
Member
Registered: 2020-05-16
Posts: 8

Re: Can two XSIbackup instances use the same repository simultaneously?

Ah, I do remember reading about the lock files and rebuilding via the --repair argument now that you mention it! Cool, thank you for the fast information as always.

Offline

#4 2021-05-22 18:44:50

kd.gundermann
Member
Registered: 2018-02-27
Posts: 34

Re: Can two XSIbackup instances use the same repository simultaneously?

I have tried to use the same repository (destination) for a replica job with XSIBackup-DC,
but it fails as there is already running a replica job on another host:

[root@Server2:/vmfs/volumes/9b02496c-d7ee603d/Server2/xsibackup-dc] /xsi-dir/xsibackup \
> --replica=cbt \
> "VMs(SQL2012)" \
> "/vmfs/volumes/xsibackup01/SNAPSHOT/" \
> --subfolder="SQL2012" \
> --description="SQL2012"
|---------------------------------------------------------------------------------|
||-------------------------------------------------------------------------------||
|||   (c)XSIBackup-DC 1.5.0.3: Backup & Replication Software                    |||
|||   (c)33HOPS, Sistemas de Informacion y Redes, S.L. | All Rights Reserved    |||
||-------------------------------------------------------------------------------||
|---------------------------------------------------------------------------------|
                   (c)Daniel J. Garcia Fidalgo | info@33hops.com
|---------------------------------------------------------------------------------|
System Information: ESXi, Kernel 7 Major 0 Minor 0 Patch 0
-------------------------------------------------------------------------------------------------------------
License: 00050654000000000000000008f1ea767ab0 | (c)XSIBackup-DC
-------------------------------------------------------------------------------------------------------------
PID: 7982678, Running job as: root
-------------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC replicating data to /vmfs/volumes/xsibackup01/SNAPSHOT
-------------------------------------------------------------------------------------------------------------
Performing --replica action
-------------------------------------------------------------------------------------------------------------
Item number 1 in this job
-------------------------------------------------------------------------------------------------------------
SQL2012 Hardware Version is: 11
-------------------------------------------------------------------------------------------------------------
All snapshots were removed, as SQL2012 is engaged in a CBT job
-------------------------------------------------------------------------------------------------------------
Virtual Machine Name: SQL2012
-------------------------------------------------------------------------------------------------------------
(c)ESXi 7.0 or higher detected, all snapshots were removed for VM SQL2012(4)
-------------------------------------------------------------------------------------------------------------
Creating snapshot VM : SQL2012 (powered on)
-------------------------------------------------------------------------------------------------------------
*** Snapshot was successfully created ***
-------------------------------------------------------------------------------------------------------------
*** Remote directory is locked ***
-------------------------------------------------------------------------------------------------------------
The remote repository or replication directory is locked
This usually means that there's some process writing data
to it at this moment, or that some process was interrupted

Check for a file named .locked in the root of it and if
you believe it has been left there by some unfinished process,
you may delete it manually to allow new operations
-------------------------------------------------------------------------------------------------------------
2021-05-22T18:38:16 | Error code 3407 at file xsibackup.c, line 3407 | Error description: failed to lock the repo /vmfs/volumes/cacffd7b-11f7dacc/SNAPSHOT
-------------------------------------------------------------------------------------------------------------
*** Snapshot was removed ***
-------------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Removed prog <tmp> dir        OK
-------------------------------------------------------------------------------------------------------------
Unlocked backup               OK
-------------------------------------------------------------------------------------------------------------
[root@Server2:/vmfs/volumes/9b02496c-d7ee603d/Server2/xsibackup-dc]

The message states

Check for a file named .locked in the root of it

but no, there is no .locked file

Offline

#5 2021-05-23 11:04:38

admin
Administrator
Registered: 2017-04-21
Posts: 2,057

Re: Can two XSIbackup instances use the same repository simultaneously?

The lock file is deleted once the process holding the repo finishes. Nonetheless there's something really 'twisted' in your question: how come you be trying to perform a simultaneous replica to a replica folder?, that makes no sense no matter how you look at it.

If you were allowed to do such thing, you would end up with a lot of meaningless data in the worst of the cases, or a mix of files belonging to different VMs if you were somewhat luckier.

A replica job is a 1:1 operation, by definition, it's even implied in the very same argument name "replica" you can't replicate two different things to a third one.

Backup repositories are a different thing. Please, revise basic functional concepts before attempting to use the software.

Offline

#6 2021-05-25 11:47:02

kd.gundermann
Member
Registered: 2018-02-27
Posts: 34

Re: Can two XSIbackup instances use the same repository simultaneously?

admin wrote:

how come you be trying to perform a simultaneous replica to a replica folder?, that makes no sense no matter how you look at it.

I have several virtual machines and I think I should be able to backup or replicate these machines to a backup-destination in different SUBFOLDERS.
E.g. 

Vm1  ->  /backup-dest/snapshot/Vm1
Vm2  ->  /backup-dest/snapshot/Vm2
...

I have tested it again and now it was running fine in parallel.

I am sorry, but maybe there was some leftover data in my first test..

Offline

#7 2021-05-25 15:50:35

admin
Administrator
Registered: 2017-04-21
Posts: 2,057

Re: Can two XSIbackup instances use the same repository simultaneously?

We understood you were replicating to the same folder, as you were complaining about the lock file.

You can of course replicate to different folders, nonetheless there are some concerns regarding parallelizing streams of data that you should take into account. You will anyway end up finding it for yourself in an empirical way.

It will be much faster copying two VMs sequentially than doing it at the same time. There is a big consideration to make in respect to that when using spinning classic HDs, as there is a physical arm that has to move around to write data. If you use SSDs or nVME disks, you will work that limitation around up to some degree.

Still SCSI/SATA controllers and network equipment (if implied) will pose some limits to parallelizing streams of data and will still make serial faster than parallel most of the times.

And last but not least, replicating VMs can be a CPU intensive task. It's not just reading and writing data, (c)XSIBackup needs to hash it, compress it and look for it in hash tables in the order of millions of records. Parallelizing can take advantage of multicore systems, still you need to take that on account, as you will have to leave some CPU cycles to the rest of the system and guests.

If you want to experiment, it's O.K, share your experience here, in any case, our recommendation for a production system is that you just let (c)XSIBackup treat VMs in a row by passing all of them to the VMs() argument.

And lastly: of course, if you use --replica=cbt Changed Block Tracking feature, you will reduce data streams to the minimum, you will be parallelizing CPU usage mainly, which is OK just as long as it's OK with your server.

Use the --options=R argument to create a test VM on the other end and check your data. You can leave that VM there, even if it's turned on, the CBT feature will take care to switch it off and re-register it on each CBT cycle.

Offline

#8 2021-12-18 14:28:34

kd.gundermann
Member
Registered: 2018-02-27
Posts: 34

Re: Can two XSIbackup instances use the same repository simultaneously?

I'm getting the same error message:

2021-12-18T14:21:50 | Error code 3683 at file xsibackup.c, line 3683 | Error description: failed to lock the repo /vmfs/volumes/66633834-7ea5a8ef/SNAPSHOT

When having a backup job of:

/xsi-dir/xsibackup \
--replica=cbt \
"VMs(SQL2012)" \
"/vmfs/volumes/xsibackup02/SNAPSHOT" \
--description="Replica SQL2012" \
--mail-to="administrator@xxxxxxx.de" \
--use-smtp="1" \
--subject="Replica SQL2012" \
--subfolder="SQL2012" \
>> /xsi-dir/var/log/xsibackup.log 2>&1

where is xsibackup placing the .lock file ?

A:  /vmfs/volumes/66633834-7ea5a8ef/SNAPSHOT/
B:  /vmfs/volumes/66633834-7ea5a8ef/SNAPSHOT/SQL2012/

Offline

#9 2021-12-18 15:04:36

admin
Administrator
Registered: 2017-04-21
Posts: 2,057

Re: Can two XSIbackup instances use the same repository simultaneously?

The lock file is placed at B. Nonetheless you can easily check that.

Locking the repo consist in creating a file via the fopen() => open system call. When you receive that error you are getting a direct response to a system call from the file system itself, thus, there is some clear reason for that to happen: full FS, inappropiate backup volume (you can't use VMFS as a target), failing hardware, etc...

You can get real time system call level information by prepending strace to your job, i.e.:

strace /xsi-dir/xsibackup \
--replica=cbt \
"VMs(SQL2012)" \
"/vmfs/volumes/xsibackup02/SNAPSHOT" \
--description="Replica SQL2012" \
--mail-to="administrator@xxxxxxx.de" \
--use-smtp="1" \
--subject="Replica SQL2012" \
--subfolder="SQL2012" \
>> /xsi-dir/var/log/xsibackup.log 2>&1

Offline

#10 2021-12-18 16:44:14

kd.gundermann
Member
Registered: 2018-02-27
Posts: 34

Re: Can two XSIbackup instances use the same repository simultaneously?

Thank you for your fast response !! (didn't expect that you are working on a weekend)

Could you please then use the full path including the subdir in the error message ?

Thanks

Klaus

Offline

#11 2021-12-18 20:08:40

admin
Administrator
Registered: 2017-04-21
Posts: 2,057

Re: Can two XSIbackup instances use the same repository simultaneously?

Next version will include the improvement.

Offline

Board footer