©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#426 Re: General matters » All Snapshots are deleted » 2021-06-15 14:16:38

That text was outdated, we corrected the information.
Nonetheless this is stated in the program's output.

#427 Re: General matters » Unable to create cbt replica (solved) » 2021-06-15 14:09:09

When a CBT sync is performed, the .vmdk file descriptors need to be updated in the .map files of the replica to reflect the new CID.
When a subsequent sync cycle is run (c)XSIBackup-DC checks that the hash of the .vmdk file descriptors is the same as those stored in the corresponding .map files. When some discrepancy is detected, the CBT scheme is reinitilized.

There are mainly two reasons that could lead to this:

1/ The remote VM was switched on without a test snapshot (--options=R) and the CID changed as per the VM activity. This is the simplest explanation and the first we think of, although at this point it seems clear that is not the case.

2/ The remote hashes of the .vmdk descriptor files failed to be updated. This is the most likely cause as of today. That's why we need the full log. If this is the cause, then there must be some special condition: it could be some bug or some faulty component of the ESXi shell (which we have found to be relatively frequent). Having the exact ESXi version and the rest of details present in the log is fundamental to delve into the issue.

#428 Re: General matters » Replica with large diff size » 2021-06-15 14:08:19

Yes, there's another post open with the same problem. As it's more recent let's stick to it to follow the issue.
When a CBT sync is performed, the .vmdk file descriptors need to be updated in the .map files of the replica to reflect the new CID.
When a subsequent sync cycle is run (c)XSIBackup-DC checks that the hash of the .vmdk file descriptors is the same as those stored in the corresponding .map files. When some discrepancy is detected, the CBT scheme is reinitilized.

There are mainly two reasons that could lead to this:

1/ The remote VM was switched on without a test snapshot (--options=R) and the CID changed as per the VM activity. This is the simplest explanation and the first we think of, although at this point it seems clear that is not the case.

2/ The remote hashes of the .vmdk descriptor files failed to be updated. This is the most likely cause as of today. That's why we need the full log. If this is the cause, then there must be some special condition: it could be some bug or some faulty component of the ESXi shell (which we have found to be relatively frequent). Having the exact ESXi version and the rest of details present in the log is fundamental to delve into the issue.

#429 Re: General matters » 'data' folder does not exit » 2021-06-15 10:58:40

That's just a small bug returning that spurious message. We believe to have fixed that in 1.5.0.9 that will be available in some days. Nothing to worry about.

#430 Re: General matters » When you quote movies you should quote them correctly ;) » 2021-06-15 10:56:41

Thank you for your feedback, we can see that there are other old dogs around this forum ;-)
We made the amendments which will be visible when the CDN cache is refreshed.

Given that Mr. Miyagui is a Japanese name, I believe there isn't any right spelling for it, as it's just a phonetic interpretation of the original name.

I'd swear Mr. Miyagui said that in Karate Kid. Nonetheless that's an archetypal character: the wise old master, I bet Yoda said something similar sometime.

#431 Re: General matters » Unable to create cbt replica (solved) » 2021-06-14 22:09:21

The remote hashes could happen to not be updated.
Could you post the full output of the previous job and the current one please?
Have you tried without quiescing?.

#432 Re: General matters » Backup LAN host from esxibackup-gui » 2021-06-14 22:06:19

Sure, you have some examples in the manual page:
[url]https://33hops.com/xsibackup-dc-full-manual-home.html#over-ip[/url]

#433 Re: General matters » Improper use of SMTP command pipelining (solved) » 2021-06-14 15:46:05

Some clients report this issue very seldomly. It is due to the SMTP server being configured with a delay in between commands. Namely, if you don't wait some time in between SMTP commands, the receving SMTP server breaks the protocol on its own.

In case of (c)XSIBackup this can be fixed by setting the last field in the smtpsrvs.conf file to 1 (seconds) or above. One second is usually enough.

If you fell here trying to debug your own script, just add some sleep time in between SMTP commands.

#434 Re: General matters » Replica XSIBackup-DC versions » 2021-06-11 15:29:16

They are just kept there to know about the history of events. They are useless from a recovery point of view.
You can manually remove them, should you ever use the replica in production.

#435 Re: General matters » Details on the --on-success and --on-error options » 2021-06-11 15:17:14

Those are in the end fake options for users not willing to understand the details of how the Bash shell works.

If you are planning to do more elaborated things like the ones you are describing, then forgetting about --on-* arguments is a must.

(c)XSIBackup-Pro Classic is in the end a shell script, you can interact with it as such and pass whatever arguments you want, or just query the outcome of the script execution and take appropiate action in each case: error | success

#436 Re: General matters » All Snapshots are deleted » 2021-06-10 07:59:02

That depends on what (c)ESXi version you are using. From (c)ESXi 7.0 all files in a virtual machine are read blocked, even when the VM is running on top of a snapshot, thus keeping preexisting snapshots is not possible. This is by design and a decission taken at (c)VMWare.

#437 Re: General matters » using DC to maintain a set of replicas » 2021-06-08 07:22:07

We don't know the finer details of your setup or procedure, it's easy to find out though: run the --replica while not doing that.

UPDATE:

Asuming you already fixed the issue with your remote replica _XSIREP registering issue, which BTW has already been addressed through a more permissive logic that simply ignores CBT if some inconsistency is detected in version 1.5.0.7, the issue you describe can be caused by some problem updating the remote .map files CID. That should in any case throw a clear error in case something goes wrong.

#438 Re: General matters » Does XSIBackup Pro work with ESXI 7? » 2021-06-08 07:19:51

Thank you for your feedback.
As Carl Gustav Jung would say: you made the unconscious conscious and a whole new world opened up in front of you.

#439 Re: General matters » Replica with large diff size » 2021-06-07 15:40:21

Well, that was very foreseable, now the question is, who or what?

#440 Re: General matters » using DC to maintain a set of replicas » 2021-06-06 09:59:29

This is so weird. There's clearly something altering the .vmdk file descriptor CID or unique identifiers that allow (c)XSIBackup to know whether the disks have changed since last synchronization. Why don't you just use a totally different volume just to make sure that you don't have some other software interfeering in your replicas?.

#441 Re: General matters » using DC to maintain a set of replicas » 2021-06-05 10:41:55

Something might be changing the CID of the .vmdk disks in the remote replica.
Are you switching the replicated VM on after a replica cycle?
If you do so you will force a new full syncronization cycle.
To prevent that from happening, you have the --options=R argument which will register the replicated VM and create an empty snapshot to be used as a sandbox. Then you can switch the VM without altering the CID of the base disks. Do not manually remove that snapshot, in fact the snapshot info contains a clear "Do not delete me" notice, let the --replica=cbt manage that snapshot. It will be wiped and recreated on every CBT round.

#442 Re: General matters » SMTP broken after upgrade to XSIBackup-Pro 1.5.0.6 » 2021-06-05 10:28:53

It is the mail-from address that is required to contain an @ sign. The username field is much more permissive, it requires 1 to 70 chars of any kind.
There is indeed an error in the REGEXP pattern which is that the hyphen or dash character needs to be at the end (or at the beginning) of the character class to be interpreted literally. We have fixed that and will be published as 1.5.0.7. We have also extended the port to allow using up to 5 digits, in case somebody wants to use some non-standard port.

We checked your server with grep regexp, which is very similar to the one we used, and it passed the test.

echo "1;smtp.domain.ca:587;backup-logs@domain.ca;1234567;abcdef123456789;none;TLS;0" | grep -E "^([0-9]{1,3});([0-9a-zA-Z._-]
{1,70}):[0-9]{2,5};([0-9a-zA-Z._-]{1,70}@.*{1,70});.{1,70};.{4,70};[a-zA-Z]{3,4};[a-zA-Z]{3,4};[0-5]{1}$"

This validation had to be added, as well as some others to prevent spurious characters in the smtpsrvs.conf file breaking execution and causing segfaults.

(*) Please note that the hyphen issue is resolved in the pattern we used. This could break behavior of the REGEXP test. Please try with 1.5.0.7 available since 2021-06-06. In fact, if you move the hyphen from the last position to the previous one in the two character classes where it is used, this very same server line will not pass the REGEXP check, thus, that is proof that the hyphen misplacement is the source of your issue too.

#443 Re: General matters » Replica with large diff size » 2021-06-04 22:40:24

Maybe a full resync took place, can you share the full log?.

#444 Re: Feature requests & improvements » Logs per Job » 2021-06-04 22:36:34

Well, you already have all that you need to easily arrange something like that. Let us know what your exact goal is and we can prepare some post on the subject which will for sure be interesting to other people.

You can exchange keys with any kind of server and log that info to a Mysql server, to some Syslog server, with which we already offer comatibility, etc...

#445 Re: General matters » VM replication completes with segmentation fault » 2021-06-03 11:45:30

That segfault is produced at the end of the backup process right?.

We know it probably has something to do with your smtpsrvs.conf file, that's why we added more strict validations in v. 1.5.0.6, so that any spurious characters or empty lines that you may have introduced by editing the file manually are conveniently filtered out to prevent uncontrolled errors.

There are other text files which are writtent to after the backup that could cause this behaviour too.
We have not been able to reproduce this kind of issue consistently though, thus we will keep on revising and reinforcing checks after the backup.

From your part, just make sure that the smtpsrvs.conf file looks healthy, you may check with some advanced text editor such as Notepad++, enable the view of all characters and check that you don't have weird things amidst the SMTP servers' config. Also make sure that you don't have characters that may need scaping in your password fields.

#446 Re: General matters » XSIDirector » 2021-06-03 11:39:46

There must be some text inconsistency, we'll check that, there's only one version of the tool. You can download from here:
[url=https://33hops.com/xsidirector-esxi-management-deployment.html]XSIDirector Home Page[/url]

#447 Re: General matters » using DC to maintain a set of replicas » 2021-06-01 13:59:09

The error:

msg = "The specified key, name, or identifier '2' already exists."

Denotes some inconsistency in your VM inventory. You have wether removed some VM directory but you didn't remove the VM entry from the inventory, you have some duplicate VM name or you provoked some conflict around the consistency of the (c)ESXi host contents metadata.

(c)VMWare allows some situations, like registering a VM by the same name. (c)XSIBackup detects VMs by name, as using the Ids would not be very convenient from a user perspective. This forces and requires coherence from a naming point of view.

In any case, the error you are getting is being thrown by [b]vim-cmd[/b], thus there must be some situation that trascends a simple duplicate name. The message states you have a duplicate key in the VM inventory.

#448 Re: General matters » Unable to backup VM seems permission error and locked (solved) » 2021-05-31 16:48:39

We aren't getting you. We asume that you are indeed running (c)vSphere (c)ESXi in your host. What would be the reason to post here if you weren't running this system?

On addition to that. Please take the time to learn what an independent disk is and how it differentiates from other types of storage in (c)ESXi: [url=https://cormachogan.com/2013/04/16/what-are-dependent-independent-disks-persistent-and-non-persisent-modes/]post on (c)ESXi disk modes[/url]

Please be aware that independent disks cannot be hot backed up, use --backup-how=cold instead.

The message is already stating that point, thus: we don't understand where your doubt exactly is or what do your expect from us.

#449 Re: General matters » Unable to backup VM seems permission error and locked (solved) » 2021-05-31 14:19:55

We don't know what we can add to the error message:

The file "/vmfs/volumes/datastore1/VM1.TEST.local/VM1.TEST.local-flat.vmdk" is locked. Independent disks cannot be hot backed up, issue a cold backup for or exclude the disk from the backup. Make sure that the VM state (on/off) was not manually changed during the backup

Is that disk independent?, is it engaged in some scheme so that another process is holding it open?
Make sure that you don't have any previous (c)XSIBackup process running

ps -c | grep xsi

#450 Re: General matters » using DC to maintain a set of replicas » 2021-05-31 11:33:38

Your asumptions are mainly correct. The --replica=cbt process generates a replica, which is an exact copy of the replicated VM. The test snapshot is spurious and its only aim is to serve as a sandbox to check the VMs, it's always discarded on the subsequent --replica=cbt cycle and if you happen to manually mix the spurious data (the one generated during a punctual VM test) by deleting the test snapshot through the GUI, you will break the replica process. This will in fact be detected on the following CBT cycle and the --replica will be resync'ed

Now on the empirical side of your tests:

The .locked file is automatically deleted at the end of any replica cycle, thus, if it is still there at the end, then something is not working as expected.

You can replicate data anywhere: a Linux compatible server or another (c)ESXi host. When you replicate to another (c)ESXi host, you happen to be doing it to some system that allows you to perform this kind of "sandbox" testing via a snapshot. Still, you will need to pay attention to some basic principles, like the ESXi hosts being coherent in terms of HW and VMFS version.

If you mix VMFS or HW versions, you will experience the same effects as if you would the same any other way, in the end (c)XSIBackup is just a tool. You take the decissions on what to do with it.

In any case we have just started an especulative thread. If you need concrete answers on concrete matters, please post the job and the output of the job. There are two definitive issues around your description:

1/ .locked file not deleted.
2/ No snapshot, even though you have requested it to be taken as per the --options=R argument.

Board footer