©XSIBackup-Free: Free Backup Software for ©VMWare ©ESXi

Forum ©XSIBackup: ©VMWare ©ESXi Backup Software


You are not logged in.

#1 Re: General matters » vmdk not found, but it's there. Backups seem to be stuck (solved) » 2021-06-16 11:49:17

The same thing happened today as well. The log file contains just the one line with " error detecting .vmdk file".
Is there a way to enable some debug logging to figure out what's wrong? I powered up the VM yesterday and worked like 5 hours on it, so the disk seem to be ok.
On the other hand, if the underlying SSD is starting to fail and that is the root cause of this, I'd like to know.

In case it matters, I'm doing a backup to a deduplicated synology nas. Would the backups be ok if I switch back to 1.14? And are they compatible? As in later on if I were to restore with 1.5 or newer, will the ones made by 1.14 be usable?

#2 Re: General matters » vmdk not found, but it's there. Backups seem to be stuck (solved) » 2021-06-15 14:35:34

Thanks. Renaming to bad sounds reasonable. I will retry the backup tomorrow.
Thank you.

#3 Re: General matters » vmdk not found, but it's there. Backups seem to be stuck (solved) » 2021-06-15 14:06:39

Well, I had to start work and the VM in question is my main dev VM. I resumed it and it started up just fine. The disk in question is usable so no idea why the backup errored.

The backup is still in that "waiting" state, nothing changed there.

#4 General matters » vmdk not found, but it's there. Backups seem to be stuck (solved) » 2021-06-15 11:25:42

mvfid
Replies: 6

On one of my ESXi's (5.1.0) the first backup, gave the following error (xsibackup 1.5.0.8):

Error code 3649 at file xsibackup.c, line 3649 | Error description: error detecting .vmdk file

I did a ls -l on that same filepath and I got results just fine.
The VM is in suspended state.
The VM has 3 vmdk files attached, each on a different storage, the other 2 were backed up ok.

For over 1 hour now it's sitting in the following state:

   21/21                                 XXXX.vmdk        486.00 B     | Done  44.76%
-------------------------------------------------------------------------------------------------------------
Total size:                                                                     76.17 GB    | Done 100.00%
-------------------------------------------------------------------------------------------------------------

Not sure if this is related to the error above or not.

I do have another esxi which is also doing backups during this time. That one seems to be progressing ok (also running 1.5.0.8) I assume this shouldn't have any impact but I'm putting it out there in case it does: the backed up VMs are different, but they do backup to the same place. This was working fine previously with 1.4 (both doing backups at the same time to the same NAS folder). And I am using the same scripts on both sides, so nothing changed from that POV, just the upgrade from 1.4 to 1.5.

Any ideas what may cause this issue? Maybe where to look to see what it's doing? And what I can do about it?
Safe to kill it and re-try the backup?
If I kill it, will that result in some garbage that I should manually cleanup?

#6 General matters » 'data' folder does not exit » 2021-06-15 10:06:51

mvfid
Replies: 2

Hello, just upgraded from 1.4.x to 1.5.0.8

every time I ran a backup, It says
"Remote 'data' folder does not exist, creating it..."
Target is a Synology NAS
Obviously, the folder exists, I've been running backups for months. I don't recall if 1.4.x did the same thing or not.
It doesn't seem to cause issues with the backup so it may just be a display thing.

#8 Re: General matters » Error code 162 during backup » 2021-05-04 10:50:41

Thank you for the details. I think I know what happened. Checking the synology logs I see that the station entered suspend mode. I have configured it to auto shutdown after 30 mins of inactivity and apparently the backup took longer with no changed chunks to send. I will need to write a script to keep the station "alive" during the backup.

Thanks.

#9 General matters » Error code 162 during backup » 2021-05-04 06:29:18

mvfid
Replies: 4

I get the following error
::: detail ::: 13.63% done | block 13117 out of 96256                                       | Done  41.99%
2021-05-03T12:17:25 | Error code 162 at file signal.c, line 162 | Error description: raised SIGTERM (13) (4) in job, num of errors: 13, check error.log
-----------------------------------------------------------------------------------------------------------
Removed current target dir: /volume1/esxi-backups/20210503080422/TH-Procura-W7-DXE10-Dev
-----------------------------------------------------------------------------------------------------------
SIGTERM (13) condition was trapped: check logs for more details
-----------------------------------------------------------------------------------------------------------
Cleaning up...


the /vmfs/volumes/5670f39a-4ffdf3c1-2dfb-0015174b3091/XSI/XSIBackup-DC/var/log/error.log file contains just the line with the error code from above, nothing else related to this VM.

This esxi server does have some HDD issues, data recovery undergoing for those HDDs and as a result 7 VMs are missing files. This is not one of those. So I do have 7 errors above this one like the following
Error code 439 at file esxi.c, line 439 | Error description: .vmx file not present at: /vmfs/volumes/5671067a-3d6f31ef-56ab-0015174b3091/DB-MsSqlServerExpress-19/DB-MsSqlServerExpress-19.vmx, did you delete the VM files?, please fix that

But these are expected for now. I'm in the process of backing up the other VMs but now with this error I am worried that maybe other HDDs/SSDs have data issues as well, so can someone please explain what that error code means?
And if there are any other log files to get more info from? Or things to enable to get extra details?

Thank you.

#10 General matters » Same data backed up multiple times » 2021-05-04 06:15:19

mvfid
Replies: 1

I'm running backup manually, shouldn't matter but mentioning it, if it does.
Command is: /scratch/XSI/XSIBackup-DC/xsibackup --backup "VMs(VM1,VM2,VM3,VM4,VM5,VM6,VM7,VM8,VM9)" adm@synology-IP:22:/volume1/esxi-backups/

You can see that during backing up of VM2 it also backs up HDDs of VM1 (lines 155-165)
Same for VM3 (lines 247-257), VM4 (lines 321-331), VM5 (lines 393-403), VM6 (lines 513-523), VM7 (lines 607-617), VM8 (lines 725-729), VM9 (lines 847-851)

Checking the synology drive data, I cannot see the drives of VM1 in the VM2 folder, so I assume the backup is ok, but it's just an assumption. Can someone investigate this and confirm that doing a restore of VM2 will NOT in fact restore anything from VM1? Because that would be nasty.
Then obviously this should be fixed since it keeps processing the same hdds for every VM in the backup and while it doesn't seem to have a huge impact, it adds up.

The second potential problem are some excluded files. For ex on VM3 lines 243-245 I see what I am assuming to be snapshots being skipped (some other VMs have this as well). In theory that should be fine because this VM does not have snapshots as far as I can tell (via snapshot manager and actually checking the folder of the VM) but then where is it getting these from?

Thank you.

#11 General matters » Minor documentation inconsistency » 2021-04-21 08:11:39

mvfid
Replies: 1

Hello

following is current doc: (c)XSIBackup Block Size

--block-size[=1M(default)|10M|20M|50M]: (optional) this is the block size that will be used to deduplicate data when using the --backup action. In case of replicas a fixed block size of 1M will be used. You can choose between: 1, 10, 20, or 50 megabyte block sizes when performing a --backup action. 10M is the default --block-size.

Note how that at beginning default is 1 M, and at the end it's 10M.

#12 Re: General matters » Supended VM cannot be powerd on during backup » 2021-03-12 15:59:59

While I understand the "while car is running getting down of it is stupid" statment, a suspended VM is not actually running.
Plus, you can make a snapshot of a suspended VM.

So adding all this together, is not that unreasonable for one to assume that a backup software that is capable of taking a hot backup of a running VM without disturbing it, should be able to handle backing up a suspended VM that later will be powered on.

I will consider this a limitation of the software, and I will schedule my backups of "work VMs" to be done during my sleeping hours, that way there is little chance to power them on. Although every now and then it happens that I work through the night, but I guess I can deal with that if/when that happens.

Thank you

#13 General matters » Supended VM cannot be powerd on during backup » 2021-03-12 14:20:52

mvfid
Replies: 3

I started a backup on a suspended VM and after a while, since the backup was still running I figured I could just power on the VM and get on with my work.
However I got an error during power on saying
"Cannot open the disk XYZ or one of the snapshot disks it depends on."

Is there a way around it? some option for the backup maybe?

#14 Re: General matters » Consistency when backup interrupted » 2021-03-12 14:04:23

That sounds good. Thanks you for the detailed explanations.

#16 General matters » Consistency when backup interrupted » 2021-03-12 13:08:10

mvfid
Replies: 2

I'm thinking ahead off possible failures and what will happen to the backup process.
Say for whatever reason, the esxi host crashes, or the network between it and the backup destination is lost.
How does xsibackup handle such scenarios? Does it have some resume support? will it cleanup the failed backup so it can be run again?
And should I worry about garbage in the backup destination?

#18 Re: General Matters » Why dedicated NAS xsi backup server? » 2021-03-12 11:46:03

I guess I am replying to my own question. Reading some posts I found post #2 on this topic to actually answer my dilemma.
XSIBackup Server Client

#19 Re: General matters » skipped excluded and open excluded » 2021-03-12 11:41:30

I think just documenting these in the manual is ok. I only posted this because there was nothing in the manual around this behavior and I also was not able to find any discussions on the subject.

Thank you.

#20 General matters » skipped excluded and open excluded » 2021-03-12 11:28:33

mvfid
Replies: 4

Hello,

I just ran my first backup, it was of a running windows VM. I will paste here some relevant lines from the output
(esxi user = root, synology user = admin)

command line: ./xsibackup --backup "VMs(xyz)" admin@a.b.c.d:22:/volume1/esxi-backups/


|||   (c)XSIBackup-DC 1.4.3.13: Backup & Replication Software                   |||
System Information: ESXi, Kernel 6 Major 0 Minor 0 Patch 0
xsibackup is licensed
Remote system: Linux
-----------------------------------------------------------------------------------------------------------
PID: 677403, Running job as: admin
-----------------------------------------------------------------------------------------------------------
LZJB compression has been enabled
-----------------------------------------------------------------------------------------------------------
Block size is 1.00 MB (1048576 bytes)
-----------------------------------------------------------------------------------------------------------
Remote xsibackup binary found at: /bin/xsibackup

Performing --backup action
-----------------------------------------------------------------------------------------------------------
(c)XSIBackup-DC setting repository at /volume1/esxi-backups
-----------------------------------------------------------------------------------------------------------
.xsitools file created at: admin@a.b.c.d:22:/volume1/esxi-backups/.xsitools
-----------------------------------------------------------------------------------------------------------
Item number 1 in this job

Virtual Machine Name: xyz
Creating snapshot VM : xyz (powered on)
*** Snapshot was successfully created ***
Virtual Machine: xyz
Backup start date: 2021-03-12T10:09:42
-----------------------------------------------------------------------------------------------------------
2021-03-12 10:09:42 | Backing up 26 files, total size is 52.46 GB
-----------------------------------------------------------------------------------------------------------
    NUMBER                                                         FILE             SIZE          PROGRESS

following are the files transferred and the following are of interest, that is the ones that are marked excluded.

xyz-ae0ee284.vswp                    [open excluded]
vmx-xyz-2920211076-1.vswp                    [open excluded]
xyz.vmx.lck                 [skipped excluded]
xyz-000001-delta.vmdk                 [skipped excluded]
xyz-000001.vmdk                 [skipped excluded]
*** Snapshot was removed ***
Item backup completed without errors
Differential blocks were added to the .blocklog database
-----------------------------------------------------------------------------------------------------------
Final checksum: 27167242484 bytes were sent and confirmed to have been written remotely
-----------------------------------------------------------------------------------------------------------
Removed host <tmp> dir        OK
-----------------------------------------------------------------------------------------------------------
Removed prog <tmp> dir        OK
-----------------------------------------------------------------------------------------------------------
Unlocked backup               OK
-----------------------------------------------------------------------------------------------------------
Removed PID                   OK
-----------------------------------------------------------------------------------------------------------

Why these files were excluded?

I don't have a test VM to play around with so I used a live one; I should probably make some but am strapped for time for the next couple weeks, I'd rather get the backups working.

So another thing that comes to mind: given the VM is running and the snapshot and swap files seems to have been skipped, what will happen if I need to restore? Will it restore to the same running state? or will it be broken?

Thank you

#22 Re: General matters » xsibackup-dc on synology nas installation » 2021-03-12 11:04:10

Right, I understand that, but your documents/manuals imply that synology nas is actually a supported system and it's easy to setup. From what you say now that seems to no longer be the case. A bit confusing.

In any case, at least the home path issue looks like an easy fix.

#23 General matters » xsibackup-dc on synology nas installation » 2021-03-12 10:13:39

mvfid
Replies: 6

I am trying to get this installed on my DS220+ (DSM 6.2.4-25554) using instructions from Run (c)XSIBackup as a server in Synology NAS
I have enabled SSH on the NAS but cannot log in using root. It appears this is some security limitation in newer versions. Using my administrator user works though.
I also enabled the "homes" as per the instructions which created the home dir on /volume1/homes/<user>/

The next problem would be that using the GUI it is not able to detect the correct home directory.
Instead it detects the system as "linux" and tries to create the authorized_keys in /home/<user>/.ssh/
I think it might be a better option to try and read the HOME env variable after being logged in.

So I had to use the cli to add the key using my admin user (not the "root" as specified in your instructions). It seems to have gone well as I can see the authorized_keys file /volume1/homes/<user>/.ssh/authorized_keys and the cli process didn't complain of anything oher than asking for the password a bunch of times.

Then I went on and opened the GUI again and checked the linked servers window, but nothing showed up there.
At this point I am assuming that the cli worked and that the GUI and cli do not use the same information to list the linked servers, which I think is a bug.

I did try to use the GUI to set up a job and it complained about no linked servers, so that's a stopper.


I also checked the web gui of the NAS and it shows some weird things there, both a "home" a "homes" folder and while "homes" contains a few users and the one I set up having the .ssh folder, "home" also has a ".ssh" folder and apparently the same authorized_keys file (by change date and file size)
However searching the system from cli does not reveal any such extra file so it might just be some synology bug.

Long story short, how do I get this setup so the gui works as well?

note that I did run a backup command manually as per the instructions and that worked. (well, at the time of this writing, it's still going on, it has 52 GB to transfer over)

Thanks

#24 Re: General Matters » Why dedicated NAS xsi backup server? » 2021-03-08 11:45:46

Thank you for your reply.
I didn't know about those options; I was reading up on this article: Running (c)XSIBackup-DC as a server in (c)Synology NAS (which btw says it's part 1 but I didn't find a link to part 2).

So my questions are in the context of this linked article.
Thank you.

#25 General Matters » Why dedicated NAS xsi backup server? » 2021-03-07 15:15:01

mvfid
Replies: 4

Hello,

I'm trying to understand where xsi backup NAS comes into play and why. How is it better than just using a NFS share for example to the same NAS?
And can it be used by different esxi hosts? (as in different versions, eg, 5 and 6 )

Thank you,

Board footer