Forum ©XSIBackup: ©VMWare ©ESXi Backup Software

andreidr

Member

Registered: 2018-10-27

Posts: 3

AWS S3 bucket size problem

Hi,

I am using AWS S3 bucket mounted as NFS for an off-site backup, besides the local backup.

When the backup starts, the program says that I have a huge amount of GB left (something like 86738893662883 GB) but it has no room for my backup (about 516 GB) and it needs to delete all the snapshots in order to free up space.

Clearly, the S3 bucket has enough space so deleting all the snapshots is not necessary.

Is this a bug?

admin

Administrator

Registered: 2017-04-21

Posts: 2,055

Re: AWS S3 bucket size problem

Well, first of all XSIBackup does not announce any sort of compatibility with AWS S3 buckets. It nevertheless supports open standards like NFS, so just as long as you are able to properly mount your NFS share in ESXi, everything should work.

Do not forget to always use NFS 3, as we state in every NFS related post; NFS 4.1 is useless in case of backups and, on the other side, does not even work from the ESXi shell.

You must note that NFS is a complex protocol in which a number of different ports are used in an intrincate and complex relationship between the NFS client and host, NFS was never designed to work accross the Internet, thus mounting NFS over a WAN is normally performed tunneled to avoid having to deal with that complex scheme and having to port it to any firewalls in between.

Beeing able to mount the NFS share, does not mean that the NFS protocol is properly set up. By what you are describing, it looks like NFS is not working fine and you are loosing some fundamental information.

https://serverfault.com/questions/58862 … e-internet

andreidr

Member

Registered: 2018-10-27

Posts: 3

Re: AWS S3 bucket size problem

S3 is set up via a VM supplied by Amazon - AWS Storage Gateway (https://docs.aws.amazon.com/storagegate … mware.html). There is not a lot of set up that I can do.

I am no computer expert, but is seems odd that XSIBackup reports a lot of free GB but still wants to free space.

admin

Administrator

Registered: 2017-04-21

Posts: 2,055

Re: AWS S3 bucket size problem

That product seems to be designed to be used from within S3 environment.
As stated: NFS is not an easy protocol to be used over a WAN. If you are not a computer expert, then you probably have chosen a very picky subject to deal with.

Are you seeing the right figures when issuing df -h from the command line?
XSIBackup just picks the information from the shell.

andreidr

Member

Registered: 2018-10-27

Posts: 3

Re: AWS S3 bucket size problem

Hi,

The Storage Gateway VM runs on the local ESXi Server. This VM is specifically designed for ESXi. The XSIBackup makes the backup to the NFS mount point shared by this local machine. The NFS Share is not over WAN. This VM has a cache drive from which the VM uploads the data to AWS S3.

From what I understand, XSIBackup "cannot" know that it is like this - it should look like a regular NFS.

df -h dows not show the right figures:

Filesystem   Size   Used Available Use% Mounted on
NFS          None   0.0B      None   0% /vmfs/volumes/AWS <<- THIS IS THE MOUNT
VMFS-6       1.8T   1.0T    781.6G  58% /vmfs/volumes/datastore1
VMFS-6       1.8T   1.8T     17.6G  99% /vmfs/volumes/NVR
VMFS-6       1.8T 579.5G      1.3T  31% /vmfs/volumes/BACKUP

admin

Administrator

Registered: 2017-04-21

Posts: 2,055

Re: AWS S3 bucket size problem

O.K., thank you for the explanation. We had just taken a quick glance at the product. Being as you say, there's nothing that should prevent the appliance from being correctly mounted and its size reported accurately.

Please, note that the issue has nothing to do with XSIBackup, as the figures you see from a simple df command are equally wrong. You should contact Amazon, they should know what's it all about, although I suspect it might be the NFS version. As stated NFS 4.1 will not work from the ESXi shell, this is an ESXi bug and has nothing to do with XSIBackup either, mount the appliance as NFS 3.