You are not logged in.
We have found custom images to be pretty buggy, not only this brand but some others too. We recommend that you stick to official VMWare releases.
When you run this command
vim-cmd hostsvc/firmware/backup_configThe [b].tgz[/b] file is saved to [b]/scratch/downloads[/b] by default.
If you have some manufacturer customized ESXi distro, or some other sort of custom one, you might need to edit the [b]src/esxbackup[/b] file to adapt it to your distro.
Yes, you just need to mount the .vmdk file to any VM and copy the file wherever you want.
FILE LEVEL RESTORE Question.
I have not read anything much concerning file level restores from the VM backups.
Is it possible to restore individual files and how? Thanks.
Can you offer some more details?, namely: backup job string and complete output log.
We are starting to mix different issues into this topic.
We had previously detected an issue with dd not being able to access a .vmdk file when checking its first 50M (Jayce pointed this out). This is most probably due to some other process holding it. We have not been able to reproduce this issue so far, and might not be a bug.
Getting a [b]Segmentation fault[/b] or [b]file too large[/b] from part of [b]dd[/b] are traces of a different sort of problem.
dd uses C file handling functions, these functions generally use a type [b]off_t[/b] to seek inside files. That error (file too large) means you have hit the limit of a data type. It could be due to your file being extremely big, or to a bug in the dd binary of your ESXi build.
We are willing to help you, but although we have insisted in getting some more details about this issue, we have nothing but comments.
If this topic has to be considered related to a real problem, please provide:
- Backup job string with all arguments (except passwords)
- ESXi version and build number.
- Logs containing the exact errors you are getting
[b][https://33hops.com/xsibackup-pro-onediff.html]OneDiff[/url][/b] works both: "locally", which in this case means from a locally attached datastore to another locally attached datastore and, over IP, namely, OneDiff moves the differential data from a local datastore to any other ESXi host reachable over IP. Obvioulsly to any local path attached to that remote server, but I should not need to especify that, it's implicit.
[b]Other end[/b] means the target device, whatsoever will be receiving the data being backed up. If I use the expression [i]other end[/i] and do not especify whether it is a local datastore or an ESXi server on the other end of the world, reachable by IP, then I'm using a vage term deliberately, because I'm talking in conceptual terms and I want to leave that option open.
If you perform a [b]OneDiff[/b] backup to a remote server (reachable over IP), you then have a VM which has been mirrored and is an exact copy of the VM that it mirrors, and that is ready to be swicthed on, thus a usable VM from all points of view. That VM is on a different host than the production ESXi host where the VM being backed up was, and thus the [b][https://33hops.com/xsibackup-pro-onediff-backup-retention-policy.html]referenced post[/url][/b], which is written in a [b][url=https://en.wikipedia.org/wiki/Socratic_method]maieutic[/url][/b] style, states that you can then chain a backup, to be performed once the first backup finishes, that is what the term [b]link[/b] means.
That additional backup can, and it's what the post proposes, be an XSITools backup, which will archive the just backed up VM to a repository. As the first backup which we will be using as the source of the second is in a different ESXi host, and maybe even in a different datacenter, the load associated to performing that second backup, won't interfeere with the production ESXi host.
A repository is a storage format that hosts data, like a .zip file. In this particular case, hosts the chunks of data that compose the XSITools deduplicated Virtual Machines. VMs stored in a repository cannot be used directly, as they are not in an operational state, just like a file cannot be used when it's stored in a .zip file, it first needs to be extracted, or in the case of an XSITools repository, restored.
You don't need a previous backup to backup to Borg, you can do so directly. The drawback is that XSIBackup will send [b]all data[/b] to the Borg server. There's currently no possibility to run a Borg client in ESXi.
Yes, you can VMFS format your NFS volume, but the fact that you can does not mean you must, or that using a different File System is not correct.
[url]https://pubs.vmware.com/vsphere-6-0/index.jsp?topic=%2Fcom.vmware.vcli.examples.doc%2Fcli_manage_storage.6.9.html[/url]
Before choosing the file system you'll be using in your NFS attached device you should take into account some considerations:
- Will it be useful to use VMFS?
- Are there other File Systems you can use?
- Do you have some advantage by using a different Filesystem like ext4, btrfs, XFS, etc...?
If you want to use that NFS attached datastore to host deduplicated data, that is compounded by thousands of indexed small chunks of data, you may decide that XFS or ext4 are best choices than VMFS, as they can handle many times the amount of inodes that VMFS can.
If you want to perform differential backups locally, then [b][https://33hops.com/xsibackup-pro-onediff.html]OneDiff[/url][/b] is your choice. It will store changed blocks in a snapshot and move only that snapshot to be merged with the previously backed up data on each backup cycle.
[b][https://33hops.com/xsitools-vmfs-deduplication.html]XSITools[/url][/b] will copy your VMs to a deduplicated + compressed repository, thus you will need to restore your VMs prior to using them, while with OneDiff the VM will be ready to be switched on. XSITools is also differential, but blocks hashes need to be compared, so each differential chunk needs at least the time to calculate its checksum added to its backup time, whereas in OneDiff the differential blocks are kept aside and don't need to be compared against anything.
Well, in sake of speed I would backup to a different physical disk or array, if you can make it be a different datastore at the end of an NFS share better, as you will, on addition, count with some sort of physical distance between your main datastore, where the production VMs are, and your backups.
To protect backups against a physical theft, just use encryption in your backup file system.
Not at all, just use [b]openssl --help[/b] to get a list of all openssl digests:
openssl:Error: '--help' is an invalid command.
Standard commands
asn1parse ca ciphers cms
crl crl2pkcs7 dgst dh
dhparam dsa dsaparam ec
ecparam enc engine errstr
gendh gendsa genpkey genrsa
nseq ocsp passwd pkcs12
pkcs7 pkcs8 pkey pkeyparam
pkeyutl prime rand req
rsa rsautl s_client s_server
s_time sess_id smime speed
spkac srp ts verify
version x509
Message Digest commands (see the `dgst' command for more details)
md4 md5 rmd160 sha
sha1
Cipher commands (see the `enc' command for more details)
aes-128-cbc aes-128-ecb aes-192-cbc aes-192-ecb
aes-256-cbc aes-256-ecb base64 bf
bf-cbc bf-cfb bf-ecb bf-ofb
camellia-128-cbc camellia-128-ecb camellia-192-cbc camellia-192-ecb
camellia-256-cbc camellia-256-ecb cast cast-cbc
cast5-cbc cast5-cfb cast5-ecb cast5-ofb
des des-cbc des-cfb des-ecb
des-ede des-ede-cbc des-ede-cfb des-ede-ofb
des-ede3 des-ede3-cbc des-ede3-cfb des-ede3-ofb
des-ofb des3 desx rc2
rc2-40-cbc rc2-64-cbc rc2-cbc rc2-cfb
rc2-ecb rc2-ofb rc4 rc4-40
seed seed-cbc seed-cfb seed-ecb
seed-ofba - openssl sha1 yourfile
b - openssl md5 yourfile
c - openssl md4 yourfileOf course you have to run any of those commands on both sides and compare the obtained checksums. You may need to switch off your VM to perform the backup and the manual checksum comparison, as a running VM could change the contents of the open disks.
thx sistemi
Well, first of all you need a fixed IP or a "kind of". You may use one of those Dynamic DNS services to get a fixed domain name:
- [url]https://www.noip.com/free[/url]
Mac OSX is Unix, but not compatible with Linux binaries. You could use Rsync on the Mac side, but that would force you to use a close version to 3.1.0 to prevent Rsync protocol errors, and even if you succeded, it would be slow.
I think your best bet would be to just use a Linux VM running on your Mac connected to your storage pool. You can download a free .ova package here: [url]https://drive.google.com/open?id=0B5o7NiDeLLWSbzlON2szTWZyUTg[/url]
This Linux VM can run with as little as 512 mb. of RAM and still offer you maximum speed when connected to any NFS or iSCSI device. That way you could use, XSIBackup's bundled Rsync and XSIDiff and store the backups to your RAID.
Depending on your connectivity possibilities in regards to the mentioned RAID, your options would range from adding virtual disks to your Linux VM and host them in the RAID, to connecting a minimal VM to your RAID as a raw device and use it just as a passthrough device.
Maybe some other process is holding the .vmdk file, running
lsof /path/to/your-flat.vmdkWould offer information about the process accessing the file.
We are trying to replicate this issue, but will remove that check, or at least offer an argument to deactivate it if we can't find a permanent solution valid for all ESXi builds.
That 50M check is mainly useful with Linux disks, which will host the superblock in the first sectors. That check allows to know if the superblock has changed, and thus the contents of the disk. A positive check does not ensure disks are exact mirrors, but combined with the CID check offers a more than reasonable insight.
Yes, you can remove that line (and any other related one) if you are confident you want to remove that disk. I would sanitize the .vmx file prior to posing any other later action.
A corrupt configuration file can cause all sort of unexpected results, not only from part of XSIBackup, but from any other program accessing the VM like the vSphere GUI, vCenter, vCenter Converter, etc...
For a VM to be susceptible of a OneDiff backup, the VM has to be running on top of a snapshot. As it's only one snapshot, performance doesn't degrade perceptibly. A OneDiff mirror (_XSIBAK) cannot be OneDiff'ed, if you do, you will modify the .vmdk disks CID and the following primary OneDiff cicle will detect this condition and reinitialize the mirror, causing the VM to be copied entirely each time.
What we recommend is backing up using OneDiff and then archive the VM by using [b][https://33hops.com/xsitools-vmfs-deduplication.html]XSITools[/url][/b] to a de-duplicated repository. As the VMs will be stored in 50M chunks by default, if you then use Rsync to mirror the repository somewhere else, it will be blazing fast, if you use size + modified date comparison.
Dwonload v 10.0.3 to use OneDiff, as it contains some bug fixes.
Run this and note the output
cat /vmfs/volumes/datastore1/BKPMAN/BKPMAN.vmx | grep ".vmdk"XSIBackup just collects the information present in the configuration files
Yes, XSIBackup appends the VMName to the path, how come not?, there has to be some minimum set of rules.
You have two choices:
1 - Assume the default behaviour: [b]datastore folder/configurable date mask/VM Name[/b]
2 - Modify the program to pledge to your particular needs, it should be fairly easy to change this.
I personally don't think it's worth to go your own way, unless there are so strong and unavoidable reasons to do it, that it turns out to be a matter of do or die. The reason is that if you do, you will have to re-apply your customization every time you upgrade to a new version.
It works for us, it's a pretty simple script. Have you enabled DNS in your ESXi server?.
If you can't fix your network issue, just download and install manually, it won't take you more than a couple of minutes
[url]https://33hops.com/xsibackup-tutorial-1.html[/url]
Remove the [b][https://33hops.com/xsibackup-help-man-page.html#datedir]--date-dir[/url][/b] argument, so that XSIBackup does not create a timestamped subfolder and use something similar to
--backup-point="/vmfs/volumes/MacServer/XSIBackupVMS/$( date +'%m-%d-%Y-%H-%M')"I would use an ISO timestamp to make sure the folders are ordered alphabetically.
--backup-point="/vmfs/volumes/MacServer/XSIBackupVMS/$( date +'%Y-%m-%dT%H:%M:%S')"Still waiting to know what is that you want to accomplish and you have not yet been able to.
Yes, XSIDiff, which is a binary utility, and from the scope of the XSIBackup framework will just substitute Rsync in the first stage of an IP backup, will work the same with a turned off VM.
You have other choice, which is to use a [b][https://33hops.com/xsibackup-help-man-page.html#backuphow]warm[/url][/b] backup. This is just a cold backup, but the VM is turned on right after taking the snapshot.
Jayce, I would not set a 500M check, as you will probably cause a "Cannot allocate memory" error, or at least you will multiply the chances that it happens.
This could be caused by a broken dd then. That would not surprise me at all, there are lots of bugs in different components of busybox in different ESXi versions.
We can fix this with a dd binary compiled for ESXi, but first I would like to locate the faulty dd, in case this is the problem. Can you all provide the ESXi version + build in which you have experienced this problem?.
UPDATE:
If a dd issue is confirmed we can provide a fix for those of you using the affected build number, it basically consists in a custom dd binary to be copied to the xsi-dir/bin directory plus a OneDiff module update.
Some older or more basic dd versions may not accept the [b]50M[/b] syntax and bs=52428800 should be used instead, we'll simplify our code to take just an integer byte count, just to prevent this hipotetical situation.
Mmm, we'll review that check, it is very rare that the first 50M block is empty
[url=https://en.wikipedia.org/wiki/GUID_Partition_Table]https://en.wikipedia.org/wiki/GUID_Partition_Table[/url]
In any case, in the end it is up to VMWare and Microsoft to decide whether they allow empty space at the beginning of a physical disk. I'm not surprised that MS installer would generate such GPT partition, who else would?
We'll make thorough tests and will remove that check for MS OSs if, as it looks, they behave in such way.
Thank you Bernhard.
Yes, that is a very convenient solution that allows to set an [b]xsidefaultpath[/b] variable per host, per subdomain or any other scriptable pattern. We offer this solution, and many other advanced customizations to our enterprise users.
If you use the .inc source file approach, your customization will work when you upgrade to any new version and you won't need to modify the xsibackup source each time.
Great!. ESXi is far from perfection. It sometimes fails at performing the simplest things.
The downloads dir should be created automatically.
We'll improve this piece of code to double check this fact.