inVURTED.com

Sometimes a file or set of files in a VMFS become locked and any attempts to edit them or delete will give a device or resource busy error, even though the virtual machine associated with the files is not running. If the virtual machine is running then you would need to stop the virtual machine to manipulate the files. If you know that the virtual machine is stopped then you need to find the ESX server that has the files locked and then stop the process that is locking the file(s).

1. Logon to the ESX host where the virtual machine was last known to be running.
2. vmkfstools -D /vmfs/volumes/path/to/file to dump information on the file into /var/log/vmkernel
3. less /var/log/vmkernel and scroll to the bottom, you will see output like below:
   
Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)FS3: 130: < START vmware-16.log >
Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)Lock [type 10c00001 offset 30439424 v 21, hb offset 4154368Nov 29 15:49:17 vm22 vmkernel: gen 66493, mode 1, owner 46c60a7c-94813bcf-4273-0017a44c7727 mtime 8781867]
Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)Addr <4, 588, 7>, gen 20, links 1, type reg, flags 0x0, uid 0, gid 0, mode 644
Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)len 23973, nb 1 tbz 0, zla 2, bs 65536
Nov 29 15:49:17 vm22 vmkernel: 2:00:15:18.435 cpu6:1038)FS3: 132: < END vmware-16.log >

4. The owner of the lock is on the third line, the last part is all you need, in this case 0017a44c7727
5. esxcfg-info | grep -i ‘system uuid’ | awk -F ‘-’ ‘{print $NF}’ will display the system uuid of the esx server. You need to run the esxcfg-info command on each esx server in the cluster to discover the owner.
6. When you find the ESX server that matches the uuid owner, logon to that ESX server and run the command: ps -elf|grep vmname where vmname is the problem virtual machine. Example output below:
   4 S root 7570 1 0 65 -10 - 435 schedu Nov27 ? 00:00:02 /usr/lib/vmware/bin/vmkload_app /usr/lib/vmware/bin/vmware-vmx -ssched.group=host/user/pool2 -@ pipe=/tmp/vmhsdaemon-0/vmxf7fb85ef5d8b3522;vm=f7fb85ef5d8b3522 /vmfs/volumes/470e25b6-37016b37-a2b3-001b78bedd4c/iu-lsps-vstest/iu-lsps-vstest.vmx0
7. Since there is a process running, pid 7570 in the example, you need to kill it by following steps 5-12 on Stopping a Virtual Machine.
8. Once the kill is complete the files should be released.