Sunday, 4 September 2016

VMware More about Log files and Basic Settings with Vsphere


ESXi Host Log Files Path Location:
/var/log/auth.log store ESXi Shell authentication success and failure.
/var/log/dhclient.log store DHCP client logs, including discovery, address
/var/log/esxupdate.log store ESXi patches and update installation logs.
/var/log/hostd.log store Host management service logs, including virtual machine and host tasks and events, communication with the vSphere Client and vCenter Server vpxa agent, and SDK connections.
/var/log/shell.lo ESXi Shell usage. This log file contains a history list of all the commands that have been run at the command line on the host, including enable and disable.
/var/log/sysboot.log store VMkernel startup and module loading.
/var/log/boot.gz A compressed file that contains boot log information and can be read using zcat /var/log/boot.gz|more .
/var/log/syslog.log store Management service initialization, watch dogs, scheduled tasks, and DCUI use.
/var/log/usb.log store USB device arbitration events, such as discovery and pass-through to virtual machines.
/var/log/vob.log store VMkernel observation events, similar to vob.component.event .
/var/log/vmkernel.log store Core VMkernel logs, including device discovery, storage and networking device and driver events, and virtual machine startup.
/var/log/vmkwarning.log store VMkernel warning and alert log messages.
/var/log/vmksummary.log save A summary of ESXi host startup and shutdown and an hourly heartbeat with uptime, number of virtual machines running, and service resource consumption.

vCenter Server Log Files:
vpxd.log The main Server log file that communicates with the vCenter Server Agent (vpxa), which is located on connected ESXi hosts. This log file is useful for troubleshooting configuration and operational errors.
vpxd- This log file is useful for troubleshooting performance issues.
\drmdump\clusternnn\ DRS actions, grouped by the DRS cluster. The log files are gzipped.

Admins can see the ESXi log files using the command line by logging in ESXi Shell and running commands like cat , tail , more , less , or any command that allows you to view a file. The tail –f <filename> is a commonly used command-line command that helps to see changes being added to the log file as they happen. By this command in the background, which keeps the log file open. The other friendly option to view log files by using the vSphere Client and login to the ESXi host. The vCenter Server log files can be viewed directly on the Windows host system or by using the vSphere Client.
One of the changes that was made in vSphere version 5 logging was to separate all the log files to diagnose and troubleshoot, which will help to identify the issue and solution.
Here writing a short description about files that are responsible when a vm creation happen
The .nvram file : This small file contains the BIOS that is used when the VM boots, changes made to the hardware configuration for the VM are saved in the NVRAM file. The very great point that if file is deleted it will be automatically re-created when a VM is powered on.

The .vmx file. This file contains all of the configuration information with hardware settings. The setting which are made as part of that information is stored in text format in this file. This file can contain a variety of information about the VM i.e., RAM size, network interface card info, hard drive info and serial/parallel port info, advanced power and resource settings, VMware tools options and power management options.

VMDK files. store the contents of the virtual machine hard disk drive. These files are stored in the same directory as the .vmx file. A virtual disk is made up of one or more virtual disk files.

flat.vmdk file : Default and large virtual disk data file that is created when you add a virtual hard drive to your VM that is not an RDM.
Thick disks: this file will be approximately the same size as what you specify when you create your virtual hard drive.

delta.vmdk file : this file is create when you take snapshot of a vm. All writes to the original -flat.vmdk are halted and it becomes read-only; changes to the virtual disk are then written to these -delta files instead. The initial size of these files is 16 MB and they are grown as needed in 16 MB increments as changes are made to the VM's virtual hard disk. Because these files are a bitmap of the changes made to a virtual disk, a single -delta.vmdk file cannot exceed the size of the original -flat.vmdk file. A delta file will be created when you take snapshot and file names will be incremented numerically (i.e., vm-000001-delta.vmdk, vm-000002-delta.vmdk). These files are automatically deleted after they are merged in to the vmdk file.

The -rdm.vmdk file:This is the mapping file for the raw device mapping (RDM) format that manages mapping data for the RDM device. The disk file is presented to the ESX host as an ordinary disk file.The storage virtualization layer presents the mapped device as a virtual SCSI device.

The .vswp file : Memory swap file is created that can be used in lieu of physical host memory if ESX host exhausts all its physical memory because due to overcommitted. These files are created equal in size to the amount of memory assigned to a VM, minus any memory reservations (default is 0) that a VM may have set on it (i.e., a 4 GB VM with a 1 GB reservation will have a 3 GB VSWP file created). These files are always created for virtual machines but only used if a host exhausts all of its physical memory. In this period of time VMs can have performance issue. These files can take up quite a large amount of disk space on your VMFS volumes, so ensure that you have adequate space available for them, as a VM will not power on if there is not enough room to create this file. These files are deleted when a VM is powered off or suspended. Virtual machines will lock the .vswp, -flat.vmdk and -delta.vmdk, .vmx and .log files during time.

The .vmss file. This file is used when virtual machines are suspended state. Size of this will is approximately the same size as the amount of RAM that is assigned to a VM. When a VM is brought out of a suspend state, the contents of this file are written back into the physical memory of a host server, however the file is not automatically deleted until a VM is powered off (an OS reboot won't work). If a previous suspend file exists when a VM is suspended again, this file is re-used instead of deleted and re-created. If this file is deleted while the VM is suspended, then the VM will start normally and not from a suspended state. This is very helpful when you have to find the root cause of the OS level issue, you can convert this in to memory dump and diagnose the issue.

The .vmsd file. This file is used with snapshots to store metadata and other information about each snapshot that is active on a VM. This text file is initially 0 bytes in starting. It updated the with information every time snapshots are created or deleted. It show display name and description, and the UID of the snapshot. Once snapshots are deleted, this file retains old snapshot information but increments the snapshot UID to be used with new snapshots. It also renames the first snapshot to "Consolidate Helper," presumably to be used with consolidated backups.

The .vmsn file. This file is used with snapshots to store the state of a virtual machine when a snapshot is taken. A separate .vmsn file is created for every snapshot that is created on a VM and is automatically deleted when the snapshot is deleted. The size of this file will vary based on whether or not you choose to include the VM's memory state with your snapshot. If you do choose to store the memory state, this file will be slightly larger than the amount of RAM that has been assigned to the VM, as the entire memory contents, including empty memory, is copied to this file. If you do not choose to store the memory state of the snapshot then this file will be fairly small (under 32 KB). This file is similar in nature to the .vmss that is used when VMs are suspended.

The .log file. LOG files are created to log information about the virtual machine and are often used for troubleshooting purposes. There will be a number of these files present in a VM's directory. The current log file is always named vmware.log and up to six older log files will also be retained with a number at the end of their names (i.e., vmware-2.log). A new log file is created either when a VM is powered off and back on or if the log file reaches the maximum defined size limit. The amount of log files that are retained and the maximum size limits are both defined as VM advanced configuration parameters (log.rotateSize and log.keepOld).

After VCenter Installation, can see in service console that vm related services are there with manual, automatic and automatic (delayed start) status.


After login to VCenter first time, Admins should add the group or individuals domain user names that will be responsible to manage the VMware environment. From here you can give different level of access. Like Administrator Role, Read Only, and that are need to be done from Permission tab of VCenter. By default administrator@vsphere.local would be authorised to login from vsphere client. I gave Administrators, Administrator as Admin access.



There are 9 different access roles which you can use and assign permission as per access level.
There are other configuration also require on VCenter level.

From this tab only HA and DRS feature can be on and off on the vm cluster 


Try to keep virtual machine sizing requirements similar across all configured virtual machines. The Host Failures Cluster Tolerates admission control policy uses slot sizes to calculate the amount of capacity needed to reserve for each virtual machine. The slot size is based on the largest reserved memory and CPU needed for any virtual machine. When you mix virtual machines of different CPU and memory requirements, the slot size calculation defaults to the largest possible, which limits consolidation.

Admission control is used to ensure that sufficient resources are available in a cluster to provide failover protection and to ensure that virtual machine resource reservations are respected. There are three Admission control policies:
1. Specify Failover Hosts Admission Control Policy
2. Percentage of Cluster Resources Reserved Admission Control Policy
3. Host Failures Cluster Tolerates Admission Control Policy

The following recommendations are best practices for vSphere HA admission control:
  1. Select the Percentage of Cluster Resources Reserved admission control policy. This policy offers the most flexibility in terms of host and virtual machine sizing. When configuring this policy, choose a percentage for CPU and memory that reflects the number of host failures you want to support. For example, if you want vSphere HA to set aside resources for two host failures and have ten hosts of equal capacity in the cluster, then specify 20% (2/10).
  1. Ensure that you size all cluster hosts equally. For the Host Failures Cluster Tolerates policy, an unbalanced cluster results in excess capacity being reserved to handle failures because vSphere HA reserves capacity for the largest hosts. For the Percentage of Cluster Resources Policy, an unbalanced cluster requires that you specify larger percentages than would otherwise be necessary to reserve enough capacity for the anticipated number of host failures.
  1. If you plan to use the Host Failures Cluster Tolerates policy, try to keep virtual machine sizing requirements similar across all configured virtual machines. This policy uses slot sizes to calculate the amount of capacity needed to reserve for each virtual machine. The slot size is based on the largest reserved memory and CPU needed for any virtual machine. When you mix virtual machines of different CPU and memory requirements, the slot size calculation defaults to the largest possible, which limits consolidation.
  1. If you plan to use the Specify Failover Hosts policy, decide how many host failures to support and then specify this number of hosts as failover hosts. If the cluster is unbalanced, the designated failover hosts should be at least the same size as the non-failover hosts in your cluster. This ensures that there is adequate capacity in case of failure.
Its very important part of HA configuration and wil be covering in more advanced very soon.


Here you can configure restart priority and isolation process.

 Its all default settings,and will be covering in more details very soon

VMware Datastore Hearbeating provides an additional option for determining if host is in failed state or not.

vCenter automatically selects at least two datastores from the shared datastores(to enable this yu should have atleast 2 shared datastorage). It’s preferable to have VMware Datastore heartbeating selected on every storage device in vmware environment. It can be check from the properties cluster that which datastores has been selected.

This option enables to avoid false restarting of VMs in cluster in case only a management network has failed. The default number of heartbeat datastores is two, and maximum valid heartbeat is five. It can override the default value by an editing advanced attribute: das.heartbeatdsperhost.

 This option enable automation level, in example at time of high memory, cpu or storage utilization on host, so how DRS should react. In Manual option, every time admins to run DRS and then it will show recommendation. In Partially option, vcenter will recommend to migrate the vms to another or which host. In Fully Automated option vcenter will itslef do all actions which are require to balance the cluster performance.

An important thing to remember is that even with DRS Groups in place, VMware High Availability will not be inhibited by these rules. In this option you can specifiy the at the time of action which vm will be moved to which host and other criteria. its usefull in terms of Licensing, where its based on quantiy or bind with specific host for application vendor.

Here Affinity and Anti-Affintiy rules work. In circumstances where application require that any two move should move together to specific host is called Affinity Rules, and in case of Domain controller, where as per best practice it should be configured like that 2 Domain controller should not be on same host. Hope you got the point where am trying to highlight the concept.Will be configuring this option later in lab.

Level of automation in Vsphere are
1. Fully Automated
2 .Partially Automated
3. Manual
4.Default
5. Disabled

 In this option vcenter will recommend and suggest as per option. In Off Mode no recommendation, In Manual it will display the recommendation but action would require manual intervention and as word explain automatic, it will recommend and perform the action with any human or manual activity.

It shows for all host in cluster when last time power management was applied


EVC allows to migrate vm between different generation of CPUs.EVC cannot enable vMotions between AMD and Intel processors, a VM can have problems if vMotioned to a host with a differing set of CPU instructions. We will be doing more on later very soon.

 From here you can save the swapfile location for vms in cluster.

1 comment: