Hardware (VMware supported)
1 CPU- CPU compatibility for vMotion and Fault Tolerance.
- Hardware-Assisted CPU and MMU virtualization
- No Ethernet bottle neck for iSCSI and NFS
- Multipathing for writing large amount of data to strorage, and should not share Eherenet link with swith other systems
- No over-subscription in storage network
- Separate VLAN for iSCSI/NFS networking
- Assign separate storage processor for separate system with heavy I/O loads
- load-balancing I/O to all HBA and storage processors
- Configure maximum queue depth for HBA
- NFS and software iSCSI will consume some CPU resource from hosts
- Storage vMotion requires sufficient available storage bandwidth.
- Server class NIC without bottle neck from end to end
- Checksum offload, TCP Segmentation Offload (TSO), 64-bit DMA, Jumbo Frame, multiple Scatter Gather elements per Tx frame
- 10G NIC for NetQueue, receiving performance increase
- NIC teaming
- Disable power saving settings eg. CIE for halt state,
- enable all populated socket and enable all cores in each socket
- Enable hyper-threading
- Disable node interleaving
- Enable hardware virtualization features for CPU and MMU (VT-x, AMD-V, EPT, RVI)
- Disable any unneeded devices from BIOS, such as USB, and Serial ports.
ESX and Virtual Machines
1 ESX General Considerations
- Enough resource for service console, ESX only
- Provide only as much as required resource for each VM
- Disable all unused physical hardware from BIOS to free up interrupt resource and memory too
- Unused virtual hardware should be removed to save resource too.
- User new virtual hardware Version 7 for VM
- For a small percentage of CPU-bound workloads, there will be noticeable degradation in both throughput and latency.
- VMware doesn't offer CPU over-commitment
- Service console CPU reservation
- esxtop
- user multiple vCPU only for multi-threaded applications, otherwise downgrade performance.
- Hyper-Threading: be careful with using CPU affinity
- NUMA effective with at least two CPU core per NUMA node, two node minimum; when VMs with more vCPUs than the number of cores at a NUMA node, it won't benefit from NUMA scheduler any more.
- Hardware CPU/MMU virtualization is preferred, or just Automatic for VM
- Memory overhead: host (console and kernel), VM overhead
- Memory sizing: avoid over-allocationg memory to reduce VM memory overhead.
- 32bit Linux should not have more than 896MB memory to reduce performance.
- Memory overcommit (TPS, Ballooning, Swapping), host-level swapping should be avoid though
- Large page for Guest OS performance, not for TPS anymore.
- Hardware MMU virtualization
- VMFS file are preferred, though consume little more ESX resource than RDM
- Independent persistent disk mode has best performance, while nonpersistent and snapshot have performance penalty with right use case.
- Partition alignment recommendation from array vendor
- Multiple VMFS volume on same LUN or multiple VM concurrently access same VMFS volume could decrease storage performance.
- Medadata intensive operations can impact VM I/O performance. (Backup, clone, file permission; cron job to open/close files; thin disk)
- Queue depath
- Correct PSP for storage array: A/A-> Fixed; A/P->MRU
- Efficient CPU resource available on the network throughput of virtualized application
- Separated vSwitches for VM, service console and VM kerenel
- VM at same host should be on same vSwitch to reduce CPU and traffic overhead
Guest OS
1. Guest OS General Consideration- VMware tools might not able available for unsupported OS
- Disable Screen Saver and Window animations.Disable X server in Linux if not used
- Schedule backup and anti-virus at off-peak hours, and not run simultaneously at same host. Distribute the task across time.
- Use either NTP or VMware tools for time synchronization, not both
- Paravirtualization is depreciated now.
- Correct time interrupt rate for different guest OS
- Custom storage adapter inside VMware Tools package
- Queue depth
- Guest storage drive to eliminate large I/O request splitting
- guest partition alignment from array vendor
- Use new VMXNET3 whenever possible
- If not, use flexible
- TSO with supported OS
- Increase receive buffers in the receiver network device, with host CPU workload to gain VM networking performance.
Virtual Infrastructure Management
1. General Resource Management- Shares are more flexible for resource management
- Use Resource Pools for delegated resource management
- vApp is better for a multi-tier service VM group, compared with regular resource pool.
- Configuration maximum for vCenter
- Disconnect vSphere client from vCenter when not used
- Avoid aggressive alarm settings.
- Separate VUM from vCenter if possible
- Separate Convertor from vCenter
- Windows 2008 64bit
- Management Web Service for Link Mode and Search. If not using these, disable the service
- Enough resource for database
- Log file on separated disk
- Reindex database for performance
- Appropriate logging level for business
- vMotion compatibility
- Storage vMotion with storage bandwidth and at off-peak hour.
- EVC mode for DRS
- Powered off unused VM could help DRS
- DRS level
- DRS affinity only when required
- Resource pool setting could affect DRS
- WOL is preferred
- HA also affect DPM
- DPM prefers automatic mode, or disable on specific host in the cluster
- FT traffic is asymmetric, so place the primary FT VM on multiple host.
- Not more than 4 FT VM on one host
- CPU reservation for the primary FT VM
- Power management scheme are consistent between hosts
- Reduce timer interrupt rate at the guest OS
No comments:
Post a Comment