VMware EVC

VM can be vMotioned to new generation hardware, and vMotion back to its original host, as long as the VM has not been powered off on the destination host with new hardware.

When VM is powered off and powered back on new host, the new host EVC mode setting will taken into effect. If VM hardware version is too old , there could be kernel panic, at least for Linux OS. Guest OS might prompt user interaction in VM console to confirm regular boot process, depending on vendor, then guest OS will start successfully. However after this power-back process, VM could not vMotion back to its original host anymore, since EVC setting is working for the VM. EVC compatibility prevents reverse process.

ansible-doc vmware.vmware_rest.vcenter_ovf_libraryitem

Under - deployment_spec section, Ansible-doc does not provide enough detail on `additional_parameters'  as displayed below:

- `additional_parameters' (list): Additional OVF parameters that may be needed for the deployment. Additional OVF parameters may be required by the OVF descriptor of the OVF package in the library item. Examples of OVF parameters that can be specified through this field include, but are not limited to:

• {@link DeploymentOptionParams}
• {@link ExtraConfigParams}
• {@link IpAllocationParams}
• {@link PropertyParams}
• {@link ScaleOutParams}
• {@link VcenterExtensionParams}

This causes vwmare_rest API call to deploy third-party OVF from content library very difficult to fill in all necessary properties in correct syntax.

Link below contains the class detail, but not syntax format. (I understand that a programmer will be able to figure it out)

https://vdc-repo.vmware.com/vmwb-repository/dcr-public/1cd28284-3b72-4885-9e31-d1c6d9e26686/71ef7304-a6c9-43b3-a3cd-868b2c236c81/doc/operations/com/vmware/vcenter/ovf/library_item.deploy-operation.html

https://vdc-repo.vmware.com/vmwb-repository/dcr-public/1cd28284-3b72-4885-9e31-d1c6d9e26686/71ef7304-a6c9-43b3-a3cd-868b2c236c81/doc/structures/com/vmware/vcenter/ovf/property-structure.html

https://vdc-download.vmware.com/vmwb-repository/dcr-public/4f867b1e-c524-4e4b-b79e-4321822b5c70/a3cd39a7-4eaf-4e10-9ad2-589f9dfe1f9a/doc/Com/Vmware/Vcenter/Ovf.html

Link below provide a quick way to get json format via vCenter API explorer. 

https://stackoverflow.com/questions/47912744/propertyparams-when-deploying-vm-from-ovf

Just convert it into yaml format for readability, then use it in Ansible playbook properly.

PS: deploy ovf from content library is much faster than from local ova/ovf file, if you need to deploy it into multiple VM instances, and one time upload is worthy.

Ansible "module_stderr": "env: python: No such file or directory\n"

Recently I got the following error while running existing Ansible playbook, which run successfully before :

fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "env: python: No such file or directory\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 127}

Ansible complains about python not found. I am using Homebrew python.

which python

python: aliased to python3

which python3

/opt/homebrew/bin/python3

stackoverflow option 2 works for me
https://stackoverflow.com/questions/3655306/ubuntu-usr-bin-env-python-no-such-file-or-directory

Just create symbolic link for python, required by Ansible:

ln -s /opt/homebrew/bin/python3 /opt/homebrew/bin/python

Sync Office 365 Calendar to Pixel 6 Calendar app

In case your company email policy blocks Pixel 6 native Calendar app to access Office 360 email/calendar, You can do so via the help from Outlook email app from Microsoft.

In Outlook app, Office 365 account settings has one Option: Sync calendars. Turn it on, and Office 365  calendar will show up on your Pixel 6 Calendar, so you have one place to check all your schedules on Pixel 6.

Sometimes, Outlook updates might overwrite this setting, so just go back and enable it again.

Yes, you can publish your Office 365 calendar as URL, then access it via Google calendar from web browser, but currently, native Calendar app on Pixel can't not subscribe published Office 365 calendar yet.

Chrome browser password export/import

We all know how to import/export bookmarks in web browser, while it comes time that we will also need import passwords saved in web browser. Google Chrome provide nice feature to do so.

Credit to https://www.howtogeek.com/715223/how-to-export-and-delete-saved-passwords-in-chrome/, and https://www.alphr.com/import-passwords-google-chrome-using-csv-file/

chrome://settings/passwords has function to export password into CSV files.

For import, you will need to enable password import feature first: chrome://flags, then type in “password import” into the search bar to enable password import. 

Import function will display above Export Password, after Chrom is relaunched.

This is very handy when you have a new computer to set up.

Samsung Galaxy Tablet 10.1 GT-N8013 on Android 11 with LineageOS 18.1 (ALPHA/BETA)

Flash (firmware update) on 8 years old original Samsung Galaxy Tablet 10.1 GT-N8013 with latest Android 11 is possible now. theupgradeguide.com provided detailed steps and screenshots nicely to follow.

html6405 from xda-developers provides all the Downloads for : Samsung Galaxy Note 10.1 Wifi. Make sure that the latest TWRP recovery file and magisk_patched_n8010.img file are download from same place, instead of img file from twrp official site. This also applies to LineageOS and Open GApps zip files. Official sites might not provide support for GT-N8013 anymore.

TWRP image replace recovery.img file on GT-N8013, which is done by Odin or Heimdall (Jodin is Java wrapper around Heimdall) via Odin mode, aka bootloader mode. TWRP img is used in recovery mode: power + volume-up (release buttons after Samsung screen is displayed; volume-up key is on the left to SD card slot)

MagiskSU from topjohnwu replaces SuperSU for rooting recent version of Android devices. Magisk Manage app is included in Lineage OS, which will be used for creating magisk_patched_*****.img in place, so you can rename it as boot.img and flash boot partition in Odin mode (or just select "Direct Install" method, which will flash boot image in place.). After reboot, GT-N8013 is rooted, and passes SafetyNet Attestation checking via Magisk.

LineageOS 18.1 on GT-N8013 from htm6405: lineage-18.1-20210114-HTML6405 seems to have the common issue on LineageOS with charging over a hour, which might freeze and could not resume from standby screen. Hopefully this will be resolved by next release.

LineageOS boot up process is a little slow (initial boot is even slower), compared to original Samsung boot. I did not find an option to fast boot it yet.

After flashing GT-N8013 with LineageOS 18.1, Android display device info as GT-N8010, I understand the img file I used did  have subfix with n8010. It is fine.

Android (Trust System security) complains that This build was signed with public key, and SELinus is not enforcing, you system has been weakened, so I had to disable the trust, not the best practice really.

Also credit to Sonny Kurniawan on Blogger, who provided details on flashing Android Nougat. Yes, my stocked GT-N8013 has been collecting dust all these years. 😅

Always xda link for more and latest info.

As to OEM bootloader unlock, JJEgan from xda-developers says GT-N8013 is unlocked. I enabled Developer debug mode in original ROM (Android 4.1.2), and I don't recall there is option for bootloader unlock either. 

I used Heimdall on OS X Mojave to flash TWRP and magisk patched.img.

Update: I flashed GT-N8013 with LineageOS 18.1 BETA from html6045 with same version of TWRP as ALPHA release last December without any issue.

Note: Magisk is not installed on BETA release.

Tip: LineageOS might not include some language input method, but Gboard app from Google Playstore is pretty good, which detects extra language selected, and download most popular input method in the background.

Heimdall on OS X High Sierra/ Mojave

Samsung Galaxy devices use Odin to flash firmware, however Odin is only available on Windows unfortunately.

Heimdall is a cross-platform open-source tool suite used to flash firmware (aka ROMs) onto Samsung Galaxy devices from Benjamin Dobell, an alternative for Odin on Linux/Mac.

Packaged Heimdall binary from link doesn't work on High Sierra or latter, so you will have to compile source code from gitlab. Heimdall frontend requires Xcode 9.4 above.

bfabiszewski @bfabiszewski has merge request: Fix build on macOS High Sierra for Heimdall, which replaces three CMakeLists.txt files from source. Without these fixes, cmake step from Benjamin Dobell will fail.

I followed fix from bfabiszewski on OS X  Mojave and built Heimdall binary successfully.

It seems that best practice is to run Heimdall with sudo, and also for heimdall front end. One caveat or feature I noticed is that Heimdall tends to release kernel drive after some operations, which might cause next task failed with initialization error. If this happen, just disconnect/reconnect usb cable, and run heimdall detect to make sure Heimdall can detect Samsung Galaxy devices.

Install pip on Python3

https://stackoverflow.com/questions/31273157/how-to-install-pip-on-compiled-from-source-python

Custom build Python might not come with pip, so you can rebuilt with the following options:

./configure --with-openssl=/usr/local/opt/openssl --with-ensurepip=install

pip need Python ssl module, which in turn need openssl.

Or,  python get-pip.py will also install pip. again, assuming Python is build with ssl module.

Download get-pip.py via:

curl -sSL https://bootstrap.pypa.io/get-pip.py -o get-pip.py

The ensurepip package provides support for bootstrapping the pip installer into an existing Python installation or virtual environment. This bootstrapping approach reflects the fact that pip is an independent project with its own release cycle, and the latest available stable version is bundled with maintenance and feature releases of the CPython reference interpreter.

In most cases, end users of Python shouldn’t need to invoke this module directly (as pip should be bootstrapped by default), but it may be needed if installing pip was skipped when installing Python (or when creating a virtual environment) or after explicitly uninstalling pip.

python -m ensurepip

This invocation will install pip if it is not already installed, but otherwise does nothing. To ensure the installed version of pip is at least as recent as the one available in ensurepip, pass the --upgrade option:

python -m ensurepip --upgrade

By default, pip is installed into the current virtual environment (if one is active) or into the system site packages (if there is no active virtual environment).

Setting up GDB for Debugging Python 3.9.1 on OS X Mojave

https://unconj.ca/blog/setting-up-gdb-for-debugging-python-on-os-x.html

brew install gdb and codesign gdb (updated procedure from: https://sourceware.org/gdb/wiki/PermissionsDarwin)

Brew installed python3 is already in thin format, no need to use lipo to extract, although you can still use lipo to check python executable format. (Non-fat file: python3 is architecture: x86_64)

Providing python3 interpreter (not symbolic link) with path if necessary, after gdb , and this will solve error message like: "0x7ffeefbff800s": not in executable format: file format not recognized

Had to compile Python3 from source to get rid of the following warning: ( ./configure --with-pydebug). Uncommenting SSL related lines in Modules/Setup file is to load ssl module in Python

Reading symbols from python...
(No debugging symbols found in python)

Also build gdb from source code again (./configure --with-python=/usr/local/bin/python3 --disable-werror), as brew info gdb failed with Brew installed python@3.9. Codesign command again for gdb.

Python gdp support: https://devguide.python.org/gdb/

 ~/.gdbinit with content below:

set startup-with-shell off

add-auto-load-safe-path /usr/local/bin

Jenkins ssh-agent plug in

https://support.cloudbees.com/hc/en-us/articles/222838288-SSH-Credentials-Management-with-Jenkins

https://support.cloudbees.com/hc/en-us/articles/222978868-How-to-Connect-to-Remote-SSH-Agents-#:~:text=In%20Jenkins%3A&text=Choose%20SSH%20Username%20with%20private,useful%20Description%20for%20this%20credential.

Jenkins ssh-agent only takes user and private key as credentials for build jobs, not user and password, otherwise you will get error below:

FATAL: [ssh-agent] Could not find specified credentials

[ssh-agent] Looking for ssh-agent implementation... 

[ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine)

.....

Permission denied, please try again. 

Permission denied (publickey,password).

Target remote server must have completed setting for ssh login via private key before. (public key in ~/.ssh/authorized_keys)

NFS Server Provisioner on K8S

Another dynamic storage provisioner from Quay. NFS server inside K8S for bare-metal/on-premise kubernetes cluster.

https://github.com/kubernetes-incubator/external-storage/tree/master/nfs

nfs-common and nfs-utils need to be installed on kubelet, otherwise error log might include: 

• bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program.
• Warning  FailedMount  24m  kubelet, MountVolume.SetUp failed for volume "pvc-2c49da00-3431-4e3c-a1cc-84d1daa4a6ab" : mount failed: exit status 32

In case there is error: pod has unbound immediate PersistentVolumeClaims, it has nothing with storage provision, which just indicates that pod does not not have storage class defined in manifest file or K8S doesn't have default storage class.

To set up default storage class in kubernetes:

• kubectl patch storageclass storage-class-name -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Jenkins on K8S

Jenkins can deploy via Rancher catalog or Helm chart.

Jenkins K8S deployment will need persistent storage. Jenkins service can be exposed or access via ingress rules.

Jenkins master -> Credentials -> add Credentials -> kind: Kubernetes Service Account

Manage Jenkins -> Manager Nodes and Cloud -> Configure Cloud (for new version of Jenkins)

• Test Connection 
• Pod Retention: Never (Jenkins slave pod will be terminated after build job completes)
• Slave pod will be placed into jenkins namespace by default

Jenkins slave on K8S

Use K8S as Jenkins slave/agent via Jenkins kubernetes plugin. (continuous integration)

• From Jenkins master, install Kubernetes plugin (This plugin integrates Jenkins with Kubernetes)
• Credentials -> Add Credentials -> Kind: Secret file (kubeconfig file from K8S)
• Manage Jenkins -> Manage Nodes and Cloud -> Configure Clouds (new version of Jenkins) 

1. Credentials: select Secret file name just created from drop-down menu, and Test Connection
2. Jenkins URL and tunnel match Jenkins master. No need https:// for tunnel
3. Pod label will be used for slave pod label in K8S
4. Pod Template: name will be prefix for slave pod name prefix in K8S; blank namespace will create slave pod in default namespace in K8S;
5. Pod Template -> Labels is critical, which help Jenkins master decides which builder will be used for the build job. 
6. Pod Template: Usage: only builds job with label expression matching this label
7. Define container template details: name: jnlp; docker image: jenkins/jnlp-slave:latest; working directory: /home/jenkins/; also environment variable value pair: {JENKINS_URL: http://jenkins-master:8080} (or adding other agent as Pod Template)

When creating a new build job, check Restrict where this project can be run, enter Label expression: Pod template label in previous step 5, so Jenkins master will use K8S pod to execute build task.

Jenkins slave started in K8S, then terminated in K8S after build job completes with proper pod retention setting in Pod Template.

Use K8S as Jenkins slave via Jenkin Kubernetes Continuous Deploy Plugin (continuous deployment on kubernetes)

• From Jenkins master, install Kubernetes Continuous Deploy Plugin
• Credentials -> Add Credentials -> Kind: Kubernetes Configuration (KubeConfig); directly copy content of  kubeconfig file into textext. The ID of this Jenkins credential will match value of  kubeconfigId in file: Jenkinsfile, which is in source code Repositories, such as github, or private repository
• If Kubernetes plugin is already configured, no more configuration is required.

When creating a new pipeline in Jenkins, in Pipeline section: select Pipeline script from SCM, then select proper SCM, repository and branch. Default script path is: Jenkinsfile

Jenkins pipeline checkout source from SCM, and start slave pod on K8S to build images and upload container images into registry such Docker hub or any private/public registry defined in Jenkinsfile from SCM. Then slave pod on K8S is terminated with the completion of Continuous Integration/Delivery process. Finally, pipeline script will deploy app Pod via manifest in Jenkinsfile, always pulling container images from registry.

Delete Terminating hanging namespace

clean up content in finalizer section via json/yaml file or directly edit namespace via kubectl

Methold below always works
Assuming local is terminating...
kubectl get namespace local -o json > local.json         
kubectl get namespace local -o json \
            | tr -d "\n" | sed "s/\"finalizers\": \[[^]]\+\]/\"finalizers\": []/" \
            | kubectl replace --raw /api/v1/namespaces/local/finalize -f -     

Rancher on baremetal k8S

• cert-manager on K8S (validate cert-manager is up correctly!, otherwise rancher installation might fail with error: x509: certificate signed by unknown authority)
• ingress-nginx on baremetal (you might need edit NodePort option with your load balancer)
• Install helm3
• helm install rancher rancher-latest/rancher \

  --namespace cattle-system \
  --set hostname=rancher.my.org ###--dry-run first would be good idea.

• kubectl -n cattle-system rollout status deploy/rancher ###verify status
• successfully installation of Rancher in namespace: cattle-system, will also create extra namespaces: local, p-xxxxx, p-yyyyy, user-zzzzz, cattle-global-nt, and cattle-global-data. local is used for cluster hosting Racher; user-zzzzz is for user authentication. Not sure for the function of the rest yet. I would guess global is for cross cluster functionalities.

tdnf install -y kubelet kubeadm kubectl --nogpgcheck

tdnf install -y kubelet kubeadm kubectl fails with error below:

Error processing package: Packages/548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm
Error(1508) : GpgKey Url schemes other than file are not supported

tdnf install -y kubelet kubeadm kubectl --nogpgcheck ####disable gpg check as workaround

credit to: https://unix.stackexchange.com/questions/207907/how-to-fix-gpg-key-retrieval-failed-errno-14


cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

Ingress-nginx on ARMv7 Raspberry Pi 3 Model B

https://github.com/kubernetes/ingress-nginx

Note after Installation Guide:

1. User Metallb as load balancer on bare metal for K8S on Raspberry 3.0B
2. In the mandatory.yaml file, change container image into:quay.io/kubernetes-ingress-controller/nginx-ingress-controller-arm:0.30.0 (thanks alexellis on github)
3. In the cloud-generic.yaml remove line:externalTrafficPolicy: Local. Kubernetes default externalTrafficPolicy is: Cluster. (It matters if Weave Net is used for networking)
4. kubectl create -f mandatory.yaml -f cloud-generic.yaml (one liner, so nginx-ingress-controller would not complain non-existence of service:ingress-nginx, which gets an IP address assigned from Metallb.
5. check logs inside nginx-ingress-controller for readiness or other errors if any.

I tried Helm 3 without luck. It seems there is repo issue.

Open source load testing tool

https://k6.io/blog/comparing-best-open-source-load-testing-tools

Performance testing is a type of testing for determining the speed of a computer, network or device. It checks the performance of the components of a system by passing different parameters in different load scenarios.

Load testing is the process that simulates actual user load on any application or website. It checks how the application behaves during normal and high loads. This type of testing is applied when a development project nears to its completion.

Stress testing is a type of testing that determines the stability and robustness of the system. It is a non-functional testing technique. This testing technique uses auto-generated simulation model that checks all the hypothetical scenarios.

Conclusion:

• Performance testing is a testing method used to determine the speed of a computer, network or devices.
• Load testing simulates real-world load on any application or website.
• Stress testing determines the stability and robustness of the system
• Performance testing helps to check the performance of website servers, databases, networks.
• Load testing is used for the Client/Server, Web-based applications.
• Stress testing is done unexpected test traffic of your website.

https://www.guru99.com/performance-vs-load-vs-stress-testing.html

NFS share on OS X Sierra Version 10.12.6 (16G2128) MacBook Pro (15-inch, 2017)

https://support.apple.com/en-us/HT202243

Create NFS share on OS X for NFS client connection

• mkdir <path to NFS share>
• chown -R nobody:nobody <path to NFS share>
• sudo nano /etc/exports, and add line:

<absolute path to NFS share> -maproot=nobody --alldirs ##allow client to mount at any point within NFS file system

• sudo chmod 640 /etc/exports
• nfsd status (if not running: nfsd enable && nfsd start)
• showmount -e

Mount NFS share from OSX command line:

• mkdir /mnt
• sudo mount -o hard,nolock <NFS share path> /mnt
• mount | grep nfs
• ls /mnt

Kubernetes Metrics Server on Raspberry Pi 3 Model B (ARM v7)

Metrics Server exposes core Kubernetes metrics via metrics API. Without Metrics Server,Horizontal Pod Autoscale (HPA) and kubectl top command will not work.

Github did not provide deployment yaml file for Raspberry Pi cluster, so I have to change original deploy/kubernetes/metrics-server-deployment.yaml

1. replace amd64 with arm (under containers and nodeSelectors section)

2. modify args into: (two extra lines)

        args:

          - --cert-dir=/tmp

          - --secure-port=4443

          - --kubelet-insecure-tls ###get ride error: http: TLS handshake error

          - --kubelet-preferred-address-types=InternalIP ### unable to fully scrape metrics from source kubelet

3. deploy Metrics Server and all the rest manifest yaml files in the same directory , and wait for a while to get rid off error:unable to fetch node metrics for node