Definite's Extractor

My findings on Life, Linux, Open Source, and so on.

Install Google-Chrome in OpenStack RHEL 7 instances

We have automated tests that require runnable Google Chrome. Yet the Google Chrome kept crashing.

The first encountered is:

libGL error: failed to load driver: swrast
libGL error: Try again with LIBGL_DEBUG=verbose for more details.

This one is easy, install mesa-dri-drivers solved this.

Then cames:] : InitializeSandbox() called with multiple threads in process gpu-process

My initial guess was SELinux, but journalctl returns nothing about it. After a few hours, I thought, how about firefox? Maybe it helps to set the SELinux and install the missing dependencies? And… Tada, both Firefox and Google Chrome worked. Eventually, I dug out that Google Chrome requires fonts to works. Specifically, liberation-fonts-common and liberation-sans-fonts.

To sum up, following command worked for me:

sudo yum -y install mesa-dri-drivers liberation-fonts-common liberation-sans-fonts

Clamav: troubleshooting of clamdscan

clamdscan is much faster to run than clamscan, however, it requires clamd which is a bit harder to setup, so I have some tips for troubleshooting:

ERROR: Could not lookup : Servname not supported for ai_socktype

Usually you should check the permission, especially whether the current user is in group clamscan (the primary group of the clamd running user).

lstat() failed: Permission denied. ERROR

This is usually because clamd running does not have the permission to run the is run as non-root user.

So you will need to enlist User clamscan (the user that runs clamd). You need to logout and login to make that change effective.

If it is still failed with the same error messsage, it is still possible that you are fooled by ACL permission. Use getfact to check it. The reason? When you ls, you get:

drwxr-xr-x+ 2 testuser testuser 40 Jul 17 15:19 /tmp/test

But your actual ACL (getfacl /tmp/test)might look like:

getfacl: Removing leading '/' from absolute path names
# file: tmp/test
# owner: testuser
# group: testuer


The Clamav image is from

Google Trips V.S. TripIt

Google Trips does make you travel easier … if you make it’s life easier.

It automatically recognizes the email from big name booking websites (e.g., airbnb) as well as airline. But if you order is from independent providers like many caravan parks, you are out of luck. You cannot even add them even you want to type everything yourself.

Google Trips also cannot capture the booking or reservation change via phone. You cannot blame them to not having this feature, but practical product should be able to allow user to make change.

Google Trips does have its strength beside the auto trips creation.  It provides a easy way to add city (but not necessary add reservation) and downloads offline maps that include Google reviews on  “things to do” and “food and  drink”.

TripIt offers some kind of automation, but you have to mail to, which is an extra step from Google Trips. Airlines and big name booking sites also got good support there: the important fields like date, confirmation number are prefilled. You do have to enter the information for smaller providers, but at least you CAN enter them. Not to mention that you can enter the cost in each item to track the total cost.

My advice? Install both of them. Use TripIt for handling the accommodation, airticket, car hire. And use Google Trips for local food and attraction.



Jenkins SSH: Invalid PEM structure, ‘—–BEGIN…’ missing

Quick Solution

  1. Paste the RSA private key (e,g. content of ~/.ssh/id_rsa) to Jenkins, like this:jenkins credentials
  2. Ensure the ~/.ssh/known-hosts in Jenkins master has agent/slave host key. Like:
    stdbuf -o0 -e0 ssh-keyscan -H  &>> ~/.ssh/known_hosts
  3. Ensure the ~/.ssh/authorized_keys of Jenkins agent/slave has Jenkins master’s RSA key.
  4. (Optional) if you can access the terminal of Jenkins master, test the connection by using:


Because the issue Presence of ECDSA SSH keys breaks SSH credentials plugin seems to affect the Jenkins masters that have both ECDSA and RSA.

Jenkins: No entry currently exists in the Known Hosts file for this host

I have encountered the following error message when I was trying to connect Jenkins slave after plugin update:

[SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file.
 Key exchange was not finished, connection is closed. There was a problem while connecting to

I have tried to connect using ssh in console, it connected successfully, but Jenkins still refuse to connect.

Then I discovered that, if I provided the RSA Host key, Jenkins can now connected to slaves.

I guess the reason is ssh just use the known host key to determine whether it is known, and be able to fallback to RSA for actual authentication. On the other hand, Jenkins does not seem to have the fallback. You given RSA identity, then Jenkins expect RSA in known_hosts.

The issue is already filed as JENKINS-42959 Failed known_hosts verification for SSH agent. In the meantime you can use following workaround:

stdbuf -o0 -e0 ssh-keyscan -H <host> &>> ~/.ssh/known_hosts

The stdbuf here is for printing the stdout and stderr as the order of time they appear, just like what you would see in console. Otherwise the stderr will go first and then stdout.

在微信 (WeChat) 回報 Feature Request 的歡樂體驗

我把手機 reset 後,開始了漫長的回復過程。其他的軟體,從遊戲到Facebook、line都是只給帳密就什麼都恢復了…除了微信。


(我也想這樣做,可是光reset 生不出舊手機)
(我錯了,我不應該假設 Facebook 能,作為競爭對手的微信也能)
(雖然 Facebook 用戶更多,可是騰訊的工資和設備比臉書較便宜(應該吧),還是應該體諒人家)
(This is not a bug, it is a feature!)

本以為沒招了,眼睛瞄到,嘿,還有 WeChat Team 在啊,頓時躁動的心獲得解脫。馬上把
「我希望微信能像 Faceobook 和 line 一樣,手機 reset 後還能撈回群裡的歷史資料。」
這段貼給 WeChat Team。人家是專業的,必然能給專業的答案。





我沒看錯吧,要撈微信群的資料居然要動用 Facebook? 這跟當年問 iphone 誰是世界上最好的手機時,它推薦 三星手機 是一樣的無私啊。要是能用 Facebook 救微信的資料,那也是功德無量。既然這樣那還不趕快照著步驟做?

  1. 設置 -> 通用 ,可是我就是沒找著安卓上這個設定在那。查了一下,原來是蘋果的設置方法。往好處想,看來微信沒有偷查你是安卓還是蘋果,
  2. 雖然我對(轻触此处)不能直接連到 facebookapp略有微詞,不過瑕不掩瑜。可是我搜了半天,愣沒搜到 facebookapp 或 facebook 。這又使我對微信的嚴謹感到敬佩:你看,沒照步驟來,人家就不理你了,活該。



Introducing Bus Factor


The Bus factor measures the degree of the knowledge sharing. High number means the knowledge is well shared. For example, bus factor 10 means 10 team members need to be “neutralized” to stop the project working.

This lead to the following issue: How to introduce it to your team members.
Even it is called *bus factor* does not mean you should introduce as such.

I still remember when one of my colleague explaining the idea:

Consider when you are hitting by a bus …

My real feeling on that time was:

Are you trying to curse me?

That’s right. It triggers all negative feeling and reaction. In other words, that will not help knowledge sharing.

A good way to get around this is terming it with holiday factor instead. Ah, holiday, this associate with more pleasant mood, thus make the concept much easier to be delivered and heard. And, hey, we do need to consider the impact of long holidays like Christmas and Chinese New Year.

Using holiday factor also deliver following message:

If you do not share, we may need to call you in the most inconvenient time …

So, using holiday factor is more acceptable, realistic and passive aggressive. 🙂

Yet, everything has it own limitation, and the “bus factory” is no exception. One of my respected colleague, Seth Vidal, was indeed killed in car accident, and his project, yum, is postponed. However, dnf takes its place and has been adopted by Fedora community since Fedora 22.

Turn on “Ok Google” for any screen with HTC E9+ without Google Now Launcher

If you have invested to many time to organize the apps and folders, or just don’t want to change to Google Now launcher for any reason, you can still enjoy the “OK Google” from any screen.

Simply put, you can enable it with Google App.  Just follow the instruction provided from Google support.

systemd: remember to keep the daemons alive

My sinopia daemon refused to start. After dig out the journal, I found that ExecStop run straight after ExecStart, what the…

After some research, I found that for daemons, or any other program that put themselves at the background, systemd thought they are stopped, thus stop the service for you. To prevent this, you need following in your systemd service file:


so your daemons can live happily ever after.


nodejs/npm yum repo for EL7

The nodejs and npm in EL7 is too old, so I borrowed the  to latest nodejs spec from  rawhide. Long story short, the result is at:

Please read the disclaimer and do follow the installation instruction if you choose to proceed. I don’t usually put the disclaimer like that but you need to know that:

  1. The build dependency of nodejs include openssl-1.0.2, but EL7 only shipped with 1.0.1, yet nodejs can run with openssl-1.0.1.
  2. openssl-libs is an important package, without it,  yum, curl and rpm URL install won’t work, so restore it is a bit tricky. The instruction is, however, written in the copr page.

Longer story:

To build this copr, following dependencies need to go in as well:

  1. libuv
  2. crypto-policies
  3. openssl-1.0.2

libuv is piece of cake. But crypto-policies and openssl bring the worst packager nightmare: circular dependency. After F23, crypto-policies require openssl-devel to build, yet openssl require crypto-policies to run.

I eventually dug out crypto-policies from F21 and built it, thus broke the circular dependency and finished the build.