An alternate to slack on Pi 3 (Rocket.Chat)

Rocket.Chat is an open source alternative to slack, and I have been thinking about using it at home. The ubuntu install was painless and very easy, but it didn’t make sense to keep a box up all the time – at least at home. So the best home deployment platform was an sbc (Pi 3).

Installation on pi was ‘mostly’ easy (followed official link) except for the persistence layer – mongodb. Rocket.chat required a newer version of mongodb, and the github page offered a cloud-based alternative for mongodb hosting. To me the whole solution being on-premise (actually whole solution on-pi) was major convincing point, so I avoided that route. After few searches, I located a post by Andy Felong, where he had already compiled binaries for arm architecture (Pi 2 and 3 – Jessie) of mongodb 3.0.9. Although the steps he had listed to deploy mongodb using his binaries were straightforward, I still scripted it; in case if I had to reinstall things again (link to script on github).

After mongodb was installed, the Rocket.chat server started smoothly. My beta testers (my kids) were ready to jump on it as soon as I created their user accounts. So much that at one point, when a file upload was a bit slow; I saw my son clicking the button multiple times – and suddenly all of them got a yellow header in chat interface due to server unavailability. I went back to terminal and there was a stack-trace waiting. This was the right time to graduate the interactive start into supervisor-based run.

The more interesting part was integration via webhooks. Rocket.chat supports writing webhooks wrappers in javascript e.g. it was very easy to pull live images from webcam.

Here is the launch.sh:

#!/bin/bash
export PORT=3000 
export ROOT_URL=http://127.0.0.1:3000 
export MONGO_URL=mongodb://127.0.0.1:27017/rocketchat 
export ADMIN_USER=admin
export ADMIN_PASS=*****
export ADMIN_EMAIL=admin@somedomain
/home/pi/meteor/dev_bundle/bin/node /home/pi/rocketchat/bundle/main.js

And the supervisor’s run-as-svc.sh:

#!/bin/bash
echo "Note: Start this script as sudo"
svcconf=/etc/supervisor/conf.d/rocketchat.conf
if [ -f "$svcconf" ]; then
  echo "supervisor config already exists"
else
  echo "creating supervisor config - first time only"
cat > /etc/supervisor/conf.d/rocketchat.conf <<EOF
[program:rocketchat]
command=/home/pi/rocketchat/bundle/launch.sh
directory=/home/pi/rocketchat/bundle
autostart=true
autorestart=true
stderr_logfile=/var/log/rocketChat.svc.err.log
stdout_logfile=/var/log/rocketChat.svc.out.log
EOF
fi
echo "Starting process"
supervisorctl reread
supervisorctl update
supervisorctl restart rocketchat

Finally, the only disappointment was android cordova-based app. It kept on showing media less pages (no images, or css loaded) in the webview. I think it could be due to some baked expectations of SSL within the app.

Overall, a fun little project with great potential with webhooks. Some thoughts for webhooks: web cluster/pool member status, build status notifications, current server load, or how Chicago downtown looks like at this exact minute.

OAuth2 via curl – Google Calendar API

Some time and technology iterations have passed since the previous post on similar topic (Google Contact Data API via curl). Still, curl is a fun way to play with APIs – this time Google Calendar API to download calendar events.

Below is the general flow for playing with Google APIs – starting with API console project setup manually, till the App (a bash script) gets data. “Almost” all work was done in bash (json parsing was the exception).

This bash script became the data provider part for another raspberry pi endeavor – “Announce upcoming calendar events using TTS.”

GoogleAPIAccess6.png

For my own memory, I also documented the steps in Freemind.

Screenshot from 2016-12-28 03-59-57.png

googleapiaccessoauth2

(Click here to open the full image in new tab)

For the first phase (the setup), the steps were simple on API console website.

screenshot-from-2016-12-28-04-01-15

The second phase was to obtain Authorization code with the help of the user.

Screenshot from 2016-12-28 04-03-46.pngNext step was to obtain Access Token. This was done via curl. Bash is cool but it can sometimes become an obsession to do everything in bash. After a break, it seemed more logical to do json parsing in python – while keeping main control in bash. The lines were still compact.

Screenshot from 2016-12-28 04-06-06.png

Eventually, once Access Token were obtained, its time to get the data. This was the easy part.

Screenshot from 2016-12-28 04-08-42.png

Finally, once the Access Token expires, Refresh token is used to get the new one. This one is very similar to second step, but instead of authorization code, the refresh token is sent.

Screenshot from 2016-12-28 04-09-53.png

To do: Upload complete bash script on github.

Image Magick with Powershell

Background: 

  • Lots of burst images e.g. consecutive screenshots, time-lapse images, etc.
  • Need to keep only the images with significant changes i.e. de-dup almost similar ones.

Solution:

Image Magick is a great open-source software. The idea is to use its ‘compare’ module to compare consecutive images and get a numeric value which can be checked against some threshold. If the numeric value > threshold, then the change is significant. Simple!

Having done it on linux (raspberry pi), the diff-ing part was easy and tested with parameters: Fuzziness, Quiet. (reference)

The unexpected challenging part was powershell syntax to read image magick output! A simple task of loading the numeric result from stdout (or stderr in case of image magick) into a variable was such a big challenge. Appeared as if MS created hacks to make quoted arguments work. After spending hours with Invoke-Command, Invoke-Expression, iex, OutVariable, &, etc., the max that was accomplished was a console error with ps trying to execute the stdout/stderr numeric output, after executing the actual command.

C:\Progra~1\ImageMagick-7.0.4-Q16\magick : 262
At line:1 char:1
+ C:\Progra~1\ImageMagick-7.0.4-Q16\magick compare  ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo : NotSpecified: (262:String) [], RemoteException
 + FullyQualifiedErrorId : NativeCommandError

The fix was in two parts.

One was redirecting stderr (2) to stdout (1) i.e. 2>&1.

$fullcmdline = '<path to image magick>\magick <compare args> <path to imageN> <path to image N+1> tmp.jpg 2>&1'

The other was iex syntax:

$result = iex (${fullcmdline})

Once this was resolved, the rest was easy – delete files where $result did not exceed the threshold.

Azaan on Raspberry Pi

… I used to dedicate a PC to run Azaan (Call for Prayers; Adhaan). It was an always-on requirement – 24x7x365 but with very light load. At one point, I got zotac from newegg, but the potential of miniaturization was still there. This need was fulfilled by raspberry pi.

After Raspberry Pi 2 came out and took over the duties of B+, the B+ immediately found its home on top of refrigerator doing nothing but running Azaan. Initially, I created a program in python with prayer time calculations from praytimes.org. It was while loop with a sleep running 24×7. May be due to memory leaks or other reasons, I found that it stopped working after every few days.

Recently, I took another approach, which proved to be more stable.

Below is the code flow — I will add the python and bash script soon.

Screenshot from 2015-08-23 23:48:39

Only for folks literate in Warcraft 3 && R

Interesting phenomena: culture shock. But even more interesting is what happens afterwards. Some people adapt to the new-n-shiny and move on. Others keep their memories dear, and pretend the time has stopped around them … In my case, there was one segment where I tried to freeze the time: LAN gaming. Specifically WarCraft. Few years ago (or was it a decade?) when ZNK visited, we setup the things the same way and revived not-so-old memories. It was lot of fun!  The world moved forward with new versions of warcraft, but I found less time to switch my interest (iow – too old to try new stuff). Not sure about ZNK.

Anyway, it was my turn to transfer the legacy (Warcraft III – Frozen Throne) to the next generation, the kids, and of course they started asking the questions that we didn’t ask e.g. “Which race and which hero is better?”. I always played as humans and never switched (tunnel-vision), but these guys wanted to try out different races and heroes based on opponent.

We started by looking at Blizzard’s website. Copy-pasted some data in XLS and created some graphs. But the “business requirements” kept on changing e.g. asking for comparison by race and by skill. So I ended up applying some web crawling and R to solve this one.

So here is a quick-n-dirty ggplot output which partially satisfied the ‘customers’: (click to enlarge)

wc3-heroes

Comparison of Heroes

Now I think that all these years, Orcs had an unfair advantage against me!

Plotting code:

library(ggplot2)
#install.packages(‘gridExtra’)
library(gridExtra)
setwd(‘~/Dropbox/warcraft’)
#install.packages(‘qdapRegex’)
library(qdapRegex)
theme_bw()

###################### HEROS ##############################
heros <- read.csv(“heros.csv”)
heros$rname <- factor(paste(heros$name, heros$race))
heros$level <- factor(heros$level)
heros$composite <- heros$armor+heros$strength+heros$speed+heros$intel+heros$hit+heros$mana
heros$avgattack <- as.integer(unlist(rm_between(heros$attack, “[“, ” avg”,extract=T)))
heros <- heros[order(heros$race),]

g1 <- ggplot(data=heros, aes(level, strength, color=rname))
g1 <- g1 + geom_point() + geom_line(aes(group=rname))
g1 <- g1+ facet_grid(.~race)
g1 <- g1 + theme(legend.position=’none’)
#print(g)

g2 <- ggplot(data=heros, aes(level, armor, color=rname))
g2 <- g2+ geom_point() + geom_line(aes(group=rname))
g2 <- g2+ facet_grid(.~race)
g2 <- g2+ theme(legend.position=’none’)

g3 <- ggplot(data=heros, aes(level, speed, color=rname))
g3 <- g3+ geom_point() + geom_line(aes(group=rname))
g3 <- g3+ facet_grid(.~race)
g3 <- g3+ theme(legend.position=’none’)

g4 <- ggplot(data=heros, aes(level, hit, color=rname))
g4 <- g4+ geom_point() + geom_line(aes(group=rname))
g4 <- g4+ facet_grid(.~race)
g4 <- g4+ theme(legend.position=’none’)

g5 <- ggplot(data=heros, aes(level, avgattack, color=rname))
g5 <- g5+ geom_point() + geom_line(aes(group=rname))
g5 <- g5+ facet_grid(.~race)
g5 <- g5+ theme(legend.position=’none’)

g6 <- ggplot(data=heros, aes(level, composite, color=rname))
g6 <- g6+ geom_point() + geom_line(aes(group=rname))
g6 <- g6+ facet_grid(.~race)
g6 <- g6+ theme(legend.position=’bottom’)

g7 <- ggplot(data=heros, aes(level, mana, color=rname))
g7 <- g7+ geom_point() + geom_line(aes(group=rname))
g7 <- g7+ facet_grid(.~race)
g7 <- g7+ theme(legend.position=’none’)

g8 <- ggplot(data=heros, aes(level, intel, color=rname))
g8 <- g8+ geom_point() + geom_line(aes(group=rname))
g8 <- g8+ facet_grid(.~race)
g8 <- g8+ theme(legend.position=’none’)

grid.arrange(g1,g2,g3,g4,g5,g8,g7,g6, ncol=2)