Running ipfire in a Proxmox KVM

Long ago my network equipment at home was simple: just used a Belkin N600 router to connect to my ISP. All my computers and devices, including some raspberry pi, mobile phones and pads will access the internet via the router. As the number of clients increased, I found that the Belkin router did not serve well. Therefore I installed a software firewall, IPFire, on a Raspberry Pi (Model B) and replace the existing router. (I still needed the N600 as a wireless bridge so I switch it to a bridge mode)

IPFire is a great software firewall. You can have a look at the website so see the features. Running IPFire on Raspberry Pi is surprisingly stable too. Everything was fine until I change the ISP plan from 8M (8mbps/1mbps)  to 100M (100mbps/30mbps).

The problem was the hardware limitation of the Raspberry Pi. The builtin LAN and another usb ethernet adapter worked fine when I was using 8M. However after I changed to use the 100M plan I found that the maximum speed was 30M something. In order to utilize my bandwidth I planned to run IPFire on another hardware.

I didn't have a big budget at that time so I wondered if I can make use of my existing hardware. My N54L that is running proxmox seems to be a good choice. So I decided to create a KVM for this.

The config of the IPFire VM is simple:

1. 512M ram

2. 1 CPU

3. 16GB virtual disk (virtio driver)

4. one bridged network (virtio driver)

One tricky thing I have done was to passthrough USB adapter from host to the VM, which should have a better performance.

By running lsusb on host:

# lsusb

...

Bus 003 Device 002: ID 0bca:8133 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter

Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

...

Then add a line at /etc/pve/qemu-server/.conf

usb0: host=0bca:8133,usb3=yes

note that only use "usb3=yes" if you are using a usb3 device.

When the setting was done, I started the VM, the USB adapter was detected and just followed the instruction to setup. In my case I connected USB adapter to RED (internet), the virtual adapter to GREEN (internal). As I did a IPFire configuration backup from raspberry pi. I just import it back to the VM and everything ran out of the box!

I did a quick test by running the command line version of speedtest (speedtest-cli). the result was 80mpbs/26mbps which is acceptable.

[root@ipfire ~]# ./speedtest-cli

Retrieving speedtest.net configuration...

Testing from XXXXXXX (xxx.xxx.xx.xxx)...

Retrieving speedtest.net server list...

Selecting best server based on ping...

Hosted by xxxxxxxx[22.51 km]: 8.898 ms

Testing download speed................................................................................

Download: 81.54 Mbit/s

Testing upload speed................................................................................................

Upload: 25.93 Mbit/s

In summary IPFire is a good solution if you want to have a software firewall. There are other free firewalls such as pfsense or opnsense. I will try them and compare. But for now, I am happy to use IPFire.

Playing Yeelight with Python

Recently I've been playing a Yeelight Smart LED Bulb. It'd be a good starting of learning home automation, as such smart LED Bulbs do not depend on any modification of your home electricity system. This yeelight bulb is already bundled with the WIFI chip so that you can configure it using its official app (both Android and iOS are available)

Using the official app is straight forward, then I wondered can I play it 'harder'. Then I searched and found some interest use cases:

1. Controlling yeelight by Python
2. Controlling yeelight by Siri (wow!)
3. Controlling yeelight by Home-Assistant

In fact there should be more cases, including Amazon Alexa or Google Home, but I do have these products and the setup will be very similar to 2. and 3, that I will explain it in another post later. Right now I am going to share how to play the yeelight with python.

First, you need the official app to enable developer mode. Then just use python yeelight package https://yeelight.readthedocs.io/en/latest/ and you are good to go!

import yeelight

yeelight_ip='xxx.xxx.xx.xxx' # your yeelight bulb IP
light = yeelight.Bulb(yeelight_ip, effect='smooth')
light.turn_off()
light.turn_on()

# Set it to white
light.set_rgb(255,255,255)

# Set the brightness
light.set_brightness(100)

Not very difficult right? If you have a raspberry Pi you can set some cronjobs in order to turn the light on in the morning as a silent  alarm clock, or place the bulb in your kid's bedroom, then change the brightness from 100% to 1% (then off, of course) in one hour.

If you think it's too dummy to control such a bulb, I agree. That's why there will be some more elegant solutions. Next time I'll show how I setup a virtual HomeKit and ask Siri to control the bulb.

Ref:

• setup yeelight using official apps video
• Enable developer mode: http://www.yeelight.com/en_US/developer
• Python Yeelight Library: https://yeelight.readthedocs.io/en/latest/

Sending Whatsapp Messages from Raspberry Pi

yowsup is the Whatsapp library. It is written in python so we can run even with a Raspberry Pi. After the installation there is a client script that we can send message to other Whatsapp user. 

Reasons to use yowsup:

1. send messages to mobile device like SMS
2. the cost is cheap: all we need is just a mobile number ( a number that has not been registered in Whatsapp) and a Raspberry Pi ( I am using Raspberry Pi B2, but any model should be found)

Installation

$ virtualenv2 venv/
New python executable in venv/bin/python2
Also creating executable in venv/bin/python
Installing setuptools, pip, wheel...done.
$ . venv/bin/activate
(venv)$ git clone https://github.com/tgalal/yowsup

Cloning into 'yowsup'...
remote: Counting objects: 6983, done.
remote: Total 6983 (delta 0), reused 0 (delta 0), pack-reused 6983
Receiving objects: 100% (6983/6983), 1.49 MiB | 574.00 KiB/s, done.
Resolving deltas: 100% (4388/4388), done.
Checking connectivity... done.

python setup.py  install

After the setup there will be a script "yowsup-cli"

Registration

Prepare the following information:

• the phone number           (XXXXXXXX)
• the country code             (CCC)
• the mobile country code (mcc)
• the mobile network code (mnc)

Note that the phone number is in the form of Country Code + Number. And the mcc and mnc can be found at https://en.wikipedia.org/wiki/Mobile_country_code

For example, if you are using Verizon LTE in US, that the mcc and mnc should be 310 and 012 respectively.

Run the command:

./yowsup-cli registration --requestcode sms --phone CCCXXXXXXXX --cc CCC --mcc 310 --mnc 012

INFO:yowsup.common.http.warequest:{"status":"sent","length":6,"method":"sms","retry_after":1805}

status: sent
retry_after: 1805
length: 6
method: sms

If you are lucky enough you will receive a SMS from the mobile phone.

With this WhatsApp code we can run another command:

./yowsup-cli registration --register 800-269 --phone CCCXXXXXXXX --cc CCC
INFO:yowsup.common.http.warequest:{"status":"ok","login":"CCCXXXXXXXX","pw":"kaiaCmXHgRxxxvuOfbbCFHlqbLI=","type":"new","expiration":1478245880,"kind":"free","price":"$1.00","cost":"1.00","currency":"USD","price_expiration":1449782266}

status: ok
kind: free
pw: kaiaCmXHgRxxxvuOfbbCFHlqbLI=
price: $1.00
price_expiration: 1449782266
currency: USD
cost: 1.00
expiration: 1478245880
login: CCCXXXXXXXX
type: new

Note that the line pw: xxxxxxxxxxxxx . That's the password we need to put in a config file.

create a file yowsup-cli.conf with the following content:

cc=CCC # The country code

phone=XXXXXXXX # the phone number

id= # 

password=xxxxxxxxxxx # the password

Finally we can test by sending a message to a Whatsapp user (eg. CCCYYYYYYYY):

./yowsup-cli demos -c yowsup-cli.conf -s YYYYYYY “hello world”

By using the simple command we can send Whatsapp message to another user programmatically. Possible usages are to send alerts when some health check jobs failed, or writing authentication apps that can send the passcode.

Ref:

• https://github.com/tgalal/yowsup
• https://en.wikipedia.org/wiki/Mobile_country_code

Pass Store (Password Manager in command line)

Everybody is facing a problem: too many password to remember. If there are too many to store in brian, then its time to store somewhere else. Some people will simply write on a notebook, some will store it in an excel, and some will download a password manager such as KeePass/KeePassX and LastPass. But as a Linux guy I would like to try another tool in command line: Pass (http://www.passwordstore.org/)

Why need a command line password manager? Having a command line password manager can provide flexibility:

• you use SSH to connect to the Linux so it is secure and you can view a password of a particular account without touching a mouse
• you can write simple script to deal with the passwords. for example, generate a number of account/ password pair, or dump a password list
• This tool is very lightweight. You can store your passwords in a Raspberry Pi !

If you visit the website you may find it is not difficult to use. However, there are some tricks

1. You must setup gpg keyrings before initialising the pass store
2. The name of the pass store is the ID of the gpg key.

Once you setup the pass store you are good to enjoy this tool.

The following steps are the example:

Setup Trac on Raspberry Pi as a Wiki and Issue Tracking Tool

Trac with GIT

Trac is a wiki and issue tracking tool. There are other choices such as redmind and mantisbt. However, Trac is the most suitable one for running on Raspberry Pi because:

1. it is run by python, which is installed by default for raspbian. (php and ruby are not required)
2. it is relatively light weight. It can use sqlite as the database, rather than MySQL or PostgresSQL.
3. it can be run in standalone mode, without using apache or lighttpd. (although I will still use lighttpd to 'proxy' Trac)
4. It supports modern version control tool such as Subversion and GIT. (redmine also supports, but again, it's not that light weight)
5. with making use of virtual environment (virutalenv), you can setup Trac without using a privilege account.
6. it is a mature product and many organizations are using it (http://trac.edgewall.org/wiki/TracUsers)

Install and setup Trac

First, make sure virtualenv is already installed. (apt-get install virtualenv)

Remote access to your Raspberry Pi with ssl/ssh multiplexer

One big advantage of Raspberry Pi is its low power consumption. I can power on it 7x24 and remote it at anytime (of course, assumed your router is also on). SSH from internet to your Pi is simple, by just forwarding port 22 or making the Pi is the default DMZ. However, some places only allows you to visit port 80 or 443, or you have to connect to the internal via a proxy server, which also blocks every ports but 80/443.

Since port 443 is an exception, why don't we set the SSH service to listen to it? Just modify /etc/ssh/sshd_config and add a line "Port 80" or "Port 443". It does work, but wait, what about if the Pi also serves as a Web Server with SSL enabled? If you choose 443 to serve SSH, then you can't have your web server to use SSL at 443. So how to solve this situation? Here is one of the possible approach.

Create a Wiki (Moinmoin) on Raspberry

Moinmoin is a very good Wiki application without using a DB. It is very fast, with easy syntax and I recommend to use it as a quick knowledge base for home or SME usage.

 This guide is to install moinmoin on a Raspberry Pi and make it accessible via Lighttpd (using fastcgi). It is assumed a virtual host is available for this wiki (eg. http://wiki.example.com )

The moinmoin program will be installed at /usr/local/moinmoin_engine while the instance will be installed at another place (eg. /usr/local/moinmoin_instance1. )

1. Download the latest source (1.9.5 as of this moment) at http://moinmo.in
2. extract the tar ball to a temp place (eg. /tmp)
3. under the source path, run

   python setup.py install --prefix=/usr/local/moinmoin_engine

4. mkdir the instance path

   mkdir /usr/local/moinmoin_instance1

5. copy the following directories from "engine" to "instance", and change the ownership to the lighttpd user:

   cp -r /usr/local/moinmoin_engine/share/moin/data /usr/local/moinmoin_instance1
   cp -r /usr/local/moinmoin_engine/share/moin/underlay /usr/local/moinmoin_instance1
   cp /usr/local/moinmoin_engine/share/moin/config/wikiconfig.py /usr/local/moinmoin_instance1
   cp /usr/local/moinmoin_engine/share/moin/server/moin.fcgi /usr/local/moinmoin_instance1
   chown -R www-data:www-data /usr/local/moinmoin_instance1

6. modify the moin.fcgi, locate the following lines:

   # a1) Path of the directory where the MoinMoin code package is located.
   #     Needed if you installed with --prefix=PREFIX or you didn't use setup.py.
   #sys.path.insert(0, 'PREFIX/lib/python2.3/site-packages')
   sys.path.insert(0, '/usr/local/moinmoin_engine/lib/python2.7/site-packages')
   
   # a2) Path of the directory where wikiconfig.py / farmconfig.py is located.
   #     See wiki/config/... for some sample config files.
   #sys.path.insert(0, '/path/to/wikiconfigdir')
   sys.path.insert(0, '/usr/local/moinmoin_instance1')
   
   ...
   
   ## this works around a bug in flup's CGI autodetection (as of flup 1.0.1):
   #os.environ['FCGI_FORCE_CGI'] = 'Y' # 'Y' for (slow) CGI, 'N' for FCGI
   os.environ['FCGI_FORCE_CGI'] = 'N'
   
   ...
   
   # Is fixing the script name needed?
   # Use None if your url looks like http://domain/wiki/moin.fcgi
   # Use '' if you use rewriting to run at http://domain/
   # Use '/mywiki' if you use rewriting to run at http://domain/mywiki/
   #fix_script_name = None  # <-- adapt="adapt" as="as" class="anchor" here="here" id="line-24" needed="needed" span="span">

fix_script_name = ''# <-- adapt="adapt" as="as" here="here" needed="needed" pre="pre">

Next prepare virtual host by appending the following block /etc/lighttpd/lighttpd.conf:

$HTTP["host"] =~ "wiki\.example\.com" {
        
  fastcgi.server += ( "/" =>
      ((
        "socket" => "/tmp/moin.socket",
        "min-procs" => 1,
        "max-procs" => 2,
        "check-local" => "disable",
        "bin-path" => "/usr/local/moinmoin_instance1/moin.fcgi",
        "fix-root-scriptname" => "enable"
      ))
  )

  alias.url += ( "/moin_static195" => "/usr/local/moinmoin_engine/lib/python2.7/site-packages/MoinMoin/web/static/htdocs")

}

Restart Lighttpd and the your own wiki is just born! 

For more information about user and administration guide, please refer the HelpContents.

PS: So here is the wiki running on my Raspberry Pi: http://wiki.jessed121.us/ . You can walk around and feel it.

Install Lighttpd, PHP5, MySQL on Raspberry Pi

To run a 'typical' server on Raspberry Pi, a Web server is necessary. If possible, it can run application (by PHP5, Java, Perl, Python, etc...). Apache may be the most famous web server, but here I choose Lighttpd is it is not as big as Apache. In fact, a lightweight server is more suitable. Also, it will use FastCGI to run PHP and Python, I will list out what applications/ services to be run on this server.

Finally Mysql is also a popular DB. At the beginning I don't want to install it as there may be a huge overhead on the Pi. In fact, some embedded DB such as sqlite would be a better option. However, there are still applications which only official support Mysql, so at last I decide to install it and hopefully can fine-tune it later.

So here is the steps...

Install Lighttpd and PHP5

1. Login as root
2. create the user and group "www-data"
3. Install the following packages:
   • apt-get update (get the latest repository)
   • apt-get install lighttpd php5-cgi

After installation, check the /etc/lighttpd/lighttpd.conf

include_shell "/usr/share/lighttpd/include-conf-enabled.pl"

it will includes setting under /etc/lighttpd/conf-enabled. Make sure there is something like "fastcgi" and "fastcgi-php". (The settings can be fine tuned afterwards)

# ls /etc/lighttpd/conf-enabled/
10-fastcgi.conf  15-fastcgi-php.conf

Install Mysql

just install the packages "mysql-server", "php5-mysql"

My first Raspberry Pi

I bought my Raspberry Pi (with a transparent case) 2 weeks ago. Since its tiny size and low power consumption, I decided to run it in 7x24 as a light weight server.

When the Pi was just delivered, my Dad wondered its ability. Don't under estimate the configuration of it. I called it a "server" means it really fulfill my expectation.

Once I recieved it, what have I done to it: