Picocluster, is a company selling tiny ARM clusters. Recently I bought a picocluster advanced cluster kit for some distributed computing experiments.
There a several different options for the clusters. I chose to go with Odroid C2's over the standard Raspberry Pi's because they have twice the RAM which is important for playing with Spark and Hadoop, and also Gigabit ethernet instead of the Pi's USB/ethernet bottleneck. There a several different size options available. I chose to go with a 5 node cluster as it came with a switch and seems "more serious" than the 3, while the 10 node cluster seemed like overkill.
For the base operating system I discarded the default images and went with Arch Linux ARM. This was because the base image provided came with Ubuntu, X.org, and MATE, and felt too heavy. I followed the instructions for preparing the image but made sure to give static ip addresses for each node by editing the default
eth0.network. The details here depend on how your router is setup with mine using the 192.168.0.0/24 range.
[[email protected] ~]$ cat /etc/systemd/network/eth0.network [Match] Name=eth0 [Network] Address=192.168.0.51/24 Gateway=192.168.0.1 DNS=220.127.116.11
After booting I also configured hostnames, setup sudo, and ssh keys on all the nodes.
My next goal was to install docker swarm on the cluster, which allows docker containers to be deployed and load balanced across a distributed set of nodes.
I installed docker on the master node and wrote simple scripts like below to install on every node.
for server in pc2 pc3 pc4 pc5 do ssh [email protected]$server "sudo pacman -S --noconfirm docker && \ sudo systemctl enable docker && \ sudo systemctl start docker && \ sudo gpasswd -a alarm docker" done
Note that with recent versions of docker like the one provided by Arch, you don't need to install docker swarm separately.
Next I created a swarm on my master node
docker swarm init --advertise-addr 192.168.0.51
And then let the other nodes join the swarm through the provided
docker join command. Running
docker node ls will let you see an overview of the swarm.
[[email protected] ~]$ docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 6uyle1yyfxit9osy65ql2xygl pc5 Ready Active 6xjfh3zz3c6jj7p0k5eda5zjw pc4 Ready Active 6y5v4c884oqysheylmvo38hib pc3 Ready Active 7r7w0415gbb7diac2p0033khj * pc1 Ready Active Leader d7nzoim2sl1w9r57eudktt7fu pc2 Ready Active
Because the servers have an ARM architecture, they will by default not run standard Docker images. There is a good list of some ARM centric docker images provided by Hypricot and also good information on their blog. As a simple example, the below runs a whoami service provided by port 80 on all ports.
docker service create --name whoami -p 80:8000 hypriot/rpi-whoami
We can see this service running on
[[email protected] ~]$ docker service ps whoami ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR ejpbii0nafnvhv7rlhas429uj whoami.1 hypriot/rpi-whoami pc2 Running Running 4 minutes ago
Now let's scale this to 5 replicas.
docker service scale whoami=5
[[email protected] ~]$ docker service ps whoami ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR ejpbii0nafnvhv7rlhas429uj whoami.1 hypriot/rpi-whoami pc2 Running Running 6 minutes ago 6ew13jqrn9cg7i2mwieuw36zv whoami.2 hypriot/rpi-whoami pc3 Running Running 1 seconds ago 9tnt9bdxsj8r42t3b142xqwg5 whoami.3 hypriot/rpi-whoami pc4 Running Running 2 seconds ago etcnr55gum1b19lcx0sq812tf whoami.4 hypriot/rpi-whoami pc1 Running Running 2 seconds ago 0mtx4hzb6sjp3yc07otomgko1 whoami.5 hypriot/rpi-whoami pc5 Running Running 2 seconds ago
You can see the service running by running
curl -4 http://pcX on any of the nodes.
For more information check out the Docker Swarm docs.