Posted on Leave a comment

3 Steps to prune Mongo DB in Unifi Controller

You will need two session of terminal to repair, the first session to run the mongo daemon and the second session to prune the database. You will need the mongo_prune_js.js.

Step 1. Login to the first session of the Unifi Controller

Once you login successfully, Stop the unifi service using the following command

service unifi stop

make sure there is no Mongodb session using the following command

ps -aux | grep mongo

Run the mongodb using the following command

sudo mongod --port 27117 -dbpath /usr/lib/unifi/data/db -smallfiles -logpath /usr/lib/unifi/logs/server.log -journal

you should see something similar to the following window:

start mongo db

Step 2. Login to the second session of the Unifi Controller

Download the pruning script, if you don’t have it already. Edit the file using the following command

vi mongo_prune_js.js

and make sure the following line is there

var days=0;
var dryrun=true;

Then execute the pruning script as below:

sudo /usr/bin/mongo --port 27117 < mongo_prune_js.js

You should see something like the following when it finished pruning

Mongo db pruning

Once it is success you need to run it again but this time edit the mongo_prune_js.js with the following

var days=0;
var dryrun=false;

When it is running completely you should see something similar to the following:

Step 3. run the unifi service

Jump back to the first session, and stop the running mongo db using Ctrl + C. You should be back to your command prompt now.

Start the unify server using the following:

service unifi start

If all goes well you should see the following message

Unifi start service

Double check the log files to ensure that the unifi service started successfully using the following command

tail -f /usr/lib/unifi/logs/mongod.log
tail -f /usr/lib/unifi/logs/server.log

You can also check that your Unifi web interface have started successfully using the following

https://yourunifiipaddress:8443

If you encounter problem pruning because you are running out of space, you can find the files that are big using the following command

find / -type f -size +20M -exec ls -lh {} \; | awk '{ print $NF ": " $5 }'

I hope this post have saved you time and frustration.

Posted on Leave a comment

Connecting IoT Sensors data to Node-RED

This is the continuation of the Temperature sensor project in the previous post. The concept is to allow the data from sensors (temperature, motion) can be displayed in Apple Homekit, so that the user can interact with the information and control the IoT connected devices (Lights, Fan, etc). It is best described in the following picture.

The following instruction shows how to install Node-RED on a linux computer running Debian OS.

sudo npm install -g --unsafe-perm node-red

You will also need to install the mosquitto MQTT message broker, here are the command required

sudo apt-key add mosquitto-repo.gpg.key
sudo wget http://repo.mosquitto.org/debian/mosquitto-jessie.list
sudo apt-get update
sudo apt-get install mosquitto

You will also need the latest Python library, so grab them using the following instruction

sudo apt-get install python-dev

Test the installation. In this example I was using Linux Debian, so typing the command node if you get the following in the command prompt, that means the installation is successful. So then you can exit node by typing .exit command in the prompt >.

If all goes well, you can run the node-red command in the command prompt. You should get the following message. This shows that the node-red is now running at http://127.0.0.1:1880.

Node-RED Settings

Node-RED setting file called settings.js, on Linux it is located in the /usr/lib/node-modules/node-red folder. You will have another settings.js file in the .node-red folder in your home folder. This setting will be loaded by default.

Creating the flow in Node-RED

Now that you have a running Node-RED, it is time to create the flow. In this example we will create a simple flow to read temperature posted by our ESP8266. Let’s start by firing up your favourite browser and point to the following URL: http://127.0.0.1:1880/

You will be presented with a blank screen similar to the following picture. Now to start creating a flow, drop an “Inject” node from input section. We will use this as a trigger to get the temperature reading. Once you dropped it in, double click to set the property. We call the node “timestamp” and we set the interval to repeat every 4 minutes.

The next step is to connect this with an “http” node, so drop an “http” node and configure this as http GET to call a server side script in the webserver. What the script needs to return is the temperature in JSON format as below:

{"CurrentTemperature":25}

So my data_store2.php script does exactly that, as shown in the following code:

 /* readtemperature file from temp.txt file    return the value back in JSON format for HomeKit  */ 
$theparam = $_GET;
$file = './temp.txt';
$temperature = file_get_contents($file);
echo '{"CurrentTemperature":'.$temperature.'}';

Now the final step is to connect to the “Homekit” node from the Advance nodes menu. Once you drop the “Homekit” node, you can double click to configure the property as below.

Once all had been connected, it is time to deploy the node. You can do this by clicking on the “Deploy” button at the top of the Node-RED window. You will need to click the “Deploy” button whenever you make any changes to the node. Sometime the deployment might stop the Node-RED server, so you just have to run the node-red command again in the command prompt.

If all goes well, you can now test this node by clicking on the button next to “timestamp” node, the temperature should be read from the webserver and displayed in Homekit, similar to the following picture.

That conclude this session on how to configure the Node-RED to work with our temperature sensor data from ESP8266. Please let me know if you have any questions related to this and don’t forget to subscribe for update on the similar projects. The next session we are going to connect this to the Apple homekit in IPhone or IPad.

Posted on Leave a comment

UniFi network controller failed to start

I was helping a friend to fix his UniFi network controller that failed to start. I didn’t know much about it, and we think this is related to the database mongodb which is full with logs of the events which make the size bloated up.

Here are the steps that I did to fix it.

  1. Stop the Unifi Service using the following command
service unifi stop

2. Repair the database with the following command

mongod --dbpath /usr/lib/unifi/data/db --smallfiles --logpath /usr/lib/unifi/logs/server.log --repair

3. Download the pruning script using WGET

wget https://ubnt.zendesk.com/hc/article_attachments/115024095828/mongo_prune_js.js

4. Perform the test run using the following command

mongo --port 27117 < mongo_prune_js.js

5. If the mongo db is not running you might have to run the following command first

sudo mongod --dbpath /usr/lib/unifi/data/db

6. Once the test run is success you will need to edit mongo_prune_js.js to disable “dryrun”, change the line var dryrun=false;

nano mongo_prune_js.js

7. Prune the database by executing the following command, this will take a while to run depends on the size of events that you have. I had about 2 millions events.

mongo --port 27117 < mongo_prune_js.js

8. If all goes well you should see a couple of OK message like the following

{ "ok" : 1}

9. Change the db and log files permission using the following (NOTE : this is an important step)

chown -R unifi:unifi /usr/lib/unifi/data/db/
chown -R unifi:unifi /usr/lib/unifi/logs/server.log

10. Finally start the unifi service using the following

service unifi start

11. if all goes well you should be able to login to the unifi web interface now.

Posted on Leave a comment

TERM support in Screen command

When you get the following error message when trying to start screen

Cannot find terminfo entry for 'xterm-256color'.

you will need to find the which TERM is supported:

ls /usr/share/terminfo/x

this will give you a list of supported TERMs i.e.

xterm
xterm-xfree86

set the environment variable:

export TERM=xterm-xfree86

and then run the screen command

TERMINFO='/usr/share/terminfo/' screen
Posted on Leave a comment

How to bulk rename file extension in Linux or Mac

There are times where you want to rename file extension or file ending in bulk, e.g.: from

Something_ONE.jpg to Something.jpg

Someotherfile_ONE.jpg to Someotherfile.jpg

the following command can be used, but before test it with the following

ls *_ONE.jpg

for file in *_ONE.jpg
do
 ls "$file" 
done

If the above command return the right files then you can rename the files using:

for file in *_ONE.jpg
do
 mv "$file" "${file%_ONE.jpg}.jpg"
done
Posted on Leave a comment

How to copy a file with I/O errors?

If you encounter the following error

To salvage all readable data, we will use a fault-tolerant copy utility called dd.

dd stands for “disk duplication”. This is a command-line utility bundled with Mac OS X. A free version for Windows is also available for download.

Be extremely careful, even if you are familiar with command line, because dd can cause unrecoverable damage to your data if you don’t specify correctly input and output. You have been warned.

The arguments of dd are straight-forward:

if=path_of_file_with_IO_errors specifies input path
of=path_of_clean_copy_to_create specifies output path
conv=noerror,sync tells dd to be fault-tolerant

Posted on Leave a comment

How to Setup SSH for Auto Login without a Password

Used for ssh or scp or anything that needs SSH secure access.

Quick reminder…

If you’ve done it before but forgot, here’s the refresher. (Otherwise, read below.)

On server:

cd .ssh
ssh-keygen -t rsa  (hit return through prompts)
cat id_rsa.pub >> authorized_keys
chmod 600 authorized_keys
rm id_rsa.pub

On client:

cd .ssh
scp myserver.com:.ssh/id_rsa myserver.rsa
chmod 600 myserver.rsa
echo "Host myserver" >> config
echo "Hostname reblets.com" >> config
echo "IdentityFile ~/.ssh/myserver.rsa" >> config

Test it:

scp file myserver:.
ssh myserver

Background

It’s common to use ssh and scp for communicating and transferring files to and from a server. If you want to auto-login without a password, here’s how to setup SSH to use encryption keys to do so.

On the Server

Use ssh to login to your server under the account name you want to use.

Run ssh-keygen to create an encryption key pair, the public and private keys. You can just hit return for each question.

[backup@server ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/backup/.ssh/id_rsa):
Created directory '/home/backup/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/backup/.ssh/id_rsa.
Your public key has been saved in /home/backup/.ssh/id_rsa.pub.

CD to the .ssh dir and list the files to understand what you’ve got:

[backup@server ~]$ cd .ssh
[backup@server .ssh]$ ls
id_rsa  id_rsa.pub

Add the public key id_rsa.pub to the SSH authorized_keys file. Be sure to use the >> to append it to any other keys you might have:

[backup@server .ssh]$ cat id_rsa.pub >> authorized_keys

Be sure the file is hidden from other users, and you can delete the public key now.

[backup@server .ssh]$ chmod 600 authorized_keys
[backup@server .ssh]$ rm id_rsa.pub

On the Client

 

Log-off the server, so you’re back on the client. Enter your .ssh directory and download the id_rsa file (the private key) from the server to your local machine:

$ cd .ssh
$ scp backup@server.example.com:.ssh/id_rsa .
$ chmod 600 id_rsa

Try it out:

$ ssh -l backup server.example.com
[backup@server ~]$

And you can use scp for secure file transfer:

$ scp backup.tar.gz backup@server.example.com:.

Multiple SSH keys

If you login to more than one server, you can configure SSH to fetch the correct key. Here’s an example of adding to the ssh config file:

cd .ssh
mv id_rsa example.rsa
echo "Host backup" >> config
echo "Hostname server.example.com" >> config
echo "IdentityFile ~/.ssh/example.rsa" >> config

If you need a different username on your server, you can add:

echo "User backup" >> config

SSH Agent

You can setup an environment variable to make it possible for programs that use SSH to find your keys, avoiding the requirement than you enter your passphrase each time.

Insert some lines like this into your .bash_profile:

if [ -z "$SSH_AUTH_SOCK" ] ; then
    eval `ssh-agent -s`
    ssh-add
fi