Thursday 18 April 2013

Digispark / RPi / HC-SR04 measurement - 2.

OK - now pretty much happy with the digispark / HC-SR04 interface.  Using a 'newping' library from here https://code.google.com/p/arduino-new-ping/ to control the interface which makes the digispark code simpler. - The code writes to the USB every time we get an input on the USB. - I will then use the Pi to send a 'request' to the digispark. - The digispark does the measurement (median / average of 20 pings), and returns the distance in cm.

Note that I had to comment out the 'timer' functions in the newping library to get it to compile for the digispark. - Think the Arduino has some functionallity which isn't available in the digispark.

HC-SR04 trigger connected to digispark pin 0.
HC-SR04 echo connected to digispark pin 2.

(Mine is a rev A digispark which means that the onboard LED is connected on Pin1, and I use this to flash when data is transmitted.)

Code for the digispark sketch.


#include <DigiUSB.h>
#include <NewPing.h>

//
// AJR - 17/4/13.  USing HC-SR04 to measure distance
// and output on the USB so it can be read on an RPi.
// Use with the receive python progs
// from the Digispark examples.
// DigisparkExamplePrograms-master.
//
// Utilising the NewPing library from https://code.google.com/p/arduino-new-ping/
// Note that this will only compile if several of the timer functions are removed
// from the library.  I think there are some bits missing on the digispark which
// would otherwise be present in arduino.  One thing it is missing is memory!
// I keep running out of memory in the code space, and if I try to do too much here,
// then also run out. - Would like to have returned version string etc, but not
// enough room.... Ho, hum.
//

#define trigPin 0
#define echoPin 2
#define maxDist 300
// Pretty self explanatory below, but maxDist is the distance beyond
// which a ping is considered invalid.
NewPing sonar(trigPin, echoPin, maxDist);

void setup() {
  DigiUSB.begin();
  pinMode(1,OUTPUT); //give us some idea it's workng by flashing led
  pinMode(trigPin, OUTPUT);
  digitalWrite(trigPin, LOW);
  pinMode(echoPin, INPUT);
}

void measure() {
  int lastRead;
    digitalWrite(1,HIGH);
    DigiUSB.delay(100);
    // - This is where we call the NewPing library.  Don't think digispark has enough memory to do 50
    // but it seems reliable with 20.
    unsigned int uS = sonar.ping_median(20);
    DigiUSB.println(uS / US_ROUNDTRIP_CM);
    digitalWrite(1,LOW);
    delay(50); //sleep for 0.5 seconds before sending again.
}

void get_input() {
  int lastRead;
  // when there are no characters to read, or the character isn't a newline
  while (true) { // loop forever
    if (DigiUSB.available()) {
      // something to read
      lastRead = DigiUSB.read();
          if (lastRead == '\n') {
              measure();
              break; // when we get a newline, break out of loop
          }
    }
    // refresh the usb port for 10 milliseconds
    DigiUSB.delay(10);
  }
}

void loop() {
  // print output
  // DigiUSB.println("Waiting for input...");
  digitalWrite(1, LOW);
  // Measure....
  get_input();
}


Wednesday 17 April 2013

Raspberry Pi / Digispark distance measurement 1.

I used the following sketch to be able to receive the distance measurement from an HC-SR04 ultrasound sensor connected to the Digispark on the Pi using the digispark example 'receive.py' script.

#include <DigiUSB.h>

//
// AJR - 17/4/13.  USing HC-SR04 to measure distance
// and output on the USB so it can be read on an RPi.
// Use with the receive python progs 
// from the Digispark examples.
// DigisparkExamplePrograms-master.
//

#define trigPin 0
#define echoPin 2
char data[] = "The quick brown fog jumped over the lazy dogx";
long distance = 42;
long duration;

void setup() {
  DigiUSB.begin();
  pinMode(1,OUTPUT); //give us some idea it's workng by flashing led
  pinMode(trigPin, OUTPUT);
  pinMode(echoPin, INPUT);
}

void measure() {
  int lastRead;
  while (true) { // loop forever
    digitalWrite(1,HIGH);
    DigiUSB.delay(100);
    // Try measure, and send response.
    digitalWrite(trigPin, LOW);  // Trig needs to go high on 10us pulse
    delayMicroseconds(2); // wait....
    digitalWrite(trigPin, HIGH);
    delayMicroseconds(10); // Trig pin pulse time.
    digitalWrite(trigPin, LOW);  // Now, we time to echo.
    duration = pulseIn(echoPin, HIGH);
    distance = (duration/2) / 29.1;
    ltoa(distance, data, 10);
    strcat(data, " cm");
    DigiUSB.println(data);
    digitalWrite(1,LOW);
    delay(500); //sleep for 0.5 seconds before sending again.
  }
}

void loop() {
  // print output
  DigiUSB.println("Waiting for input...");
  digitalWrite(1, LOW); 
  // Measure....
  measure();
}

Digispark / Rpi.


Talking with a relative about the arduino / Pi integration, he gave me a 'digispark' which he'd backed on kickstarter.  This device should be able to plug into the USB, and tidy up the wiring a bit from the arduino idea to measure distance........   So, how to make it work.

I don't seem able to program this on Linux - it isn't recognised on my PC - USB problems, however I can program it on Windows, then plug into the Pi, and it is recognised.  (There need to be various bits for the Pi recompiled to make the IDE work on Pi, and this is outside my needs (capabilities probably....).

On the pi...  sudo pip install PyUSB

From the digispark wiki, install the examples, load up the DigiUSB example onto the digispark, and run the recieve.py on the Pi:-


pi@raspberrypi ~/digispark/DigisparkExamplePrograms-master/Python/DigiUSB/source $ cat receive.py
#/usr/bin/python

#
# Written for PyUSB 1.0 (w/libusb 1.0.3)
#


import usb # 1.0 not 0.4


import sys
sys.path.append("..")

from arduino.usbdevice import ArduinoUsbDevice


if __name__ == "__main__":
    try:
        theDevice = ArduinoUsbDevice(idVendor=0x16c0, idProduct=0x05df)
    except:
        sys.exit("No DigiUSB Device Found")




    import sys
    import time

    while 1 == 1:
        try:
            lastChar = chr(theDevice.read())
            if(lastChar == "\n"):
                break
            sys.stdout.write(lastChar)
            sys.stdout.flush()


        except:
            # TODO: Check for exception properly
            time.sleep(0.1)


Cool. - Something was received!!  The foundation to rework the distance measurement here.

RPi / Arduino - interfacing.

Right. - So my issue is that I wrote a system on the Pi using Python with an HC-SR04 (ultrasonic distance) sensor. - This needs accurate measurement of time to give meaningful distance, and because the Pi is scheduled, multi-tasking OS, this is nigh on impossible to do, and the variance in distance measured was some 30cm over about 1.5M over a 24 hour period of monitoring.

Some tinkering with an Arduino seems to suggest that this is much more accurate in it's timing as you'd expect because it's not multi-tasking.....  So - plan is to implement distance measure on the Arduino, and use the Pi for the higher level bits. (tweet / mail / cosm etc).

http://neophob.com/2013/04/i2c-communication-between-a-rpi-and-a-arduino/

1. - sudo apt-get install arduino.
2. - Unblacklist i2c

pi@raspberrypi ~ $ cat /etc/modprobe.d/raspi-blacklist.conf
# blacklist spi and i2c by default (many users don't need them)

##blacklist spi-bcm2708
##blacklist i2c-bcm2708


pi@raspberrypi ~ $ sudo modprobe i2c-bcm2708


Add i2c-dev to /etc/modules

sudo modprobe i2c-dev.

sudo apt-get install i2c-tools

sudo adduser pi i2c

sudo i2cdetect -y 0

No errors, so this seems to be working.



sudo apt-get install python-smbus


#!/usr/bin/python

import smbus
import time
bus = smbus.SMBus(0)
address = 0x2a

while True:
        data = ""
        ch = ""
        while ch != 'x':
            try:
                ch = chr(bus.read_byte(address));
                print ch;
            except:
                print "no data..."
                time.sleep(1) # give it chance to wake up / reboot, whatever.
            if ch != 'x':
                    data += ch;
        print data
        ch = ""
        time.sleep(1);

Corresponding sketch on the arduino for distance with HC-SR04 sensor:-


#include <Wire.h>
#define SLAVE_ADDRESS 0x2A
#define trigPin 6
#define echoPin 7

int led = 13;
char data[] = "The quick brown fog jumped over the lazy dogx";
long distance = 42;
long duration;
int index = 0;

void setup() {
    // initialize i2c as slave
    Wire.begin(SLAVE_ADDRESS);
    Wire.onRequest(sendData);
    pinMode(led, OUTPUT);
    Serial.begin (9600);
    pinMode(trigPin, OUTPUT);
    pinMode(echoPin, INPUT);
}

void loop() {
  //Serial.println("Hello");
  digitalWrite(trigPin, LOW);  // Trig needs to go high on 10us pulse
  delayMicroseconds(2); // wait....
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10); // Trig pin pulse time.
  digitalWrite(trigPin, LOW);  // Now, we time to echo.
  duration = pulseIn(echoPin, HIGH);
  distance = (duration/2) / 29.1;
  ltoa(distance, data, 10);
  strcat(data, "x");
  //Serial.println(data);
  delay(100);
}


// callback for sending data
void sendData() {
    digitalWrite(led, HIGH);
    Wire.write(data[index]);
    ++index;
    if (index >= strlen(data)) {
         index = 0;
         digitalWrite(led, LOW);
    }
 }


============================









Friday 12 April 2013

Remote access to Raspberry Pi with 3G USB Modem.

OK - so the scenario I have here is that I will deploy a Pi in a remote location some 3 hours from home. - This location doesn't have network access, so I've got a 3G USB modem installed on that Pi.  Periodically it brings up the link and will report monitoring data via email, twitter, and 'COSM' in time.

Here, my aim is to be able to connect to this remotely located Pi - I've pretty much followed the post here to do this using 'autossh'.  http://linuxaria.com/howto/permanent-ssh-tunnels-with-autossh?lang=en

My 'local' server is a also a RPi, and I've allowed access through my firewall to this pi on port 22 - let's call it 'localpi', and the remote pi 'remotepi'

I have tested that access can work in the following way.

1. - On 'remotepi', bring up the 3G link and then, ssh -R 2222:localhost:22 localpi.mydomain.com
This allows me to login to 'localpi' from my remote location.
2. - Now, the crux - can I login to 'remotepi' from 'localpi'? - ssh -p 2222 localhost - Yes, that works fine.
3. - Now, we know comms now work and can start automating the setup / link. On remotepi - sudo apt-get install autossh
4. - On localpi - sudo useradd -m -s /bin/false autossh
5. - On remotepi - sudo useradd -m -s /bin/false autossh
6. - On remotepi, run sudo su -s /bin/bash autossh
7. - On remotepi, run ssh-keygen.  Leave with empty passphrase and default location for the id.
8. - Transfer the id_pub key from 'remotepi' and enter into 'localpi' into ~autossh/.ssh/authorized_keys
9. - Do the same thing on localpi, and transfer the id_pub to ~autossh/.ssh/authorized_keys on remotepi - this way the connection for autossh will work both ways if necessary (and I'm not sure it is, but you never know...) - [ Note - It might be obvious, but you put the generated public key into the authorized_keys file on the host TO which you need to connect. - That way, you don't need a password when logging into that host. - Therefore, I actually did this backwards in the above.]
10. - On remotepi. - sudo su -s /bin/bash autossh
11. - On remotepi, bring up the link manually initially so you can accept the connection. - Future connections don't need to be interactive, though if 'localpi' changes it's IP address (for example if the broadband restart), then it might be necessary to do this step again.... - as user autossh on remotepi, run autossh -R 2222:localhost:22 localpi.mydomain.com
12. - Drop the ssh connection from step 11.
13. - Now try the 'whole' command on remotepi. - i.e. autossh -M 0 -q -f -N -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -R 2222:localhost:22
14. - On localpi, I should now be able to run ssh -p 2222 pi@localhost and get connected to 'remotepi', provided the link is up. - Great.  All that remains is to script it so it starts on boot, and test it.

Testing.....
1. - Put into /etc/rc.local:-


sudo su -s /bin/sh autossh -c 'autossh -M 0 -q -f -N -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -R 2222:localhost:22 autossh@localpi.mydomain.com'





- Reboot the remotepi.  Does this start? - Yes, it does. - Great.
- Now bring up the 3G link on the remotepi. - Does autossh connect to 'localpi' without intervention?  - Yes, it does! - Excellent. - So, now if I bring up the link periodically, then I should be able to make remote connection. - Cool.

- What I have found which isn't perfect is that the 'autossh' session which is established on 'localpi' doesn't ever seem to die when the link is dropped. - The solution here is to kill all 'autossh' sessions on 'localpi', then await the reconnection from 'remotepi' before trying to ssh to remotepi. - This seems to work reliably.
 - I've setup the script to bring the connection up for 15 minutes every hour, so this should give me time to connect if necessary.



RPi as Wireless Access Point.

A project I am working on will be deployed in a location without internet access.  The Pi will have a 3G modem connected, and I will detail that setup in another post.

What I want to be able to do here is to use the WI-FI dongle attached to the Pi to allow devices to connect, and potentially use the internet via the 3G dongle. - The first step in that is just getting the AP setup so that we can connect. - Taking this blog as a guide - http://www.rpiblog.com/2012/12/turn-raspberry-pi-into-wireless-access.html

Before I started, I plugged the Pi into a wired connection rather than it's more normal wireless connection, found the IP address and connected to that so I'm logged in over wired and won't lose connection when the hostapd kicks in.  I also ran an image backkup of the SD card so that I could at least revert back to previous configuration if necessary!


sudo apt-get install iw
sudo iw -list - shows that this supports AP mode correctly.


sudo apt-get install hostapd udhcpd

Give static IP to wlan0 - in /etc/network/interfaces:-


allow-hotplug wlan0
iface wlan0 inet static
address 192.168.3.1
netmask 255.255.255.0


/etc/udhcp.conf:-

pi@raspberrypi /etc/network $ grep -v "^#" /etc/udhcpd.conf |grep -v "^$"
start           192.168.3.20    #default: 192.168.0.20
end             192.168.3.254   #default: 192.168.0.254
interface       wlan0           #default: eth0
remaining       yes             #default: yes
opt     dns     8.8.8.8 4.2.2.2
option  subnet  255.255.255.0
opt     router  192.168.3.1
option  lease   864000          # 10 days of seconds

pi@raspberrypi ~ $ cat /etc/hostapd/hostapd.conf
interface=wlan0
driver=nl80211
ssid=MySSID
hw_mode=b
channel=4
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=2
wpa_passphrase=mykey
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP


edit /etc/default/hostapd - change the following to point to the hostapd.conf file:-

DAEMON_CONF="/etc/hostapd/hostapd.conf"

edit /etc/sysctl.conf:-


# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

sudo sysctl -p (to re-read the setting)

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT

sudo service hostapd start
sudo service udhcpd start

At this point, we are working- hurray.  Now make the IP tables setup permanent.

sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"

Edit /etc/network/interfaces:-

I didn't find these last commands necessary, since the install enabled the daemons, but this  is to enable the daemons on boot.

sudo update-rc.d hostapd enable
sudo update-rc.d udhcpd enable

Excellent - so, now I have an AP which will route traffic through eth0, and assign an IP address to my connected device.
DNS works fine.
I can also connect to the eth0 IP address when I'm on that network, so all appears to be working fine....

OK - this didn't work on reboot unless the ethernet cable was plugged in... Bizzare. - It appeared that the plugd thought that the wlan0 interface was already configured, so I commented out the line in interfaces:-

#allow-hotplug wlan0

And now, the Pi starts up, issues IP addresses, and seems to work fine even when the eth0 isn't plugged on power up. - Cool.


Now - to setup for 'ppp0' via the USB modem.
- Change eth0 to ppp0 in /etc/iptables.ipv4.nat
- This is my 'connect' script using sakis3g.


echo "Connect attempt at `date`" >>$LOGFILE
/home/pi/3G/sakis3g connect FORCE_APN="general.t-mobile.uk::t-mobile:.." MODEM="OTHER" OTHER="USBMODEM" USBMODEM="12d1:1003" DIAL="*99#" >>$LOGFILE 2>&1
/home/pi/3G/sakis3g status >> $LOGFILE 2>&1
echo "Restoring iptables" >> $LOGFILE 2>&1
sudo /sbin/iptables-restore < /etc/iptables.ipv4.nat >> $LOGFILE 2>&1
echo "Connect attempt complete at `date`" >> $LOGFILE


Note here that I need to re-apply the iptables rules - it doesn't seem to work applying these at boot time when the ppp0 interface does not exist.


Huawei E160 USB Modem

I need to be able to use a USB / 3G dongle to connect the Pi to network in remote location....  This post documents trying to get this working.

1. - Install usb_modeswitch as the dongle at first appears as a CD device.

sudo apt-get install usb-modeswitch

lsusb shows the modem:-

Bus 001 Device 074: ID 12d1:1003 Huawei Technologies Co., Ltd. E220 HSDPA Modem / E230/E270/E870 HSDPA/HSUPA Modem

So, now I've unpacked the driver file in /usr/share/usb_modeswitch - Unsure yet where this is used, but pretty sure it needs to be unpacked from the archive therein.


pi@raspberrypi /usr/share/usb_modeswitch $ sudo tar zxvf configPack.tar.gz "12d1:1003"

Added the following into /etc/usb_modeswitch.conf

# choose one of these:
#DetachStorageOnly=1

DefaultVendor=0x12d1
DefaultProduct=0x1003
#TargetClass=0xff
CheckSuccess=20
HuaweiMode=1

OK - So actually I don't think I needed the usb_modeswitch as I think it's built into 'sakis3g'.

wget http://www.sakis3g.org/versions/latest/armv4t/sakis3g.gz

(See http://shkspr.mobi/blog/2012/07/3g-internet-on-raspberry-pi-success/)
(Also http://shkspr.mobi/blog/2012/06/raspberry-pi-python-and-3g-dongles-oh-my/ for SMS).

sudo pip install pyserial
Then try sms.py from the above page.  ( This worked on T-Mobile SIM - hurray.)

sudo apt-get install ppp

 T-mobile works on network manager under Ubuntu with :-
*99#
general.t-mobile.uk
user web
pass web

I did get this working under sakis3g with apn=general.t-mobile.uk / user=t-mobile / pass=..

I did get connected on the Pi with the dongle with O2 using Sakis, but only once.... Will try powered USB hub.  Having re-installed raspian, it keeps dropping out the modem.



./sakis3g connect FORCE_APN="general.t-mobile.uk::t-mobile:.." MODEM="OTHER" OTHER="USBMODEM" USBMODEM="12d1:1003" DIAL="*99#" LOGFILE="/home/pi/3G/sakis.log"

3/4/13 - My powered hub arrived - yay.  Testing again with the above command, and it seems to work for a few minutes at least. - Managed to install lynx, and then browse using that for some time over the modem.


OK - so I've been running for some time. - I cron'd a script which brings up the connection every hour, sends a mail and drops the connection - pretty much what I need it to do remotely. - All good.









Thursday 11 April 2013

Email from Raspberry Pi.

OK - Next thing I want to be able to do is to send mail from the RPi.

Following some of the instruction from http://iqjar.com/jar/sending-emails-from-the-raspberry-pi/

1 - Create a suitable gmail account to send mail from.
2. - sudo apt-get update
3. - sudo apt-get install ssmtp
4. - Edit /etc/ssmtp/ssmtp.conf


pi@raspberrypi /etc/ssmtp $ grep -v "^#" ssmtp.conf |grep -v "^$"
root=postmaster
mailhub=smtp.gmail.com:587
hostname=raspberrypi
AuthUser=myuser@gmail.com
AuthPass=mypassword
UseSTARTTLS=YES

5. - Edit /etc/ssmtp/revaliases

root:root@my.domain:smtp.gmail.com:587
pi:pi@my.domain:smtp.gmail.com:587

6. - Test it..... create a file called message.txt which will contain the message. - e.g.

pi@raspberrypi ~/mail $ cat message.txt
From: my.address@gmail.com
Subject: test email

hello world!
It works!


Now run command - ssmtp destination.address@gmail.com < message.txt

Cool - it works!

Friday 29 March 2013

Ultrasonic Distance Measurement. - HC-SR04 / Pi

Recently picked up a couple of Nokia 5110 displays, mounted on circuits with header pins suitable for connection to Arduino or Pi. - What inspired me to do so was this QRclock which works a treat, is very cool and contains all the setup for these displays - http://scruss.com/blog/2013/01/19/the-quite-rubbish-clock/
[ From a bare Raspian Wheezy install, I've put the details here just in case I need the wiring details later.
1. - sudo vi /etc/modprobe.d/raspi-blacklist.conf
2. - #blacklist spi-bcm2708
3. - sudo modprobe spi-bcm2708
4. - sudo apt-get install python-imaging python-imaging-tk python-pip python-dev git
5. - sudo pip install spidev
6. - sudo pip install wiringpi

7. - git clone git://github.com/mozillazg/python-qrcode.git
8. - cd python-qrcode/
9. - sudo python ./setup.py install

10. - Note to go by the pin functions rather than the numbers on the LCD - they may vary.


 LCD Pin       Function      Pi GPIO Pin #   Pi Pin Name
============= ============= =============== =============
 1 VCC         Vcc            1              3.3 V
 2 GND         Ground        25              GND
 3 SCE         Chip Enable   24              GPIO08 SPI0_CE0_N
 4 RST         Reset         11              GPIO17
 5 D/C         Data/Command  15              GPIO22
 6 DNK(MOSI)   Data In       19              GPIO10 SPI0_MOSI
 7 SCLK        Serial Clock  23              GPIO11 SPI0_SCLK
 8 LED         Backlight     12              GPIO18 PWM0

N.B. - See this link for details of the GPIO output pins - http://elinux.org/Rpi_Low-level_peripherals#General_Purpose_Input.2FOutput_.28GPIO.29
11. - Add in a start script to /etc/init.d so that it works on boot, and install it:-
 sudo insserv qrclock

Having got the display attached, I need to be able to measure distance for a project I'm working on. - I got a couple of very cheap ultrasonic sensors - HC-SR04 from ebay. - These were approx £5 for 2 delivered.


This was very simple to use in actual fact. - Connect Vcc pin to 5 volt output on the GPIO header, GND to ground, and the ECHO and TRIGGER pins to two of the addressable GPIO pins. (In my case, TRIGGER to pin 8, and ECHO to pin 10).  I reworked code from various sources on the net, and was able to create a distance measurement device which works tolerably well for my purposes. - It is never going to be accurate because of the way the Pi works - it's a multi-tasking OS, and the device relies on your code timing from the trigger being sent to the echo received, and you can't guarantee that the code handling the measurement is actually running at the time the echo is received. - Anyway, when there's nothing else running on the Pi, the measured distance seems to remain within about a 5% fluctuation.

Code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-

#
# AJR - 28/March/2013
# Distance script. - Cobbled together from various sources.
# All credit to original authors, but sorry I don't know exactly
# which bits from where.
# Takes an average of 3 readings from the attached Ultrasonic echo sounder.
# and converts into reading in cm of distance.
# Note that it's not accurate because the Pi is a multitasking, scheduled OS
# and the ultrasonic relies on timing the echo accurately to work out
# the distance - but, hey - ho, it's probably good enough for measuring
# river water level.....
#

import time
import RPi.GPIO as GPIO
import nokiaSPI

noki = nokiaSPI.NokiaSPI()              # create display device
noki.led(900)
noki.cls()
noki.centre_word(0, 'Distance in CM')

# Which GPIO's are used [0]=BCM Port Number [1]=BCM Name [2]=Use [3]=Pin
# ----------------------------------------------------------------------
arrgpio = [(15,"GPIO15","Echo",10),(14,"GPIO14","Trig",8)]



# Set GPIO Channels
# -----------------
GPIO.setmode(GPIO.BCM)
GPIO.setup(arrgpio[0][0], GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(arrgpio[1][0], GPIO.OUT)
GPIO.output(arrgpio[1][0], False)


# A couple of variables
# ---------------------
EXIT = 0                        # Infinite loop
decpulsetrigger = 0.0001        # Trigger duration
inttimeout = 2100               # Number of loop iterations before timeout called


# Wait for 2 seconds to allow the ultrasonics to settle (probably not needed)
# ---------------------------------------------------------------------------
print "Waiting for 2 seconds....."
noki.centre_word(3, 'Waiting for 2 seconds...')
time.sleep(2)


# Go
# --
print "Running...."
noki.cls()
noki.centre_word(3, 'Running....')

def measure():
    # Trigger high for 0.0001s then low

    GPIO.output(arrgpio[1][0], True)
    time.sleep(decpulsetrigger)
    GPIO.output(arrgpio[1][0], False)

    # Wait for echo to go high (or timeout)

    intcountdown = inttimeout

    while (GPIO.input(arrgpio[0][0]) == 0 and intcountdown > 0):
        intcountdown = intcountdown - 1

    # If echo is high

    if intcountdown > 0:

        # Start timer and init timeout countdown

        echostart = time.time()
        intcountdown = inttimeout

        # Wait for echo to go low (or timeout)

        while (GPIO.input(arrgpio[0][0]) == 1 and intcountdown > 0):
            intcountdown = intcountdown - 1

        # Stop timer

        echoend = time.time()


        # Echo duration

        echoduration = echoend - echostart
        # Display distance

    if intcountdown > 0:
        intdistance = (echoduration*1000000)/58
        return intdistance
    else:
        print "Distance - timeout"
        noki.cls()
        noki.centre_word(3, 'Timeout')
        return 1000 ### This should indicate that
                    ### the distance has timed out.

    # Wait at least .01s before re trig (or in this case .5s)

    ###time.sleep(.5)

def measure_average():
    count = 1
    distance = 0
    while ( count <= 3 ):
        distance = distance + measure()
        time.sleep(0.1)
        count = count + 1
    distance = distance / 3
    return distance

###
### Main program. - Take the output of the measure routine and
### average over three tries.
###
try:
    # Never ending loop
    # -----------------
    while EXIT == 0:

        distance = measure_average()
        if distance > 500: # we have timed out...
            print "Timeout"

        ### print "Distance = " + str(distance) + "cm"
        print 'Distance = %.0f cm' % (int(round(distance))) # print to nearest CM
        noki.cls()
        noki.centre_word(1, 'Distance')
        ### noki.centre_word(3, str(distance) + 'cm')
        noki.centre_word(3, '%.0f ' % (int(round(distance))) + 'cm')
        time.sleep(0.5)

except KeyboardInterrupt:
  # User pressed CTRL-C
  noki.cls()
  noki.centre_word(1, 'Prog End')
  # Reset GPIO settings
  GPIO.cleanup()
Here is the finished distance measure. - It's lying on the table, so this is the distance to the ceiling.


Sunday 3 February 2013

Nook - e-reader / Pi Display - Attempt 2.

OK - so I've finally got something close to what I was looking for here. The Nook can be sited standalone from the Pi and wirelessly grabs the image of the weather. - A script runs on the RPi (every hour from cron). - It generates a file (test.png) into an output directory - this graphic is generated using the script for the Kobo from here with minor modifications:-

http://www.mobileread.com/forums/showthread.php?t=194376

Then, on the Nook, I created a directory for a custom screensaver - /screensavers/weather.  The nook (rooted as previously with nookmanager) runs a program 'FolderSync' which periodically (every 6 hours) syncs the output directory on the RPi with the screensavers/weather directory on the Nook.

Finally, the Nook is configured so that the screensaver kicks in after 10 seconds by modifying the settings.db from http://forum.xda-developers.com/showthread.php?t=1721665.
Code:
adb pull /data/data/com.android.providers.settings/databases/settings.db settings.db

sqlite3 settings.db
update system set value=10000 where name='screen_off_timeout';
.q

adb push settings.db /data/data/com.android.providers.settings/databases/settings.db

The Kobo script is modified (actually much simplified) to create the output test.png as below:-
Code:
import urllib2
import pygame, os

from xml.dom.minidom import parseString
from datetime import date, datetime, timedelta
from subprocess import call

os.environ['SDL_NOMOUSE'] = '1'
print("Kobo Wifi weather forecast started.")

def index_error(error):
    print(error)
    print("Failed to fetch weather data.")
    print("Double check your location settings by running:")
    print(" cat location")
    print("If the information is incorrect, re-set your location with:")
    print(" set_location.sh")
    
def to_hex(color):
    hex_chars = "0123456789ABCDEF"
    return hex_chars[color / 16] + hex_chars[color % 16]
    
def convert_to_raw(surface):
    print("Converting image . . .")
    raw_img = ""
    for row in range(surface.get_height()):
        for col in range(surface.get_width()):
            color = surface.get_at((col, row))[0]
            raw_img += ('\\x' + to_hex(color)).decode('string_escape')
    f = open("/tmp/img.raw", "wb")
    f.write(raw_img)
    f.close()
    print("Image converted.")
    
    
def get_weather_data():
    
    print("Getting weather information . . .")

    try:
        location = open("location", "r")
        lat = location.readline().strip()
        lon = location.readline().strip()
        location.close()
    except IOError:
        print("\nCouldn't open location file.")
        print("Run the 'set_location.sh' script to set your location for the weather script.")
        return 1
    #print(lat, lon)
    #weather_link = 'http://graphical.weather.gov/xml/SOAP_server/ndfdSOAPclientByDay.php?whichClient=NDFDgenByDay&lat={0}&lon={1}&format=24+hourly&numDays=5&Unit=e'.format(lat, lon)
    weather_link='http://free.worldweatheronline.com/feed/weather.ashx?q={0},{1}&format=xml&num_of_days=5&key=525804183f140652120211'.format(lat, lon)
    #print(weather_link)
    weather_xml = urllib2.urlopen(weather_link)
    weather_data = weather_xml.read()
    weather_xml.close()

    dom = parseString(weather_data)
    
    unit_file = open("unit.txt", "r")
    unit = unit_file.read()
    unit_file.close()
    unit = unit.strip().upper()

    h_temps = dom.getElementsByTagName('tempMax%s' % unit)
    l_temps = dom.getElementsByTagName('tempMin%s' % unit)
    highs = []
    lows = []
    for i in h_temps:
        try:
            highs.append(str(i.firstChild.nodeValue))
        except AttributeError as error:
            print("Error getting temperature highs: " + str(error))
                    
    for i in l_temps:
        try:
            lows.append(str(i.firstChild.nodeValue))
        except AttributeError as error:
            print("Error getting temperature lows: " + str(error))

              
    conditions = []
    for con in dom.getElementsByTagName('weatherDesc')[1:]:
        conditions.append(str(con.firstChild.nodeValue))
        
    now = datetime.now()
    day3 = now + timedelta(days=2)
    day4 = now + timedelta(days=3)
    day5 = now + timedelta(days=4)
    days = ["Today", "Tomorrow", day3.strftime("%A"), day4.strftime("%A"), day5.strftime("%A")]

    # images
    # The first image link is for the current weather, which we don't want.
    icons = dom.getElementsByTagName('weatherIconUrl')[1:]
    img_links = []
    for i in icons:
        try:
            link = "icons/" + str(i.firstChild.nodeValue)[72:]
        except AttributeError as error:
            print("Error getting icon links: " + str(error))
        img_links.append(link)
        
        
    #print(img_links)
    #print(highs, lows)
    #print(conditions)
    #print(days)
    
    display(days, highs, lows, conditions, img_links, unit)


def display(days, highs, lows, conditions, img_links, unit):
    
    print("Creating image . . .")
   
    pygame.font.init()
    #pygame.mouse.set_visible(False)

    white = (255, 255, 255)
    black = (0, 0, 0)
    gray = (125, 125, 125)

    # AJR - try init display - output to PC screen for testing.
    # pygame.display.init()
    ### screen = pygame.display.set_mode((600, 800))
    screen = pygame.Surface((600, 800))
    screen.fill(white)

    tiny_font = pygame.font.Font("Cabin-Regular.ttf", 18)
    small_font = pygame.font.Font("Fabrica.otf", 22)
    font = pygame.font.Font("Forum-Regular.ttf", 40)
    comfortaa = pygame.font.Font("Comfortaa-Regular.ttf", 60)
    comfortaa_small = pygame.font.Font("Comfortaa-Regular.ttf", 35)

    # Dividing lines
    pygame.draw.line(screen, gray, (10, 200), (590, 200))
    pygame.draw.line(screen, gray, (10, 400), (590, 400))
    pygame.draw.line(screen, gray, (200, 410), (200, 790))
    pygame.draw.line(screen, gray, (400, 410), (400, 790))

    # Today
    date = small_font.render(days[0], True, black, white)
    date_rect = date.get_rect()
    date_rect.topleft = 10, 15

    high = small_font.render('high: ', True, black, white)
    high_rect = high.get_rect()
    high_temp = comfortaa.render(highs[0], True, black, white)
    htemp_rect = high_temp.get_rect()
    high_rect.topleft = (50, 100)
    htemp_rect.topleft = high_rect.topright

    low = small_font.render("low: ", True, black, white)
    low_rect = low.get_rect()
    low_rect.topleft = (400, 100)
    low_temp = comfortaa.render(lows[0], True, black, white)
    ltemp_rect = low_temp.get_rect()
    ltemp_rect.topleft = low_rect.topright


    condition = font.render(conditions[0], True, black, white)
    con_rect = condition.get_rect()
    con_rect.centerx = 300
    con_rect.top = 5
    # Make sure words don't overlap
    if con_rect.left < date_rect.right:
        con_rect.left = date_rect.right

    image = pygame.image.load(img_links[0])
    image.set_colorkey((255, 255, 255))
    img_rect = image.get_rect()
    img_rect.center = (300, 125)
    degrees = pygame.image.load("icons/%s.png" % unit)

    screen.blit(condition, con_rect)
    screen.blit(high, high_rect)
    screen.blit(degrees, htemp_rect.topright)
    screen.blit(degrees, ltemp_rect.topright)
    screen.blit(high_temp, htemp_rect)
    screen.blit(low, low_rect)
    screen.blit(low_temp, ltemp_rect)
    screen.blit(image, img_rect)
    screen.blit(date, date_rect)


    # Tomorrow
    date = small_font.render(days[1], True, black, white)
    date_rect = date.get_rect()
    date_rect.topleft = 10, 210

    high = small_font.render('high: ', True, black, white)
    high_rect = high.get_rect()
    high_temp = comfortaa.render(highs[1], True, black, white)
    htemp_rect = high_temp.get_rect()
    high_rect.topleft = (50, 300)
    htemp_rect.topleft = high_rect.topright

    low = small_font.render("low: ", True, black, white)
    low_rect = low.get_rect()
    low_rect.topleft = (400, 300)
    low_temp = comfortaa.render(lows[1], True, black, white)
    ltemp_rect = low_temp.get_rect()
    ltemp_rect.topleft = low_rect.topright


    condition = font.render(conditions[1], True, black, white)
    con_rect = condition.get_rect()
    con_rect.centerx = 300
    con_rect.top = 210
    if con_rect.left < date_rect.right:
        con_rect.left = date_rect.right

    image = pygame.image.load(img_links[1])
    image.set_colorkey((255, 255, 255))
    img_rect = image.get_rect()
    img_rect.center = (300, 325)

    screen.blit(condition, con_rect)
    screen.blit(high, high_rect)
    screen.blit(degrees, htemp_rect.topright)
    screen.blit(degrees, ltemp_rect.topright)
    screen.blit(high_temp, htemp_rect)
    screen.blit(low, low_rect)
    screen.blit(low_temp, ltemp_rect)
    screen.blit(image, img_rect)
    screen.blit(date, date_rect)



    # Day 3
    date = small_font.render(days[2], True, black, white)
    date_rect = date.get_rect()
    date_rect.centerx = 100
    date_rect.top = 410

    high = small_font.render('high: ', True, black, white)
    high_rect = high.get_rect()
    high_temp = comfortaa_small.render(highs[2], True, black, white)
    htemp_rect = high_temp.get_rect()
    high_rect.topright = (100, 630)
    htemp_rect.midleft = high_rect.midright

    low = small_font.render("low:  ", True, black, white)
    low_rect = low.get_rect()
    low_rect.topright = (100, 710)
    low_temp = comfortaa_small.render(lows[2], True, black, white)
    ltemp_rect = low_temp.get_rect()
    ltemp_rect.midleft = low_rect.midright


    condition = tiny_font.render(conditions[2], True, black, white)
    con_rect = condition.get_rect()
    con_rect.centerx = 100
    con_rect.top = 450

    image = pygame.image.load(img_links[2])
    image.set_colorkey((255, 255, 255))
    img_rect = image.get_rect()
    img_rect.center = (100, 540)

    screen.blit(condition, con_rect)
    screen.blit(high, high_rect)
    screen.blit(degrees, htemp_rect.topright)
    screen.blit(degrees, ltemp_rect.topright)
    screen.blit(high_temp, htemp_rect)
    screen.blit(low, low_rect)
    screen.blit(low_temp, ltemp_rect)
    screen.blit(image, img_rect)
    screen.blit(date, date_rect)



    # Day 4
    date = small_font.render(days[3], True, black, white)
    date_rect = date.get_rect()
    date_rect.centerx = 300
    date_rect.top = 410

    high = small_font.render('high: ', True, black, white)
    high_rect = high.get_rect()
    high_temp = comfortaa_small.render(highs[3], True, black, white)
    htemp_rect = high_temp.get_rect()
    high_rect.topright = (300, 630)
    htemp_rect.midleft = high_rect.midright

    low = small_font.render("low:  ", True, black, white)
    low_rect = low.get_rect()
    low_rect.topright = (300, 710)
    low_temp = comfortaa_small.render(lows[3], True, black, white)
    ltemp_rect = low_temp.get_rect()
    ltemp_rect.midleft = low_rect.midright


    condition = tiny_font.render(conditions[3], True, black, white)
    con_rect = condition.get_rect()
    con_rect.centerx = 300
    con_rect.top = 450

    image = pygame.image.load(img_links[3])
    image.set_colorkey((255, 255, 255))
    img_rect = image.get_rect()
    img_rect.center = (300, 540)

    screen.blit(condition, con_rect)
    screen.blit(high, high_rect)
    screen.blit(degrees, htemp_rect.topright)
    screen.blit(degrees, ltemp_rect.topright)
    screen.blit(high_temp, htemp_rect)
    screen.blit(low, low_rect)
    screen.blit(low_temp, ltemp_rect)
    screen.blit(image, img_rect)
    screen.blit(date, date_rect)

    # Day 5
    date = small_font.render(days[4], True, black, white)
    date_rect = date.get_rect()
    date_rect.centerx = 500
    date_rect.top = 410

    high = small_font.render('high: ', True, black, white)
    high_rect = high.get_rect()
    high_temp = comfortaa_small.render(highs[4], True, black, white)
    htemp_rect = high_temp.get_rect()
    high_rect.topright = (500, 630)
    htemp_rect.midleft = high_rect.midright

    low = small_font.render("low:  ", True, black, white)
    low_rect = low.get_rect()
    low_rect.topright = (500, 710)
    low_temp = comfortaa_small.render(lows[4], True, black, white)
    ltemp_rect = low_temp.get_rect()
    ltemp_rect.midleft = low_rect.midright


    condition = tiny_font.render(conditions[4], True, black, white)
    con_rect = condition.get_rect()
    con_rect.centerx = 500
    con_rect.top = 450

    image = pygame.image.load(img_links[4])
    image.set_colorkey((255, 255, 255))
    img_rect = image.get_rect()
    img_rect.center = (500, 540)

    screen.blit(condition, con_rect)
    screen.blit(high, high_rect)
    screen.blit(degrees, htemp_rect.topright)
    screen.blit(degrees, ltemp_rect.topright)
    screen.blit(high_temp, htemp_rect)
    screen.blit(low, low_rect)
    screen.blit(low_temp, ltemp_rect)
    screen.blit(image, img_rect)
    screen.blit(date, date_rect)

    update_time = "Last updated at " + datetime.now().strftime("%l:%M%P")
    last_update = tiny_font.render(update_time, True, gray, white)
    screen.blit(last_update, (5, 770))
   
    # AJR - try to output display instead of converting to raw.
    ### pygame.display.update()
    # AJR - and now save the image we've generated on the surface 'screen'
    pygame.image.save(screen, "output/test.png")
    #call(["/mnt/onboard/.python/eink_update.sh"])
    ###convert_to_raw(screen)
    ###call(["/mnt/onboard/.python/display_raw.sh"])
    
    
#try:
get_weather_data()
#except IndexError as error:
    #index_error(error)
The only irksome part I've not managed to work around is that the screensaver does not automatically update when the new copy of the image is sync'd. - I have to push the 'n' button on the nook to wake it, and then the new weather is displayed when it goes to sleep again (after 10 seconds).

Still - I'm happy :)


Sunday 27 January 2013

Nook - e-reader / Pi Display - Attempt 1.

OK - This isn't strictly related to the Raspberry Pi as such - at least not yet, but it was because of running the weather station software on the Pi that I bought the Nook. - I want to be able to use the Nook as a remote display, pulling data from the weather station's Raspberry Pi, and displaying it remotely - e.g. at my office for example, or stuck on the wall. - I've seen similar things with the Kobo etc.

So, this is step 1. - So far, I've got the Nook, I've rooted it, go the market working, installed Kindle application and then got sidetracked into reading 'Life Of Pi'.

 - The Nook (Nook simple touch) seems easier to hold / read on than my kindle keyboard - It seems lighter, and easy to turn the pages with just a touch rather than using the side buttons as on the kindle, but that's maybe just personal preference.  I can't see me buying much from the B&N Nook store as I've really committed to Kindle previously, although by using Calibre with a decoder, I could install my purchased items onto the Nook. - I didn't want to be tied into one platform in the long term......  Anyway. - This post is just for notes on the install so far.

1. - Register the Nook with B&N having turned it on for the first time and setup connection to Wi-Fi etc.
2. - Download and follow instructions to create an SD card with NookManager on. - See this post - http://forum.xda-developers.com/showthread.php?t=2040351
3. - Reboot the Nook with the NookManager and BACKUP the Nook before proceeding.
4. - Transfer the backup from the SD card somewhere safe.
5. - Repeat steps 3 and 4 so we know we have a backup from which we can hopefully recover the Nook should the root fail but nothing guaranteed.
6. - Reboot with Nook Manager and 'root' the device.
7. - Reboot and verify it's all working OK.
8. - Reboot with Nook Manager - make a backup at this stage.
9. - Install NTGAppsAttack as per - http://forum.xda-developers.com/showthread.php?t=2086582
10. - Assuming this has worked OK we now have market etc installed.
11. - Reboot and install Kindle app from the market. - I've now got that installed and working fine with my Kindle books.

... to be continued.

(28/1/13) Much googling, and playing with the screensaver on the Nook shows that what may be possible is to create a directory on the Nook 'screensavers' directory e.g. /screensavers/weather.  Then, mount the Nook (in this case on the RPi), copy an image file into the 'screensavers/weather' directory on the Nook, then remove the nook. - The Nook will timeout and if you have it to display files from the 'weather' screensaver - Brilliant - it doesn't even need rooting to do that.

Then, I found this from the Kindle. - http://www.mpetroff.net/archives/2012/09/14/kindle-weather-display/
Currently trying to get the script running on the Pi to create the image which can then be transferred to the Nook.  Problem I've hit so far is that I can't 'rescan' the USB bus to remount the Nook periodically.  - When it's 'removed' from the Pi, the device files are removed. - I can't find a way to get udev to think it's been replugged ..... yet.


I can force 'removal' of the Nook on the USB with :-

echo 1 > /sys/bus/usb/devices/1-1.2/remove

But can't find a way to remount. - The device files are gone, and I don't know how to rescan the bus to force recognition as if it has been replugged.


I didn't originally want to do it this way because it involves a dedicated Pi for the purpose. - Originally I'd wanted a script running on the Nook which would pull the image from the Pi. - However, I'm a simple guy and I like simple solutions, so this is good enough if it works. - The upside is that I only need one power supply - the Nook charges OK from the Pi. albeit slowly. - I'll just have to get another Pi to use as my general purpose Pi if this project works :)

TBC....

OK - now looking for the essentials for the Kindle scripts. - namely pngcrush and rsvg_convert. - pngcrush was easy:-

apt-get install pngcrush

rsvg_convert took a little more finding, but I think is provided by librsvg2-bin

apt-get install librsvg2-bin

Run the scripts from the above link and put the resultant image into the screensaver directory on the Nook, et voila:-


OK - It's not entirely what I want, but as a proof of concept it pretty much works OK.  Now I just need to know how to rescan the USB to remount the Nook at intervals....

(Update 3/2/13.) - I could find no way of re-establishing / rescanning the USB devices once one had been removed. - I don't mean just re-mounting a filesystem here, I mean once the device files are removed, I could find no way of getting them recreated / re-establishing as if the device had been replugged. - Anyway, see next post, I'm now getting the file via ftp periodically which is a better solution anyway.


Monday 21 January 2013

Backup

OK - So I've now got a couple of PC's backing up / syncing with the RPi attached to the USB hard drive. - I need to be able to quickly rebuild if this machine fails. - Since I originally created the hard drive filesystem with rsync, it seems reasonable to use this to backup to another machine on the network.  The following command copies the filesystem, and I should be able to use this to recreate the RPi / USB disk system relatively quickly if it fails.  I've also taken a copy of the SD card on a windows machine, so the rebuild procedure will be to recreate a new SD card, if necessary, add a new USB hard drive, and then use my rsync backup to restore the hard drive filesystem. - I really should test this!

~/weatherpi $ rsync -avxS root@weatherpi:/ ./backup --exclude "/My Documents"

The RPi with the hard drive seems to run 'Unison' very well - much more quickly than my old 'Buffalo Linkstation' at any rate.  It initially took around 10 hours to copy >100Gb of data from one PC to the USB attached drive, but did so with no hiccups.  I have then brought in another PC which only took about 20 minutes to do the initial synchronisation of the same data. - We have several PC's around the house, but all the user data - music, photos, documents etc is stored in /My Documents on each, regardless whether they are linux or windows.  Then, I use unison to sync the data across the various machines by syncing with the Rpi box.  For the Linux boxes, I can pretty simply rebuild from a CD.  For the windows machine, I also keep a 'Macrium' backup copy on a portable USB drive.  This strategy means that I have several copies of my data which should provide some resilience against a lost disk etc., and protection against failure on the windows system with a full byte copy of the disk.  (I have tested recovery of this also when I've swapped disks in the machine).

Saturday 19 January 2013

Raspberry Pi - home server.

So, I've now got a Raspberry Pi which is running from a usb attached hard drive, and it's running weather monitoring software, and a web server which will be 24x7, but generally just for personal use, what else can I do with this?  - Well, for probably the last decade I've been running a 'Buffalo Linkstation' NAS.  This was hacked to allow it to run 'Unison' backup software, 'squid' proxy, and 'Twonky' media server. - It has done this very well during this time, but it's running out of space, and IDE drives are obsolete and expensive.....  The RPI should be able to handle these tasks pretty well.

- Unison was simple. - 'sudo apt-get install unison2.27.57'
sudo unison -socket 1966 >/dev/null 2>&1 &

In due course I will put in an auto startup script so that it's always running, but currently it's duplicating my 100Gb of photo's and music from another PC. - This takes time the first time it's run, but subsequent sync is quick.

- Twonky is available for the Pi - see here http://www.raspberrypi.org/phpBB3/viewtopic.php?f=66&t=11866  This details relatively simple install. - I need to wait for my media copy to complete before I can configure, but it runs, scans directories, and is visible to the media player devices on my network, so it all looks good - only downside is I might need to buy a new license since the one for Twonky 3.1 on the Buffalo which I bought many years ago is not valid on this install. - Again, I also need to add this for auto startup when I'm happy with the setup, but it looks good.

USB Hard drive

Finally, I got to do something useful with the Pi.  This is one of two useful uses so far :)   For Christmas I got a weather station from 'Maplin' with a USB interface, and it was relatively straightforward to get the Pi pulling data from this and publishing it in a web page using pywws.  I found relatively little on the web about running the Pi from a hard drive, so these notes are my reference if I need to rebuild this. Most of the info can be gleaned from the thread here, though it's a little fragmented and somewhat incomplete - http://www.raspberrypi.org/phpBB3/viewtopic.php?f=26&t=9117

- It is not possible to 'boot' the Pi from a hard drive - it needs to boot from the SD card, but it can 'run' from a USB attached hard disk. - I was worried that a 24x7 system would quickly wear the SD card, especially as the weather station software processes data and writes to files every minute or so.  This, and improved storage - perhaps as a backup server was my driver for using the external hard disk.  (NB - in fact, the weatherstation died after 4 days and I'm currently waiting on a replacement, but at least I have the logging / publishing framework in place).

 - Experiment with several 2.5" external USB disks show that the Pi doesn't provide enough power to run them without external power supply, but I had a spare 320Gb 3.5" external drive with it's own power which I've now dedicated to this.

- Step1 was to install the system from an image onto the SD card, and get it running and configured with all the software I wanted (primarily pywws and lighttpd), get networking etc configured to my satisfaction.  Now, make a backup of the working SD card before proceeding!

2. - Plug the hard drive into the Pi and create the partitions I needed using fdisk - In my case, I just created a single large partition which will later be the 'root' of the Pi, and a 256Mb swap partition:-


root@weatherpi:/etc/lighttpd# fdisk /dev/sda

Command (m for help): p

Disk /dev/sda: 320.1 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders, total 625142448 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xfa24485f

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048   624617471   312307712   83  Linux
/dev/sda2       624617472   625141759      262144   82  Linux swap / Solaris


3. - mkswap /dev/sda2
4. - mkfs.ext4 /dev/sda1
5. - mkdir /media/sda1
6. - mount /dev/sda1 /media/sda1
7. - cd /
8. - rsync -avxS / /media/sda1 --exclude /media/sda1
9. - cd /media/sda1/etc
10. - update the path of root in fstab to be /dev/sda1 instead of /dev/mmcblk0p2.
11. - cd /boot
12. - edit cmdline.txt and change the root= to /dev/sda1 from /dev/mmcblk0p2.
13. - Cross fingers and reboot.....  I was lucky, and the Pi booted first time and I could access so I didn't have to faff with connecting to a monitor / keyboard etc.

Check that it's actually all configured the way I expect with the huge root filesystem:-

root@weatherpi:/etc/lighttpd# df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          294G  1.7G  277G   1% /
/dev/root       294G  1.7G  277G   1% /
devtmpfs        117M     0  117M   0% /dev
tmpfs            24M  220K   24M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            47M     0   47M   0% /run/shm
/dev/mmcblk0p1   56M   17M   40M  30% /boot

All looks good. - My SD card should live a long time as all the 'wear' activity is now on the hard drive.

14. - Now we've got it all booted, remember about the swap partition.....  edit fstab and add in the swap:-

    /dev/sda2       none            swap    sw                 0      0

Now, 'swapon -a'
'free' should now show the swap:-

pi@weatherpi /etc $ free
             total       used       free     shared    buffers     cached
Mem:        237868      53164     184704          0       6808      28580
-/+ buffers/cache:      17776     220092
Swap:       262140          0     262140


Great - final reboot to check this all works OK then I'm happy.


15. - I should also have disabled the auto generation of a 'swap file' now that we have a proper swap partition. - sudo update-rc.d dphys-swapfile disable  Might also remove /var/swap having done swapoff /var/swap. - The auto swapfile generation was running prior to this and it's not necessary.