Showing posts with label open source. Show all posts
Showing posts with label open source. Show all posts

Tuesday 28 September 2021

Yet Another New Home Server

This year has seen me doing more in the way of little tech projects at home than I have done for a while, perhaps due to covid lock downs so if that's the case then I'll take this small positive from an otherwise rubbish situation.  Typically for me, these projects have focused around open source projects and some IoT.  More on those in some separate blog posts when I get around to writing them up.  But for now, I wanted to make some notes on my new home server set up.

I've had an array of different low powered home servers of the years that I've previously written about, namely the NSLU2, TinyTuxBox, Joggler and for the past many years a simple ReadyNAS box that I specifically bought for the Intel processor as it made compiling different bits and pieces a whole lot easier back in the day.  However, I have recently relegated the ReadyNAS box from home serving duties, keeping it only for its native NAS services because using it for other things has become increasingly difficult without updating the entire base OS (which is possible by I'm reluctant to do) due to down level software libraries like an ancient version of openssl.

In with the new then and I moved away from Intel architecture as it's now so much easier to compile for Arm chips and went with the, wait for it, drum roll, rather obvious choice of a Raspberry Pi 4.  Specifically, a Pi 4 Model B, 4GB.  I've paired it with the official Pi case power supply, micro HDMI cable and shoved in an A2  SanDisk Extreme 64GB SDXC card.

And so to the notes, my initial target for this new box would be as follows:

The Lounge

IRC might be a bit old hat but tons of open source project still use it for their more synchronous communications.  ZNC is the choice of old for staying connected to your IRC channels.  For those not familiar, it acts as a relay to the IRC servers you want to connect to.  Effectively, it connects as your IRC client to the servers and presents your local IRC client with an endpoint through which you can connect.  This allows you never to miss any messages and see the IRC conversation even when you're not actually online.  Matrix seems to be taking some of the old IRC community's attention with various projects setting up bridges between Matrix and IRC.  However, the relative newcomer project called The Lounge shows just how far web technologies and web sockets have come.  It's a darned site (pun intended) easier to install configure and use than ZNC so I'm a massive convert and big fan of the project.

The project is relatively stable in the master branch and doesn't release particularly often so I've open for the run from source approach to take advantage of all the latest development.  Other than that, I've only made 3 changes to the default configuration prior to starting up my The Lounge server:
  1. host: "127.0.0.1"
  2. reverseProxy: true
  3. theme: "morning"
As you can see, these are all pretty simple and somewhat trivial changes.  The host setting binds the listener to the localhost interface, thus making it suitable for use with a reverse proxy and not exposing the service outside of the Pi 4.  The reverseProxy setting tells the server it's expecting to run behind a reverse proxy (the clue is in the name I guess).  Finally, I've switched to using a dark mode theme rather than the default light mode.  That's it, the remainder of the configuration is all about which IRC servers and channels to connect to along with the usual IRC bits of registering your nick and logging into the nick server.

Mosquitto

This is even simpler to get going than The Lounge due to the fact it's bundled with Raspbian so you can just apt-get install it.  I've created a configuration based on the bundled example config file but changing:
  1. pid_file (probably just because I'm old fashioned like that)
  2. user (to drop privileges)
  3. listener (to specify a port number)
  4. certfile and keyfile (for SSL)
  5. log_dest (create a specific log file for the broker)
  6. clientid_prefixes (a bit of added security to only allow certain client IDs to connect to the broker)
  7. allow_anonymous (quite an important one!)
  8. password_file (so that connections are authenticated)
Hopefully, that gives me something secure as well as providing me with the broker functionality that I need.

Node Red

Again, simple to install as it's bundled with Raspbian.  It does like to run under the default "pi" user though, which is a bit of a shame security wise.  All I've done to the configuration is ensure it's listening only on the local interface and enable the adminAuth section such that I'm required to enter a user name and password to access the user interface.

NGINX
 
Another simple install due to using the bundled version that comes with Raspbian.  However, this time around there's a lot more configuration to do since I'm using it to front a reverse proxy onto The Lounge and Node Red.  This gives me a few advantages such as being able to restart NGINX in order to load new SSL certificates without interrupting the underlying services i.e. something like IRC can stay connected even though new certs are loaded.  Both The Lounge and Node Red support SSL in their configuration so this also means I only need to configure certificates in one place and have a single route through which I can access all my home services.  The idea and bulk of the configuration for doing this comes directly from one of the guides available for The Lounge.

server {
    # redirect HTTP traffic to HTTPS
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 301 https://$host$request_uri;
}

server {
    # SSL configuration
    #
    listen 443 ssl default_server;
    listen [::]:443 ssl default_server;
    ssl_certificate /path/to/your/server.crt;
    ssl_certificate_key /path/to/your/server.key;

    server_name your.server.name.com;
 
    # Add this if you want to do web serving as well
    root /var/www/html;
    index index.html index.htm;
 
    location / {
        # First attempt to serve request as file, then
        # as directory, then fall back to displaying a 404.
        try_files $uri $uri/ =404;
    }
 
    # Configure reverse proxy for The Lounge
    location ^~ /YOUR_PREFERRED_IRC_URL_GOES_HERE/ {
        proxy_pass http://127.0.0.1:9000/;
        proxy_http_version 1.1;
        proxy_set_header Connection "upgrade";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;

        # by default nginx times out connections in one minute
        proxy_read_timeout 1d;
    }

    # Configure reverse proxy for Node Red
    location ^~ /YOUR_PREFERRED_NODERED_URL_GOES_HERE/ {
        proxy_pass http://127.0.0.1:1880/;
        proxy_http_version 1.1;
        proxy_set_header Connection "upgrade";
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;

        # by default nginx times out connections in one minute
        proxy_read_timeout 1d;
    }
}
 
Let's Encrypt
From Wikipedia: "Let's Encrypt is a non-profit certificate authority run by Internet Security Research Group that provides X.509 certificates for Transport Layer Security encryption at no charge."

The model for using letsencrypt is pretty simple.  They sign your SSL certificates, free of charge, but their signing expires within 90 days.  Hence, they're encouraging a high turnover of certificates by regular renewals.  This means that realistically you need to automate the process of certificate signing.  To do this I'm using the getssl script which makes life extremely easy when coupled with a cron job to kick off the script on a regular basis.  I'm running it every day and the script decides whether to replace my existing certificates.  It all sits there quite nicely running in the background and doesn't get in the way at all, restarting NGINX only when a new certificate is put in place.  Due to the fact that NGINX is decoupled from the services it is proxying the other services aren't interrupted.

Wednesday 2 December 2020

Open Sourcing a NetworkManager VPN Plugin

It's not every day I find myself publishing a new project to open source and even less so when that requires release approval at work.  I hope, over the years, I've written some useful bits and pieces and this time around I was keen to publish my work on the Internet rather than internally within the company.  This requires following due process of course and seeking the relevant approval for the publication to take place.

Fortunately, in the right circumstances, IBM are very amenable to releasing code to open source.  I was convinced enough that a NetworkManager plugin to add to the existing list of VPN plugins would not conflict with the business that an open source approval would be fairly trivial.  Happily, I was correct, and going through the process wasn't too arduous with a few forms to fill in.  These were, of course, designed much more for bigger releases than I planned so vastly over-engineered for this particular release but at least due diligence was applied.

On to the project and the code.  It's not a world-changer but a small VPN plugin for NetworkManager to drive Cisco AnyConnect and made available as NetworkManager-anyconnect on GitHub.  So I now know more than I'd care to mention about the inner workings of NetworkManager VPN plugins.  They're not very well documented (hardly documented at all in fact) so they're quite hard work to produce by looking over existing code in available plugins.  I started off from the OpenVPN plugin which turned out to be a mistake as the code base is vastly bigger than that required for a plugin as simple as the one I wanted to write.  Were I to start again, I would recommend starting from the SSH VPN plugin instead as this is actually very nicely set out and doesn't include a lot of the shared bloat that comes with other plugins that are formally a part of NetworkManager.


Friday 18 January 2019

Self-Signing SSL/TLS Certificates

Things have changed a bit since I last looked into setting up a Certificate Authority (CA) and using that to self-sign my own certificates, not least that the use of the Common Name (CN) field appears to have changed. Chrome in particular seems to insist on the use of the Subject Alternative Names (SAN) extension rather than (or in addition to) using the CN field. So these are my notes on how to set up your own CA and use that to sign certificates. I'm conscious this is bound to go out of date so at the time of writing I'm working with Firefox 64, Chrome 71 and OpenSSL 1.1.1.

Setup
First of all, create a config file along the lines of the following and call it anything you like but for these notes I'm going to call it ssl.conf.  Note, if you want to you can start with a different template or look at your own openssl.cnf file which on Linux is commonly found at /etc/pki/tls/openssl.cnf.

[ req ]
default_bits       = 4096
distinguished_name = req_distinguished_name
req_extensions     = req_ext

[ req_distinguished_name ]
countryName                 = Country Name (2 letter code)
countryName_default         = GB
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = England
localityName                = Locality Name (eg, city)
localityName_default        = MyCity
organizationName            = Organization Name (eg, company)
organizationName_default    = MyOrg
commonName                  = Common Name (e.g. server FQDN or YOUR name)
commonName_max              = 64
commonName_default          = localhost

[ req_ext ]
subjectAltName = @alt_names

[alt_names]
DNS.1 = localhost

You can change any of this template and indeed you'll need to change the common name for the certificates you're generating. The CN can be changed either on the command line during certificate creation or by changing the default in ssl.conf. You will also need to change the list of names under the "alt_names" section, this list should contain one line for each host name your machine might be known as. The list starts at DNS.1 for the first entry, then you can add DNS.2 for the second entry and so on.

NOTE: the specification and a lot of the documents available in this space indicate that an IP address can be used in the CN.  My testing seems to indicate that while this is the case, certificates produced in this way will be rejected by modern browsers.  Hence, you should list only hostnames as the CN but IP addresses still appear to be acceptable in as "alt_names".

Create a Certificate Authority
You'll need a certificate and key file to act as your own CA:

openssl genrsa -out RootCA.key 4096
openssl req -x509 -new -nodes -key RootCA.key -sha256 -days 3650 -out RootCA.pem -config ssl.conf

You can inspect the certificate with:
openssl x509 -in RootCA.pem -text -noout

Create a Certificate Signing Request (CSR)
Now you have a CA you can create a CSR that can be used with your CA certificate to generate a client certificate:

openssl genrsa -out server.key 4096
openssl req -new -key server.key -out server.csr -config ssl.conf

You can inspect the certificate with:
openssl req -text -noout -verify -in server.csr

This time it's really important to ensure your host names are listed under the "X509v3 Subject Alternative Name" section of the certificate.

Generate a Signed Certificate
You can now use the CSR to create a signed certificate that can be used to serve up content over a secure connection:

openssl x509 -req -in server.csr -CA RootCA.pem -CAkey RootCA.key -CAcreateserial -out server.pem -days 3650 -sha256 -extensions req_ext -extfile ssl.conf

Note: if you want to create a different format of certificate here you can simply replace server.pem in the above command with something like server.crt, for example.

You can inspect the certificate with:
openssl x509 -in server.pem -text -noout

Again, it's really important to ensure your host names are listed under the "X509v3 Subject Alternative Name" section of the certificate.

Use the Certificate Server Side
You can now put your server.pem and server.keyfiles to work and serve up content over a secure connection.  There's too many ways to do that to list here but it could be used with a web server to serve HTTPS or a websockets server to serve some sort of socket connection as a couple of examples.

If you want to inspect the certificate that's being used on the server, replace <hostname> and <port> in the command below:

openssl s_client -connect <hostname>:<port> | openssl x509 -noout -text

Use the Certificate Client Side
My use case here is with a web browser and so you'll want to import your <i>RootCA.pem</i> into your browser environment.  There are two main ways of achieving this, you can either:
  1. Import directly to the browser
  2. Import to the key store on your operating system
It's quicker and easier to import directly to the browser but this will of course only cover that one browser on your system whereas if you use the operating system method then any application that consults the OS for certificates will see your CA certificate.

For Firefox, go to "View Certificates" in the preferences; click the "Authorities" tab and then the "Import" button; select your <i>RootCA.pem</i> file and click OK.

For Chrome, go to "Manage Certificates" in the settings; click the "Authorities" tab and then the "Import" button; select your <i>RootCA.pem</i> file; click the check boxes to trust the certificate and click OK.

Sunday 21 October 2012

HTC Desire Cyanogen Mod

I have the HTC Desire phone that to all intents and purposes is a decent phone even by todays standards and was an excellent phone when it was released some 2½ years ago at the time of writing.  That is, with the rather fundamental flaw for a phone that boasts an App Store - a complete lack of internal storage space.  It was built with 512MB flash storage which after HTC had muddled with it was cut to a mere 150MB for users to install applications.  That wasn't so bad back in the day, but these days you just have to install a standard set of Google apps and a few other utility applications and you're done, no more room.

The plight of the HTC Desire was giving me enough grief that I decided to do something about it by flashing CyanogenMod on it.  This like any third-party phone hack is a risky business but I figured if it failed miserably I could always restore the original HTC firmware or more likely get rid of the phone and move to something like the Samsung Galaxy S3 instead.  It turns out Cyanogen is quite the saviour of the HTC Desire and since the instructions for installing it were somewhat less than clear I wrote down what I did based on a Cyanogen forum post and a wiki page, so here it is...

I re-iterate what the forum post states:

#include <std_disclaimer.h>
/*
* Your warranty is now void.
*
* I am not responsible for bricked devices, dead SD cards,
* thermonuclear war, or you getting fired because the alarm app failed. Please
* do some research if you have any concerns about features included in this ROM
* before flashing it! YOU are choosing to make these modifications, and if
* you point the finger at me for messing up your device, I will laugh at you.
*/

Backup
I found it really useful to take a backup of things that were on the phone and put them to one side for safe-keeping both in case something went a little wrong but also to help with restoring the phone when running the new OS.  There are utilities you can use to backup your contacts (or sync them with your Google account), backup your SMS messages, even backup your saved games if you're that worried about carrying over your Angry Birds records to the new phone.  I also took a complete copy of the SD card so I had all the pictures and other things that had accumulated on there as well.

My Phone
BRAVO PVT1 SHIP S-ON
HBOOT-0.93.0001
MICROP-031d
TOUCH PANEL-SYNT0101
RADIO-5.11.05.27
Aug 10 2010,1752:18

Why don't you just S-OFF?
All the instructions talk about S-ON and how to go about gaining S-OFF without really saying anything about what this is.  It turns out that HTC implement a self-protection feature on the phone that is designed to prevent phone hacks and third-party installations that they call S-ON (presumably security on?).  With this switch set in the "on" position your phone cannot be modified.  Unfortunately for HTC they left a security flaw on some of their devices that means this switch can be altered to the "off" position.  Hence, the first thing that must be done to an HTC phone is use some software to take advantage of this and set S-OFF.

Use Revolutionary to set S-OFF
You need to find out some information about your phone similar to the info I've put above.  This will be different between models but for the HTC Desire you simply turn the phone off, then turn it on again while keeping the volume-down key depressed until the information is displayed.

The guys responsible for the software at revolutionary.io insist that you give them some information on your phone which turns out to be a sensible idea since they can attempt to ensure that this beta software doesn't screw up your device or that if it does they can prevent it being installed on similar devices in the future.  You simply go to their web site, fill out a form and download the software.

Put the phone into USB Debugging mode (Settings -> Applications -> Development -> USB debugging) and run the software.  It needs to be run as root under Linux.  It does it's thing and then reboots the phone, you'll now see you've got S-OFF set instead of S-ON if you view your phone information with the volume-down button trick once again.

One other note about revolutionary, being an open source enthusiast, is that they don't provide the source code.  Doing so would expose the security flaw that the software takes advantage of and thus would allow HTC to patch their phones accordingly and stopping this sort of modification instantly.

Use ClockworkMod to Back Up your current HTC firmware
Revolutionary gives you the option of installing ClockworkMod which is definitely worth doing.  ClockworkMod is an improved ROM manager and boot loader with some more advanced features over the stock HTC loader that comes with the phone.  One of these advantages is the ease of which a full backup of the phone's flash memory can be taken.  This appears to be called a NANDroid backup if you come across that term.  I assume that it's essentially just booting into a small program from the boot manager without accessing/mounting the flash device such that the entire flash device can be consistently backed up - a bit like a partimage or dd copy of an unmounted hard disk or partition under Linux.

Boot the phone into the bootloader (this is the same volume-down while holding power when the phone is turned off trick).  Use the volume keys to navigate up/down the menus and the power button to select the desired menu item.  You need to head for the "recovery" option which will take you through to ClockworkMod.  Select the "backup and restore" option to create a backup of the phone as it stands right now - this will give you a fighting chance of restoring the phone as-is should something go pear-shaped or for any other reason you decide to return to the stock HTC firmware.

The backup is placed on the SD card so you need to boot the phone again, access the SD card from your computer with a USB cable and store the backup image safely away somewhere - this is really important since the SD card will be wiped later along with your backup unless you copy it somewhere else first!

Get CyanogenMod Binaries onto the Phone
I grabbed the following files but there may be later versions now:

The first of these is the CyanogenMod package that will be your new operating system on the phone.  The other two packages are Google apps installed on top of the new OS.  I forget why you need the Google ones, but presume you can't get them from the Play Store on Android.

You'll need to copy these into the root directory of the SD card, still using the stock HTC firmware.

Install CyanogenMod
Boot the phone back into the recovery mode with the volume-down trick and selecting the "recovery" option to access ClockworkMod once again.  This time select the "Wipe data/factory reset" option and let that do its thing.  Then select the "Wipe cache partition" and let that complete as well.  Then you need to select the "Choose Zip from SD card" option and select the CyonagenMod zip file.  This will install the OS onto your phone after which you can choose the "Choose Zip from SD card" option again to select the Google apps zip and once again for the other Google apps.

During the installation you should create an ext data partition on the SD card that will be used a little later after the installation has completed (keep reading).  I have the stock 4GB class 2 micro SD card that came with the HTC Desire still in my phone so I opted to carve that up to use 1GB for data and keep 3GB free for use as a normal Android SD card partition.

Reboot
Reboot the phone and simple as that, you've got rid of HTC and you're booted into something that feels a lot closer to what you get on a Samsung phone i.e. a fairly stock version of Android which in this case is is 2.3 Gingerbread.

Install and Configure S2E
This app is the primary reason for installing Cyanogen, it's not the OS that's so much the saviour of the HTC Desire but the fact that using an app such as S2E (simple2ext) you can format part of your SD card for use as internal storage and thus completely resolving the tiny storage problem.  S2E is only compatible with Cyanogen and as far as I'm aware you can't do anything equivalent in the HTC stock firmware.

Once the phone has booted into Cyanogen for the first time, just fire up the Google Play store and find the S2E app.  Install it to the phone and run the configuration part of the app.  This lets you pick which parts of your installation you want to move to the ext partition you created on the SD card.  You can move over whatever you think you need to and when you're done just reboot the phone once again.

The S2E configuration will have installed a little script into the boot sequence of the phone such that on this reboot (i.e. the next reboot after running the S2E configuration) the script run and move the data you have selected onto the SD card.  It will only do this each time after you've run the S2E configuration and made changes.  These reboots can take quite a while since you could be moving quite a lot of data from your internal storage onto the SD card so be patient while it completes and boots back into Cyanogen.  Once you've done that you're good to go with using the phone again.

Final Thoughts
Having worked through all that, you'll need to tailor your phone to how you like it once again.  Restore the backed up SMS messages, contacts, files, etc.  You'll need to completely re-download any apps you liked on the HTC, configure those, set up widgets, etc, etc.  All standard stuff really, except this time you'll actually have some space in which to install things.

I'm currently using Cyanogen 7.2.0.1 which seems stable enough, it has crashed the phone a couple of times, just after I installed it but now it seems to have settled down and is running absolutely fine.  Running apps from the SD card using S2E can seem to make the phone rather laggy too so tweaking the settings in the S2E configuration to give a good balance of data stored on the SD and on the internal memory seems to be wise, along with tweaking other settings such as the read buffer.  I dropped the read buffer from the standard 2MB down to 512KB and that seems to have removed the lag.  Another way of reducing lag would be to update the SD card, say to one of the latest class 10 cards in which case it'll probably be quicker than the internal memory anyway!  I've not done that as yet and seem to be running just fine as I am for now.

Friday 24 February 2012

Autoqueue

A nice little piece of code I came across quite a while ago and have been meaning to blog about ever since I wrote some code to extend it properly to the Rhythmbox music player is a generic cross-player autoqueue project.

The idea of the project is to generate interesting tracks to queue up automatically in your player while you're listening to music based on what you're currently listening to.  Put simply, seed your playlist with a few tracks of the sort you want to listen to, turn the plugin on and it will continue to populate your queue with similar tracks.  It's great when all you want to do is have some non-particular music in the background (say when you're coding for example) and you don't want to bother with managing what's playing right now.  If you're in the mood for some acoustic through to heavy metal, just seed with what you want and let autoqueue do the rest.

The Rhythmbox plugin for this project wasn't particularly mature when I picked it up so I've modified it so now I've got a nice little button in my player that I can use to switch the plugin on or off depending on whether I want to choose the tracks or allow the plugin to do it for me.  Autoqueue attempts to be generic about the players it supports by essentially providing a library that player plugins can talk to over a dbus interface.

The autoqueue project itself currently supports 2 mechanisms to determine track similarity.  There's what the project seems to hold as the primary method which is comparing the audio signal of the track you're listening to with those in your music library.  It uses an external library called mirage for this which was originally written for the Banshee player.  Then there's the method I prefer to use which is to call out to last.fm to ask for the similar tracks to the current one playing and queue up one from the list you already have in your library.  Eventually, I'd like to extend this so it takes into account your last.fm profile such that the similar tracks picked are ones you prefer to listen to.

While it's a great little project and I love this type of idea for generating dynamic playlists, the project is fairly stale, not much activity in development and the code repository is way more up-to-date than any released zip files.  However, if you're interested it's easy to get going so long as you know how to write a plugin for your player.

<edit>See comments from the project maintainer below for latest updates on project status at the time of writing.</edit>

Wednesday 13 April 2011

Rhythmbox Artist Prefix Plugin

I've recently started using the Rhythmbox music player.  It seems pretty simple to use and like other Gnome applications is actually quite powerful under the covers but hides a lot of the power from basic use.  You sometimes have to dig a little to find a feature you're looking for.  That said, with no amount of digging was I able to find a feature to enable artists to be sorted while ignoring certain prefixes.  This is a pretty bulk standard feature of most music players and allows artists such as "The Beatles" to be sorted under "B" for Beatles rather than "T" for The.  There's quite a lot of discussion to be found on this in various Rhythmbox bug reports and on the mailing list.  The view of the developers is that it's not possible to automatically provide a one size fits all solution so they implemented the ability to allow users to manually add a sort tag to each track.

I've just written a first versions of a plugin I'm calling Rhythmbox Artist Prefix which allows the user to choose whether to have Rhythmbox attempt to automatically sort artists ignoring certain artist prefixes.  If you use Rhythmbox then give it a try!

The plugin works by querying the Rhythmbox database for artists with the given prefixes and that don't currently have a sort order defined (which allows the user to manually override the sort order derived by the plugin).  So long as the plugin is active it will watch the database for changes too.  The first time you run the plugin it will automatically add an entry to the sort order of all tracks returned by the query and if you leave it running then any time Rhythmbox finds new tracks matching the query their sort order will be updated as well.  Whenever the plugin is notified of a track by an artist such as "The Beatles" and that track doesn't already have a sort order, it will chop off "The" from the artist name and add the remainder (in this case "Beatles") to the artist sort order property for that track.  Quite simple really and I'm amazed it hasn't been done before.

Tuesday 25 May 2010

Pre-built Joggler Images

It seems everyone buying a Joggler is doing so for their own good reasons with thoughts about how to hack the device to their needs. I include myself in this too, why else write these blog posts? The majority of people aren't so interested in the hacking side of things as simply getting the device to do what they want and that's where using a pre-built, already hacked, operating system image can come in very handy indeed.

The Images
Easily the two most popular images are Ubuntu based although there are Joggler communities for Android, Mer and Meego, all of which are Linux based platforms. I've only tried the first of these but can lend my recommendation to either of

As I said, both are Ubuntu based with the key difference being the first image uses the Ubuntu Netbook Remix (a.k.a UNR, Ubuntu Netbook Edition and UNE) while the second one uses "normal" Ubuntu.  You'd perhaps expect them to be in fierce competition with each other but in good open source spirit quite the opposite is true. Both are produced by nice guys who help each other (and the rest of the community out) by sharing what they do.

There's no sense in me reproducing how to download and use either of these images.  Both come with pretty bullet-proof instructions on what to do.  I would, however, heartily recommend you produce the images from a Linux system having recently helped a friend struggling with downloading the images successfully and finding the right tools to do the job under Windows.

Fair's Fair
From my own investigations and messing around with putting Fedora on the Joggler I can certainly vouch for the amount of time and effort these guys have put into producing such good quality, usable hacks.  So, if you do more than just evaluate these for your own use i.e. you really are using your Joggler with either of these images then do give serious consideration to spending a few quid giving something back to the guys as they ask through a small donation.  I have no idea how much they might stand to make through donation and other revenue such as the Google ads, but it's probably not a huge amount and is definitely well earned.

Modification
If you try the images out and there's something you don't like, fine, you don't have to stick with that particular quirk.  There's never going to be a one-size fits all approach so if you don't like, for example, the on screen keyboard, the user interface or something else then feel free to change it.  It's your system once you've downloaded and booted it after all.  There are a lot of, dare I say, complaints on the forums about things not being quite right.  If it's something major then let the guy know who produced the image, it will probably get fixed in their next version.  Also, there's a huge wealth of help and support out there in various places so try looking for someone who's already "been there, done that".

Subject to some of the Joggler Hardware Gotchas it is possible to use these images on devices much bigger and faster than a 4GB USB stick.  I've tried with an 8GB stick and USB hard disks from 80GB through 320GB and even one disk of 1TB.  Simply follow the instructions for either image to write to your larger device as if it were a 4GB stick.  Then use a tool such as gparted (again I recommend Linux simply because it's easy and the tools are readily available) to resize the second partition to be as big as you like.  If you move to a hard disk (rather than USB stick) then you can undo some of the modifications made in the images for efficiency and to try and protect your stick:
  • Change the file system back to ext3 from ext2
  • Add a swap file or swap partition
You can change the file system type from ext2 to ext3 either before or after changing the size of the partition.  Simply boot the image on your Joggler and run the command "tune2fs -j /dev/sda2" then edit the file /etc/fstab to change bit that says ext2 to say ext3, save that file and you're all done.  The reason for making this change is your files will be a little safer in the event of unplanned events such as sudden loss of power.  This was quite rightly turned off for the USB stick images because more data is written to the device and since USB sticks are only capable of being written to a relatively small number of times the trade-off between stick reliability and file system reliability has to be made.

Adding a swap file (if you don't know what this is then have a go at understanding paging)is probably easier in many ways than adding a swap partition and with a recent version of Linux such as the ones used in these images doesn't lead to any loss of performance as might have been expected in older versions of Linux.  Feel free to add a swap partition in gparted though, don't forget to add a line to /etc/fstab to tell Ubuntu to use this partition as swap.  To add a 1GB swap file simple run the command "dd if=/dev/zero of=/swap bs=1M count=1024; swapon /swap" then add a line to /etc/fstab so next time you reboot Ubuntu will use this file for swap space.

You can find a lot more of my information about the Joggler at my Joggler Index post. I also have a list of Joggler Bookmarks.

Thursday 4 February 2010

Contributing to BloGTK

I was meaning to talk about BloGTK before my previous post on the topic but never quite had the time so here's a quick follow-up to fill in the holes...

At the end of last year I decided it was about time I filled in a scripting gap in my knowledge, namely Python, so set about finding something to code.  I'm also a blogger (D'uh) and have always been frustrated at the lack of a decent off-line client for the Linux desktop.  Enter BloGTK, both a blogging client and written in Python.

Step 1 before thinking any further about contributing or reading any source code is to seek permission from my employer.  I was glad (and surprised) to find getting permission to contribute to open source projects is actually very quick and simple.  A short note to my manager and a couple of days delay later yielded the rather simple response:
Graham

Approved

Regards, ...

BloGTK didn't do a couple of things I consider essential for my blogging purposes, the main one being support for uploading and embedding images.  I wrote and submitted a couple of patches to the project, learning a lot about Python along the way.  The latest development snapshot of BloGTK (which will probably become version 2.1) includes support for uploading to Picasa and Flickr.  If you went looking for Flickr support as a result of my previous post then you'll either have to grab the development code early or wait a while until the release of the 2.1 version.  In the mean time I'll do battle with Flickr to find out why they think BloGTK is a personal web site or gallery in order to publicise the App through the App Garden.

Tuesday 2 February 2010

Flickr Appgarden Violation‏, really?

Today I unexpectedly received the following from Flickr in reference to BloGTK.
Hello,
 
The App Garden is a place for software developers to
showcase applications they've created. Because your app
doesn't appear to fit this description it has been set to
private.
 
Your API key is still active and fully functioning , it's
just not public in the App Garden.
 
The App Garden is not a place to showcase a personal
website or gallery. Please only publicize app pages for
applications that you have developed and that are related
to
the Flickr API. You can review the App Garden guidelines
here:
 
http://www.flickr.com/services/apps/about/
 
Thank you for your understanding.
 
Regards,
Flickr Staff
I hold Flickr in very high regard for their openness, interaction with the community and the great API.  This is also the reason I'm happy to sign up for a pro account vs other services I use heavily (such as Geocaching.com) which are a closed managed community.  So I was surprised to find this note in my inbox this morning which appears to be an automated message or at best the staff member simply hasn't looked at the application before criticising and taking action.

Friday 26 September 2008

Squeezebox Duet: My Impressions

Having recently had my little whinge about setting up the SqueezeBox Duet it's about time I said something about the box itself. Both professional and user reviews around the web are pretty much universally glowing with praise for it and I have to say to a large extent I agree.

First of all, there's the hardware which is sleek looking with a glossy black finish. The hardware is in two parts (ignore the odd cable here and there), the SqueezeBox Receiver:

Which, while not belittling the technicality of it, is to the consumer essentially just a dumb receiver box. It takes an audio stream over either a wired or wireless Ethernet connection and spits it out through your stereo using a digital or analogue audio connection. Then there's the controller:

Which is an altogether more exciting piece of kit. It's a wifi-device that can talk directly to your wifi router or be bridged onto your wired network via the Receiver if you don't have wifi. It features a rather strangely arranged set of controls that become familiar after a little usage. There is a jog dial with selection button, then buttons for playback, volume, control of the playlist and navigation around the menus. As you can see, it's got a nice full colour LCD display too. This means you can wander around anywhere within wireless range, control your music and get feedback directly on the device on just about anything you might want to know.

Software wise, things also come in two parts (well three if you count the firmware on the receiver). There's the software on the controller and some software to run on your PC which is listed as optional but I would consider very much essential. Both of these parts are open source with decent plugin interfaces and documented APIs you can access.

The controller runs a customised embedded Linux distributions known as Squeeze OS which is responsible for running the user interface application on the controller called Squeeze Play. The nice thing about this is you can take the user interface part, Squeeze Play, and run it on your desktop too. While the controller software is open source and pluggable, I haven't found anybody having written any useful applets to run within Squeeze Play yet but it's still early days so hopefully those people producing screen savers will come up with some good ideas for applets. That said, I can have a bit of a hack around too to see how easy it is to program for and whether I can produce something useful too.

The open source server software is called Squeeze Center and should be run somewhere with access to your local music collection. It supports a huge number of formats including flac, ogg and mp3 of course. Squeeze Center is hugely flexible and configurable, mostly through a bunch of supplied plugins. It provides a web interface as well as access to other interfaces so it can also be used to control the music playing as well as being the configuration hub for the entire setup.

Out of the box Squeeze Center provides access to your local music collection via a number of sorting methods (such as by artist, album, song, genre, etc) as well as providing standard playlist functionality. It also allows you to listen to Internet radio through a number of free and paid for services such as shout cast streams, mp3tunes and last.fm (a last.fm scrobbler plugin is also included). It gives you a favourites menu so you can tag anything as a favourite and later get quick access to it through this menu. Last but not least there's the Extras which doesn't do a huge amount out of the box (this is where most 3rd party plugins are accessed) but it does give you access to podcasts - you can subscribe to RSS or OPML podcast feeds and have them streamed to your hi-fi, cool!

I like to use dynamic playlists and that's some functionality that doesn't get included by default so I've found some plugins that do that for me now. Being a UK resident I want access to the BBC audio content and some kind person has written a plugin to access all their content too, including Live Radio streams, listen again streams as well as BBC podcasts. The final plugin I'm finding useful at the moment is one that automatically adds tracks to my music collection when I copy them to my music directory using a groovy Linux kernel feature, Inotify, otherwise you have to rescan your collection manually after adding new tracks. The plugins I've installed are tagged in Delicious.

Overall, the system seems to work very well. It's great having the flexibility of having an entire music collection at your finger tips ready to play, no messing with things that spin any more, and having access to all the additional content over the Internet is another real bonus. The packaging and interface of the components are really nice and are only set to get better over time, particularly as they're open source. This means I can get my hands on extra code in the form of those useful plugins as well as writing my own. But it also means when Slim Devices get caught up in adding support for new products or bug fixing as they inevitably will, new functionality will always be progressing at least through the community. So now I've got everything fixed up in my config, listening to music at home is quite a cool experience, happy days!

Tuesday 26 August 2008

When is open source not open source?

I've been having a pretty major headache trying to get my SqueezeBox Duet to do media streaming ever since I bought it many months ago. The SqueezeBox is also the primary reason for buying a low power home server that I've been blogging plentifully about too. I'll save the post about my trials with the SqueezeBox itself for the day when I resolve the issue; either with a fix or by sending the darned thing back for a refund. However, it's got me thinking about some other things along the way while I've been trying to solve the various problems.

We're all familiar with open source. It can have a pretty simple definition i.e. where source code for a piece of software is made available publicly. Dealing with Slim Devices which are now a Logitech company has sparked thoughts about just how many different types of open source there are and when open source crosses the line to no longer feel like open source even if I can still download the source code.

I'm very used to working with what I think of as standard open source projects. These are the ones that generally started with one person writing something small to solve a problem they had or because they didn't like other versions of software trying to do the same thing. The more mature of these are generally run by a person or small group of individuals who control the project for the benefit of the community of users in a not-for-profit fashion. Numerous examples spring to mind, not least the Linux kernel itself, but huge amounts of different software some of which get grouped e.g. those from the Apache Foundation, KDE, Gnome, Mozilla; and those that stand alone e.g. Pidgin, X-Chat, Samba, rsync; to name just a few that pop into my head.

I think where I'm going with this post is to look at what happens when open source stops being not-for-profit and these projects tread the line between this standard type of open source project and become something else. This is where I've got to in my experience with SqueezeCenter, but clearly this isn't the first for-profit open source software. I have no idea what the first is/was but there are definitely some prominent examples out there such as MySQL. Slim Devices employ developers to write and maintain SqueezeCenter which is central (although billed as optional) to the hardware they sell. In their case, the software is free to use and completely open source, it's written in Perl. The way this model changes the community is quite interesting.

In my experience so far, instead of having lots of developers donating their time to the project to learn, fix, maintain and progress the code base, developers are paid to do this. In the case of SqueezeCenter this seems to cut down the community to just those developers paid to do so. Why would other people contribute to something for which they could be paid, or for which other people are paid? Plenty of people get paid to write code for the Linux kernel but in this and other examples there is a distributed interest for that code rather than the single point of interest of one company. In spite of the fact I don't feel compelled to contribute to SqueezeCenter directly though, I still take great comfort from knowing it's open source and I think that's where the benefit to Slim Devices comes from. If I did have a problem, I could fix it myself, maybe sending a patch or possibly maintain my own personal patch set if I got that deeply involved.

So how many types of open source are there and when does open source no longer feel like open source? A good philosophical point for debate for which I think there is no real answer. There are so many reasons for using the open source methodology that it can bring benefits in so many different ways whether you're a developer, employee, company or simply a user. Each open source project tends to have a different feel to it, in my experience so far this has generally been governed by those who run or contribute to the project. However, there are clearly some other factors that might determine what it feels like to be part of a project that releases its source code. I'm sure the reasons for these are as numerous as the benefits they bring.