Tuesday 5 February 2013

Upgrading Fedora 17 to 18

It's been a while since I talked about upgrading Fedora but back in late 2009 and early 2010 I wrote about upgrading my home PC from one version of Fedora to the latest version at the time:

These were about the time (shortly after) the Fedora project introduced its latest piece of tooling, used for upgrading between distribution versions, called preupgrade.  Several years down the line and my PC and the more complex configuration than average that caused a few issues at the time have changed.  Preupgrade has been changed too, dropped in fact.  The Fedora project have moved away from it in favour of producing a new tool called Fedup (yes, yes, nice name).

It really was about time that preupgrade was replaced, as far as I know it was only ever produced as a short-term solution to upgrading Fedora without the need for booting from CD/DVD/USB/PXE and going through the Anaconda upgrade process.  It was a proof of concept if you like.  It's also been openly described within the project as "just a set of hacks" and other such non-complementary terms.  So why the need for something new?  Well I think that's already been best described elsewhere buy the guys behind Fedup so I'll just point you at a little background information on Fedup instead.

Having tried it first at work to upgrade a virtual machine from F17 to F18 I found that Fedup actually works pretty well, in fact to the user it doesn't really appear to do much differently to preupgrade.  You simply start the tool, tell it what to upgrade to, wait while it downloads the packages, then reboot to perform the upgrade, all done.  I recently tried it at home as well and it worked perfectly, no issues at all, which seems to be the experience of all the others users I've talked to about using it as well.  So far then it's a bit thumbs-up and thanks to the Fedup and Fedora teams for making it all so easy for us.

The one pitfall is that you can't upgrade straight from F16 to F18.  Traditionally with Fedora you can jump a version so you've always been able to upgrade directly from any current Fedora version that hasn't gone end of life yet to the very latest without going through intermediate versions.  Between F16 and F18 that wasn't possible though so if you're still on F16 you'll need to preupgrade your way through F17 before doing a Fedup to F18 if that's where you want to get to.  However, from F17 onwards Fedup should support the same version jumping as preupgrade did, it's simply that Fedup isn't available until F17 that causes this particular issue.



Tuesday 29 January 2013

Broadband Woes

At the end of November 2012 my until then rock-solid reliable Netgear DG834PN ADSL router started reporting being disconnected from the Internet. I know this because being a decent router it has the facility to email its logs to me periodically so I have a record of everything that goes on with my Internet connection going back for years. I eventually worked out that the disconnections were being caused by phone calls connecting to the house which can happen even when we're out because we have an answering machine. So then to work, on diagnosing whether the fault lies with my equipment or with one of the service providers (Plusnet using BT line).

First I tried connecting my Netgear to the master phone socket since usually it is connected to an extension running around the house. No joy. The Netgear exhibited exactly the same behaviour from both sockets and disconnected when phone calls came in or went out of the house (only on connection, not when the phone was ringing).

Next is to try some different micro filters. No joy. I was still getting disconnected in spite of trying a couple of different ones while connected to the master socket including a genuine BT micro filter I purchased specifically to get a decent quality one for this test.

Now I've got this far I'm starting to think it's a line fault and prepare to do battle with Plusnet to get them to assess my line and perhaps send out a BT engineer to fix the problem. I go on-line to their help system. Their phone system is OK being in the UK but you have to go through a million-and-one press this to do that options. In order to raise a fault you have to answer a bunch of questions to confirm you've made all the checks that I would take for granted. Master phone socket, tick. New micro filter, tick. New router, now hang on a minute here. To the best of my (comparatively extensive) knowledge this question isn't right, routers aren't effected by phone calls, the entire point of an ADSL micro filter is to filter the line so you don't get noise during phone calls.

It seems Plusnet would not allow me to raise a line fault with them unless I had gone through all of these questions. I thought I couldn't honestly answer that I had tried a new router in spite of the fact I couldn't see how a new one would solve the problem. I decided to humour the web page and try a different router. I put the call out on social media and quickly tracked down someone a few streets away from us who had a spare I could try.

The router I borrowed was a cheap and nasty get the job done type router sent out by TalkTalk to their customers when they sign up. No wonder the guy I borrowed it from didn't use it and preferred his trusty old Netgear much like myself. However, it worked! No more disconnections when the phone connected and that was the case for whichever micro filter I used and whether I was connected to the master phone socket or the extension. So then, problem solved, it's my router after all.

Unable to keep the router I had on loan I did a bit of research into which router I should buy next to replace my Netgear. I'm a bit of a Netgear fan-boy, I have a ReadyNAS as well so it was hard to admit, based on hundreds of reviews from dissatisfied customers, their routers are now crap. I thought about going straight to an FTTC connection but wanted to get something fixed in the short term so a cheap router to keep us going for a while is the order of the day. I couldn't decide what to get and noticed Plusnet “give” you a router for a small fee to cover the postage. I decided to go with one of these.

A couple of weeks later, after I had reminded Plusnet to send me the router, it seems they received the order and took my money but somehow forgot to put it in the post, I was the proud new owner of a router made by Technicolor (or Thompson) called the TG852n. Wow wow wow, it really is quite the most appalling box of circuits, wires and firmware I've ever had the displeasure of being insulted by. It's the sort of thing I imagine an electronics company something akin to Frugal Electronics Inc. would produce, corners cut everywhere which can only be to save development cost, and credits the user with absolutely intelligence whatsoever.

I tweeted about what a bunch of crap itis and Plusnet responded wondering what I thought the issues were with it. It was difficult to explain in a 160 character response so I picked a couple of annoyances amongst plethora of wrongness that is the TG852n and sent them back. Here's my top 5:
  1. The user interface is restricted, very restricted. You cannot change anything but the very simplest of options. The first thing I wanted to do was switch it over to using my usual subnet range, nope, can't do that, you're fixed at the address range you're given. That is, unless you use the command-line interface. Most people reading this will know that I'm a fan of the command-line more than most being Mr Linux, but I really don't want to learn another one just to configure my router at home, give me a user interface that works. This brings me onto...
  2. You can telnet to the router. I didn't have to do anything to do this, you can just log in with the admin user name and password. I don't want that port open on my router thanks very much although it is comforting to know the router is a capable device scuppered only by an incapable user interface. What if I were to accidentally route port 23 from the Internet? No, just no.
  3. On the subject of open ports, port 80 appears to be open from the Internet. I don't know why, I didn't tell it to do that. Stop it. I haven't checked what other ports might have been left conveniently hanging for someone to try and get in. Remember I said I have logs going back years, I know how often my IP address is port scanned and password cracked. It happens most days.
  4. Firmware updates are more or less not supported. You certainly can't update the firmware via the user interface (surprise surprise) and if you do want to manually update it then you have to do it via TFTP (presumably via PXE boot) from a machine on your home network. Nope again, no thanks, that's rubbish. It seems, however, that Plusnet can push firmware updates to your router. I don't want that either, get off my network[1], I'll manage my end and you stick to managing yours. Finding firmware updates is nearly impossible too, they're not supplied by the manufacturer so you're left to trawl the Plusnet forums looking for posts where someone has linked to a more up-to-date version than you currently have.
  5. It's a four port Ethernet router, it has one Ethernet light. Apparently, I'm too dumb to need to see whether traffic is being routed to a certain port on the router via the convenient mechanism of a green light provided by almost every router on the planet[2]. I'm left with one light that might wave about a bit if some traffic happens to pass through the router to the Ethernet ports at some point. Oh, and they're 100MB/s ports too, but what do you expect for a free router I guess?

The final straw for me in the TG832n story and why I've decided to stop using it is that it also disconnected when the phone rang the other day. It reconnected shortly afterwards and everything was fine. However, it's now in a state where sometimes it will disconnect if the phone rings and sometimes it will not. Any which way, it always shows a red “Internet” light and a green “Broadband” light which implies that I have an active ADSL connection but that I am not dialled through to Plusnet properly. Another victory for this router then, with the red light on and glowing brightly I've got full Internet access from any of my wired or wireless devices in the house. So the router is lying to me as well[3]. I could try a firmware update, but that would be difficult.

I appreciate I'm probably not the target audience for this router. Its cut down and simplified user interface that buries what it considers “advanced” options in completely unfathomable places is probably targeted more at the likes of the silver surfer, someone who just wants to plug it in and get connected to the Internet without worrying about it any more than that.  However, the TalkTalk supplied router I have on loan (a Huawei HG532) is also a little confusing to use but it blows the Plusnet one out of the water in terms of the configuration options offered via the web GUI and would surely confuse the heck out of all but the most technical users.

Before I decided to try the Plusnet standard router my research was leading me towards Asus and (as I said before) away from Netgear. I've not tried any Asus networking kit before but I've like their motherboards for years in my home builds and we have a Transformer Prime tablet at home which is excellent. I've yet to receive this one but hope it solves the disconnection problem as well as being something that doesn't exhibit quite so many of the things I've been ranting about above. If it doesn't solve my connection issues then there may be more to say here yet.

[1] I don't really expect that Plusnet would be able to get onto my network but it's a distinct possibility.

[2] Yes, I know it's just a hub and isn't switching the traffic so is effectively broadcasting all the traffic anyway in which case one light is sufficient but I'd rather have 4.

[3] Another possibility is that the light stays red to suggest there has previously been a problem. The wafer-thin documentation actually says when the Internet light is red “Connection to Internet failed. Restart your router and see page 7. if the problem persists, open your web browser and see page 12.”

<update 5th Feb 2013>
I'm now using an Asus DSL-N12U that I purchased and reviewed at Amazon.
</update>

Tuesday 6 November 2012

Combining Different Versions of Dojo

During a project I've been working on for the past couple of months my office mate and I needed to work out how to combine a recent version of Dojo and use it on a page where an old version of Dojo was already present.  Our reasons were simple, we were writing a Dojo Mobile app and hence required the use of a new version of Dojo.  However, we were displaying content within our app from a product that embeds an older version of Dojo.

With not much in the way of documentation on how to do this available, I asked our local Dojo guru for some advice and he pointed me towards the "noGlobals" option in dojoConfig.  This is the key to getting things working as it stops the new version of Dojo from writing to the global name space i.e. when you include the old version of Dojo it sets up a global variable called "dojo" if you simply include the new version of Dojo it will also write to the "dojo" global variable.  That means which ever version of Dojo you include last will always be the one you're using when you reference the "dojo" variable.  With the "noGlboals" option in place (which is adhered to by Dojo 1.7 and above I believe) you can continue to use old versions of Dojo from the global name space in conjunction with locally scoped newer versions of Dojo.

It appears there's still not a great deal of examples out there on how to do this so this is my attempt to put together a simple example to show how it's done by combining the ancient Dojo 1.3.3 with the latest at the time of writing Dojo 1.8.1:

<html>
<head>
<title>Dojo Mix-Up</title>

<!-- 
  lets start by including a really old version of Dojo 
-->
<script type="text/javascript" 
  data-dojo-config="isDebug: false, parseOnLoad: false" 
  src="dojo-release-1.3.3/dojo/dojo.js">
</script>

<!-- 
  configure Dojo to load without trampling the global namespace
-->
<script language="JavaScript" type="text/javascript">
  var dojoConfig = {
    noGlobals : true,
  };
</script>

<!-- 
  load the latest version of dojo
-->
<script type="text/javascript" 
  data-dojo-config="isDebug: false, parseOnLoad: false"
  src="dojo-release-1.8.1/dojo/dojo.js">
</script>

</head>

<body>

<!--
  Shove in a couple of divs to write to
-->
<div id="oldDojo"></div>
<div id="newDojo"></div>

<script type="text/javascript">
  // output our global namespace Dojo version (1.3.3)
  dojo.byId("oldDojo").innerHTML = dojo.version;
  
  // output our new Dojo version (1.8.1)
  require(["dojo/dom", "dojo"], function(dom,dojo){
    dom.byId("newDojo").innerHTML = dojo.version;
  });
</script>

</body>
</html>

Sunday 21 October 2012

HTC Desire Cyanogen Mod

I have the HTC Desire phone that to all intents and purposes is a decent phone even by todays standards and was an excellent phone when it was released some 2½ years ago at the time of writing.  That is, with the rather fundamental flaw for a phone that boasts an App Store - a complete lack of internal storage space.  It was built with 512MB flash storage which after HTC had muddled with it was cut to a mere 150MB for users to install applications.  That wasn't so bad back in the day, but these days you just have to install a standard set of Google apps and a few other utility applications and you're done, no more room.

The plight of the HTC Desire was giving me enough grief that I decided to do something about it by flashing CyanogenMod on it.  This like any third-party phone hack is a risky business but I figured if it failed miserably I could always restore the original HTC firmware or more likely get rid of the phone and move to something like the Samsung Galaxy S3 instead.  It turns out Cyanogen is quite the saviour of the HTC Desire and since the instructions for installing it were somewhat less than clear I wrote down what I did based on a Cyanogen forum post and a wiki page, so here it is...

I re-iterate what the forum post states:

#include <std_disclaimer.h>
/*
* Your warranty is now void.
*
* I am not responsible for bricked devices, dead SD cards,
* thermonuclear war, or you getting fired because the alarm app failed. Please
* do some research if you have any concerns about features included in this ROM
* before flashing it! YOU are choosing to make these modifications, and if
* you point the finger at me for messing up your device, I will laugh at you.
*/

Backup
I found it really useful to take a backup of things that were on the phone and put them to one side for safe-keeping both in case something went a little wrong but also to help with restoring the phone when running the new OS.  There are utilities you can use to backup your contacts (or sync them with your Google account), backup your SMS messages, even backup your saved games if you're that worried about carrying over your Angry Birds records to the new phone.  I also took a complete copy of the SD card so I had all the pictures and other things that had accumulated on there as well.

My Phone
BRAVO PVT1 SHIP S-ON
HBOOT-0.93.0001
MICROP-031d
TOUCH PANEL-SYNT0101
RADIO-5.11.05.27
Aug 10 2010,1752:18

Why don't you just S-OFF?
All the instructions talk about S-ON and how to go about gaining S-OFF without really saying anything about what this is.  It turns out that HTC implement a self-protection feature on the phone that is designed to prevent phone hacks and third-party installations that they call S-ON (presumably security on?).  With this switch set in the "on" position your phone cannot be modified.  Unfortunately for HTC they left a security flaw on some of their devices that means this switch can be altered to the "off" position.  Hence, the first thing that must be done to an HTC phone is use some software to take advantage of this and set S-OFF.

Use Revolutionary to set S-OFF
You need to find out some information about your phone similar to the info I've put above.  This will be different between models but for the HTC Desire you simply turn the phone off, then turn it on again while keeping the volume-down key depressed until the information is displayed.

The guys responsible for the software at revolutionary.io insist that you give them some information on your phone which turns out to be a sensible idea since they can attempt to ensure that this beta software doesn't screw up your device or that if it does they can prevent it being installed on similar devices in the future.  You simply go to their web site, fill out a form and download the software.

Put the phone into USB Debugging mode (Settings -> Applications -> Development -> USB debugging) and run the software.  It needs to be run as root under Linux.  It does it's thing and then reboots the phone, you'll now see you've got S-OFF set instead of S-ON if you view your phone information with the volume-down button trick once again.

One other note about revolutionary, being an open source enthusiast, is that they don't provide the source code.  Doing so would expose the security flaw that the software takes advantage of and thus would allow HTC to patch their phones accordingly and stopping this sort of modification instantly.

Use ClockworkMod to Back Up your current HTC firmware
Revolutionary gives you the option of installing ClockworkMod which is definitely worth doing.  ClockworkMod is an improved ROM manager and boot loader with some more advanced features over the stock HTC loader that comes with the phone.  One of these advantages is the ease of which a full backup of the phone's flash memory can be taken.  This appears to be called a NANDroid backup if you come across that term.  I assume that it's essentially just booting into a small program from the boot manager without accessing/mounting the flash device such that the entire flash device can be consistently backed up - a bit like a partimage or dd copy of an unmounted hard disk or partition under Linux.

Boot the phone into the bootloader (this is the same volume-down while holding power when the phone is turned off trick).  Use the volume keys to navigate up/down the menus and the power button to select the desired menu item.  You need to head for the "recovery" option which will take you through to ClockworkMod.  Select the "backup and restore" option to create a backup of the phone as it stands right now - this will give you a fighting chance of restoring the phone as-is should something go pear-shaped or for any other reason you decide to return to the stock HTC firmware.

The backup is placed on the SD card so you need to boot the phone again, access the SD card from your computer with a USB cable and store the backup image safely away somewhere - this is really important since the SD card will be wiped later along with your backup unless you copy it somewhere else first!

Get CyanogenMod Binaries onto the Phone
I grabbed the following files but there may be later versions now:

The first of these is the CyanogenMod package that will be your new operating system on the phone.  The other two packages are Google apps installed on top of the new OS.  I forget why you need the Google ones, but presume you can't get them from the Play Store on Android.

You'll need to copy these into the root directory of the SD card, still using the stock HTC firmware.

Install CyanogenMod
Boot the phone back into the recovery mode with the volume-down trick and selecting the "recovery" option to access ClockworkMod once again.  This time select the "Wipe data/factory reset" option and let that do its thing.  Then select the "Wipe cache partition" and let that complete as well.  Then you need to select the "Choose Zip from SD card" option and select the CyonagenMod zip file.  This will install the OS onto your phone after which you can choose the "Choose Zip from SD card" option again to select the Google apps zip and once again for the other Google apps.

During the installation you should create an ext data partition on the SD card that will be used a little later after the installation has completed (keep reading).  I have the stock 4GB class 2 micro SD card that came with the HTC Desire still in my phone so I opted to carve that up to use 1GB for data and keep 3GB free for use as a normal Android SD card partition.

Reboot
Reboot the phone and simple as that, you've got rid of HTC and you're booted into something that feels a lot closer to what you get on a Samsung phone i.e. a fairly stock version of Android which in this case is is 2.3 Gingerbread.

Install and Configure S2E
This app is the primary reason for installing Cyanogen, it's not the OS that's so much the saviour of the HTC Desire but the fact that using an app such as S2E (simple2ext) you can format part of your SD card for use as internal storage and thus completely resolving the tiny storage problem.  S2E is only compatible with Cyanogen and as far as I'm aware you can't do anything equivalent in the HTC stock firmware.

Once the phone has booted into Cyanogen for the first time, just fire up the Google Play store and find the S2E app.  Install it to the phone and run the configuration part of the app.  This lets you pick which parts of your installation you want to move to the ext partition you created on the SD card.  You can move over whatever you think you need to and when you're done just reboot the phone once again.

The S2E configuration will have installed a little script into the boot sequence of the phone such that on this reboot (i.e. the next reboot after running the S2E configuration) the script run and move the data you have selected onto the SD card.  It will only do this each time after you've run the S2E configuration and made changes.  These reboots can take quite a while since you could be moving quite a lot of data from your internal storage onto the SD card so be patient while it completes and boots back into Cyanogen.  Once you've done that you're good to go with using the phone again.

Final Thoughts
Having worked through all that, you'll need to tailor your phone to how you like it once again.  Restore the backed up SMS messages, contacts, files, etc.  You'll need to completely re-download any apps you liked on the HTC, configure those, set up widgets, etc, etc.  All standard stuff really, except this time you'll actually have some space in which to install things.

I'm currently using Cyanogen 7.2.0.1 which seems stable enough, it has crashed the phone a couple of times, just after I installed it but now it seems to have settled down and is running absolutely fine.  Running apps from the SD card using S2E can seem to make the phone rather laggy too so tweaking the settings in the S2E configuration to give a good balance of data stored on the SD and on the internal memory seems to be wise, along with tweaking other settings such as the read buffer.  I dropped the read buffer from the standard 2MB down to 512KB and that seems to have removed the lag.  Another way of reducing lag would be to update the SD card, say to one of the latest class 10 cards in which case it'll probably be quicker than the internal memory anyway!  I've not done that as yet and seem to be running just fine as I am for now.

Tuesday 5 June 2012

Building a New PC

This post is related to a couple of my recent posts, the New PC Build post that explains the kit I ordered for this build with a little reasoning thrown in for good measure, and the New PC Install Notes post where I dumped a few of my thoughts about the software installation.  However, this post is specifically about the hardware build.

This is what I started out with, a set of shiny new boxes and an old PC which would be gutted back to the case and have only the case reused.


It's a nice Antec case and I couldn't find anything much better around today so didn't see the point in buying another one even if it did mean I had to strip it out before I started.  This is the shot with the case stripped back and all the stuff I used to have in there strewn across the bed I was doing the build on.


I only had 1 case fan in my previous build and decided to upgrade to include 2 fans in this build by including this Antec fan.  It seems to me that a lot of the new cases you buy these days have moved away from the ATX specification for front-to-back airflow as they have all sorts of fans and vents all over the place.  I quite like the idea of front-to-back airflow to give a nice continuous stream of air to the parts inside.


Here's the fan mounted in the front of my old case.  It was supplied with some little rubber grommets for screwless fixing into any case.  However, my case already had the purple clips you can see in this picture in place so I had no need for the grommets and the fan just clipped into place very quickly and easily.


Next up it was the power supply.  Here's the one I chose while still shrink-wrapped.


It turns out this is probably the most over-packaged PSU in the world.  The box is designed to give the wow factor when you open it, the likes of which are usually reserved for high-end consumer products such as a squeezebox, it has a fold-out design in a glossy box to reveal a very neatly packed PSU with all the cables hidden underneath.


The fan fitted pretty quickly and easily as well, the build was going well so far.


One of the things I use my PC for most is for photos.  I decided when putting together a spec for the new machine to include an internal card reader for convenience.  I also wanted to mount a USB 3 socket on the front of the box so this little reader from Akasa seems to fit the bill perfectly.


Here's the card reader after being unpacked and fitted.  It's much like fitting an old floppy drive, being a 3½" device and you simply connect it up to power and in this case the USB 3 motherboard header and it's job done.  I've not included a floppy drive in this build so it also makes the front of the case look less bare.


One of the indulgences of this build, given they're expensive against a traditional hard disk, is the inclusion of an SSD drive.


Here's the content of the box, you get a molex to SATA converter (presumably in case you're fitting it to some ancient power supply) as well as the fixing screws, a SATA cable and conveniently a 2½" to 3½" mounting bracket - which I needed as my case doesn't have any 2½" mounts in it.  You also get the manual and CD but those are still sealed ;-)


This is round the back of the SSD after fitting, inside the case.  You can see it's a pretty simple affair as you'd expect with just a SATA and power connectors on the back.  You can also see how the 2½" to 3½" mounting bracket works from this picture.  The bundle of wires at the top of the picture is coming from the back of the Akasa USB 3 card reader and slot.


There's plenty of better pictures of the Asus P8Z77-V LX board out there, but this is my board fresh out of the box and ready to be fitted into my case.


This is the board in place in my old case.  It was really very easy to fit as I'd removed an old Asus board before fitting this one and all the mounting holes were in exactly the same place.


Here's the new Ivy Bridge 3570K CPU out of the box with some Arcitc Silver 5 thermal compound.  Interestingly the stock cooler doesn't look too bad and is certainly much easier to fit than previous coolers I've worked with.


As I said above, I was pleasantly surprised to see the stock cooler being so easy to fit.  It was (as has been usual for coolers) a screwless installation with the four plastic fasteners secured in place with a simple firm (but not too firm) press towards the motherboard.


Here's the memory in box, ready to be fitted.  As usual for DIMMs it was a pretty undramatic installation process as they slotted in very nicely next to the CPU.  These don't have any huge fins on them for cooling and have a very simple heat sink arrangement making them low profile so I had no worries about overhanging coolers interrupting the fitting of the DIMMs since I would be using all four slots in the above picture.


All in all it was a pretty painless installation process and I've been using the machine for a couple of weeks already without any real issues.

Sunday 27 May 2012

Natural Language Processing Course


Over the first few months of this year I have been taking part in a mass online learning course in Natural Language Processing (NLP) run by Stanford University.  They publicised a group of eight courses at the end of last year and I didn't hesitate to sign up to the Natural Language Processing course knowing it would fit very well with things I'm working on in my professional role where I'm doing more and more with text analytics and continuing my work in speech to text.  There were others I could easily have signed up for too, things like security or machine learning, more or less all of them are relevant for something I'm doing.  However, given the time commitment required I decided to fully commit to one course and the NLP one was to be it.

I passed the course with a grade of 85% which was well above the required 70% pass mark.  However, the effort and time required to get there was way more than I was expecting and quite a lot more than the expected time the lecturers (Chris Manning and Dan Jurafsky) had said.  From memory it was an 8 week course with 10 hours a week required effort to complete the work. As it went on the amount of time required went up significantly, so rather than the 80 hours total I think I spent more like 1½ times that at over 120 hours!

There were four of us at work (that I know of) who embarked on the course but due to the commitment of time I've mentioned above only myself and Dale finished.  By the way, Dale has written an excellent post on the structure and content of the course so I'd suggest reading his blog for more details on that stuff, there's little point in me re-posting it as he's written such a good summary.

In terms of the participants on the course, it seems to have been quite a success for Stanford University - this is the first time they have run courses in this way it seems.  The lecturers gave us some statistics at a couple of strategic points throughout the course and it seems there were around 40,000 people registering an interest, of which around 5000 were watching the lecture material and around 2000 completed the course having taken part in the homework assignments.

I'm glad I committed as much as I did.  If I were one of the 5000 just watching the lectures and not doing the homework material I don't think I would have got as much out of it, but the added time required to complete the homework was significant so perhaps there's a trade-off here?  It's certainly the first time I've committed this much of my own personal time (it took over the lives of myself and Dale for quite a few weeks) as I was too busy at work to spend many business hours working on the course so it was all done in evenings and weekends.  That's certainly one piece of feedback I gave at the end of the course, Stanford could make the course timing more flexible but also allow more time for the course to be completed.

My experience with the way the assignments were marked was a little different to the way Dale has described in his post.  I was already very familiar with the concepts of test, development and held-out sets (three different sets of data used when training NLP systems) so wasn't surprised to see that the modules in the course didn't necessarily have an exact answer to them or more precisely that the code your wrote to perfectly analyse some data on your local system may not get full marks as it was marked against a different data set.  This may seem unfair but is common practice in all NLP system training that I know of.

All in all, an excellent course that I'm glad I did.  From what I hear of the other courses, they're not as deeply involved as the NLP course so I may well give another one a go in the future but for now I need to get a little of my life back and have a well earned rest from education.

Tuesday 22 May 2012

New PC Install Notes


This post documents how I installed Linux and Windows side-by-side in a dual boot configuration.  This isn't something particularly difficult to do but I wanted to note down what I did so I remember in years to come.  Also, my new PC build contains some very up-to-date hardware (such as UEFI and an SSD drive) and with the combination of the many changes and updates to Fedora 17 (F17) and Windows 7 SP1 (Win7) it made the installation a first of a kind for me in quite a few different ways.

This post may well be updated in the future if I do further installation tweaks and optimisations to the system.  There's also clearly a lot more you can do in terms of system set up than I've written here, installation of drivers in Win7 very much being one of those.  This is simply intended as an installation and base optimisation guide.

Create USB Boot Sticks

Due to a fault with the DVD writer I ordered, it's on its way back to the retailer for replacement.  Not one to let this stop an installation going ahead I wrote install images for F17 and Win7 onto USB stick.

F17 is still in beta with the full release delayed until 29th May at the time of writing.  I tried writing and booting from the KDE Spin Beta Live image but it stopped with a kernel panic during the boot process after the grub screen.  I found using a later copy of the boot.iso (which is the same as a netinst.iso) from the distribution nightly builds solved the problem so clearly whatever the bug is on the beta image it's already been solved.

Creating a USB stick for Fedora is pretty easy these days, no messing around on the command line, I decided to use the liveusb-creator utility.  It fires up a GUI from which you basically just select the iso image you want to write and the USB device to write to, then hit "go" and it does the magic for you.

For me, writing the Windows stick was a little more complicated as it requires a Windows system to run the stick writing utility with the somewhat strange and long winded name of the Windows 7 USB/DVD Download Tool.  Fortunately, I had a Windows 7 VM laying around so I used that and writing the image to USB stick was similarly easy as it was for Fedora.  Simply choose the image file to write, the target device to write it on, then hit "go" and it does the magic for you.

Install Fedora 17

Once I worked around the kernel panic bug I mentioned above and booted with a later F17 image, the installation process was a fairly familiar affair.

With the new PC being a UEFI box there was no need to add the GPT partition when partitioning the SSD. I simply created one large root partition and a 4GB swap partition, leaving half the disk unallocated for the Win7 install later.

Since I didn't boot from the KDE spin image, I swapped out the default Gnome desktop (I've tried hard for a long time to Like Gnome 3 but I just don't so I'm moving back to KDE) and did a lot of package selection using the installer to save me messing around later but also to optimise the amount of data downloaded since I would be installing over my ADSL connection.

Once tip when installing Windows after Linux these days is to ensure the os-prober RPM gets installed.  This was done by default for me.  This package allows grub to detect the presence of other operating systems and add boot entries for them in the grub menu.  It'll come in very handy later on...!

SSD Optimisations for Linux

Even with a distribution as current as Fedora 17, the settings chosen for SSD usage are really rather sub-standard.  There are a lot of handy tips and guides out there for which settings you should change or add to the system to enhance both the performance and longevity of your SSD.  I decided to go with the rather comprehensive information provided for the Arch Linux distribution as they have a great wiki page on SSD optimisation.


Update /etc/fstab

I've only got one SSD in my system at the moment, I've removed all my old hard disks with no intention to use them right now as all my data is stored on my NAS.

Add the "noatime" and "discard" options to SSD partitions.  The discard option is the really super-important one as it turns on TRIM.

Mount /tmp as tmpfs by adding a line similar to the following:
  tmpfs /tmp tmpfs nodev,nosuid,size=75% 0 0

Doing this allows the system to write temporary files to RAM instead of the SSD.  This will improve performance (RAM is faster than SSD) and reduce writes to the SSD (improving the life of the SSD).  I've added an option to increase the allowable size of the tmpfs file system to 75% of RAM from the default of 50%.  This means I can run compilations or other intensive tasks in RAM without ruining the SSD and get the performance benefits too.  With 16GB main RAM, this will allocate up to 12GB for my /tmp directory.  In normal usage I wouldn't expect to use anywhere near this but it's nice to have it there for when I'm not running too many apps but doing something else such as compiling RPMs.

Change the Scheduler

The recommendation for an SSD is to move away from the default cfq IO scheduler and switch over to the noop scheduler.  This can be done in a variety of ways that are documented in the Arch Linux wiki page I've linked above.  Since I've only got 1 disk in my machine and it's an SSD I changed the scheduler option using grub.

For Fedora 17 this consisted of an edit to the /etc/default/grub file and adding elevator=noop to the existing GRUB_CMDLINE_LINUX stanza.  Then rebuilding the grub configuration with the command:
grub2-mkconfig > /boot/grub2/grub.cfg

Optimise Kernel Parameters

With 16GB main RAM I'm not expecting to do much in the way of swapping.  However, I did add a couple of lines to /etc/sysctl.conf to make the system less likely to do so:
  # make the system less likely to swap
  vm.swapiness=1
  vm.vfs_cache_pressure=50


Install Windows 7

Similarly to the F17 install, the Win7 install proceeded with little in the way of drama.  I did select the advanced install option when given the chance but that was just to ensure Win7 installed into the free space I had left on the SSD rather than rudely splat my shiny new F17 installation.

And yes, for those of you wondering why I'm installing Win7 at all, especially dual boot rather than in a VM, it's simply for those times when only Windows will do... so gaming mostly I should think.

There's not much else to do with Windows after installation.  Unlike current Linux distributions it's said to  detect the presence of an SSD drive and apply the appropriate optimisations automatically.  Whether this is entirely true or not I suspect I'll never be any the wiser to.

Update the Boot Loader

So Win7 didn't kill off my F17 installation which is always a bonus, but it does assume divine rights over the entire system so writes its boot loader all over the existing grub installation allowing you only to boot windows with no menu options for anything else - not ideal.  Now I need to re-install grub which I've always found easiest to do by booting a rescue Linux CD and using a chroot to the installed OS.  This has changed slightly with the inclusion of grub 2 so here's what I did.

Boot the F17 USB stick once again but this time choose the "troubleshooting" boot option and select to boot the "rescue environment".  Red Hat based systems have always done a great job of rescue environments so you'll boot into a text based system that asks you if you want various things turned on in the rescue environment such as networking (although they assume you want that these days) and if you want to attempt detection of installed Linux systems (which you do).

Drop out to the shell prompt and chroot to /mnt/sysimage.  Then rebuild the grub config (this is where os prober installed above comes in very handy) to include an entry for booting Win7.  Then re-install grub.  Job done.  Once you've booted to the rescue shell, the commands are something along the lines of:

  chroot /mnt/sysimage
  grub2-mkconfig > /boot/grub2/grub.cfg
  grub2-install /dev/sda
  exit
  exit
  reboot

This assumes you want to install grub to /dev/sda of course.  You'll need to exit (or ctrl+d) twice, once to get out of the chroot and once more to return to the rescue interface.  Then choose to reboot from there as it'll cleanly unmount your file systems and do a better job of clearing up than if you simply rebooted the system yourself.

Sunday 13 May 2012

New PC Build

It's been quite some time (about eight years) since I last built a PC for myself and I've been promising to build a new one for quite a while, now I've finally got round to ordering some parts.  Not that I need to, but I've justified the expense as a treat to myself after our annual bonus came through, having never really spent the bonus on anything particularly exciting before.

This is a short post detailing what I've gone for and why, not to show off, but so in years to come I can look back (as I have before) on what was available at the time and I'll have a record I can get to should I forget the detail of which parts are in my current PC.

I'm still the sort of geek that likes to build my own computers.  I like to do the research and put them together but it's still definitely the best way to get a good deal and the only way to get each component to be the exact one you want.  I also still like to have a PC for some reason, I have a laptop for work so I've already got something mobile and a PC just feels like the right solution for home use.

So with no further ado, on order are:

Case: I'm sticking with my old Antec SLK3700 case, okay so strictly speaking this isn't on order.

Processor: Intel Core i5-3570K (£175)
Sandybridge was a real game changer when it came out and firmly handed back the power to the hands of Intel  in the processor market.  I've been waiting for Ivybridge to come out for a while, either so I could buy a cheaper Sandybridge or plump for the Ivybridge if the price difference wasn't too great.  Well, for those who know their processors you'll see I've gone for the Ivybridge option.  Currently it's only 20-odd quid more than the equivalent Sandybridge processor and I think the extra expense is worth it to get the better on-chip graphics capabilities and minor improvements in speed and energy efficiency.

Motherboard: Asus P8Z77-V LX (£94)
I'm a sucker for a good Asus board, I've used them in most of my PC builds so that's where I start looking when I want a new machine.  I nearly went for the LE version of this board, it was another 20 quid, but I decided I wouldn't be using the extra features it provides (surround-sound audio and extra sata sockets) so I pocketed the difference and went for this one instead.

Memory: Kingston (4x4GB) DDR3 1600MHz XMP HyperX (£65)
I very nearly went for some Corsair low profile DIMMs but was swayed by the vendor support list for DIMMs for the motherboard chosen above.  I'm still in shock that you can get 16GB RAM for 65 quid!

Storage: Intel 120GB 520 Series SSD (£140)
I've got 4 hard disks in my PC at the moment, one of which is not connected, another is hardly used and the remaining 2 are coupled together in a striped RAID array in an attempt to get some sort of speed out of them.  Hard disks really are the bottleneck in your PC these days so I've decided to shove in a top notch SSD from Intel.  They're the only manufacturer to offer a 5 year warranty and also came highly recommended (under the Hitachi brand, Intel and Hitachi work together on their SSDs) from one of my respected colleagues in the storage department at work.  120GB should be ample for my storage requirements on the PC, the disk will be split to dual boot both Windows 7 64-bit (for those occasions when only Windows will do and a bit of gaming) and the main stay of Fedora x86_64.  My data other than the operating system already lives on a NAS.

PSU: CoolerMaster 600W Silent Pro Modular (£63)
Unfortunately my current 620W ATX power supply only has a 4 pin CPU connector rather than the currently common 8 pin EPS so I've begrudgingly had to fork out for a new PSU.  This CoolerMaster one gets some decent reviews for being quiet, efficient and delivering good consistent power within the tolerances required by ATX.  The 600W rating will also leave me with some overhead should I decide to whack in a high-end GPU at some point in the future.

Thermal Paste: Arctic Silver 5 (£5)
I've got some really old unbranded thermal paste knocking around somewhere but decided to invest in some decent stuff for this build so the Arctic Silver was the way to go.

DVD Writer: Samsung S222AL 22x with Lightscribe (£15)
I've got 2 optical drives at the moment, both are IDE and with the new breed of motherboards (or not necessarily even the ones that are particularly new) IDE is long since dead so I've opted to get a Sata DVD writer for this build.  Similar to the memory, I'm amazed you can pick something like this up for 15 quid!

Front Case Ports: Akasa USB 3.0 Card Reader (£20)
My current case has a couple of USB ports on the front, I thought it would be useful to throw a couple of USB 3.0 ports to the front of the case too.  This unit also has a built-in multi-card reader so I'll no longer have to hunt for my USB SDHC reader every time I want to copy pictures from my camera.

Case Fan: Antec TrueQuiet 120 (£7)
A 12cm fan for the front of my case, I've got a slot to fit another one in so I thought why not given the heat output I'd expect from this build.

Keyboard and Mouse: Logitech Desktop MK120 (£13)
My current keyboard is PS/2 and has seen better days.  I still like a simple keyboard with none of these funny curves or multimedia keys you can get these days so went for this cheap set from Logitech.

The obvious note in this build is a lack of a GPU.  As I mentioned above, I've left overhead in the power supply to put in a GPU in the future should I choose to do so.  I'm going to do this build and run on the GPU built into the processor.  I'll be interested to see what the performance of the HD 4000 is for my needs, if it's sufficient then great, otherwise I'll be tempted towards an NVidia GTX-560 card.  I guess it all depends on if I do a little more gaming then I do currently (which is next to none on the PC) and whether the HD 4000 is up to the job.

The other thing I've got my eye on is an up-rated cooler from the stock cooler supplied with the 3570K.  I quite like the look of the Corsair H60 Hydro should the need arise.

Saturday 21 April 2012

Home Networking

Photo by Beth
We've just finished a fairly major project at home in doing up our living room (right).  It's been fairly major in that the room is quite large and has an open stair well so we've had to include the stairs and landing too.  It's been a complete overhaul with the fire you see in the picture having been fitted, the upstairs ceiling plastered, new carpets, a little wall paper hung, painted and thoroughly decorated.  However, one thing I wanted to do as part of the redecoration was to run Ethernet cable between the TV point in the right of the picture all the way around the room past the door on the left to the computer point in the kitchen.  Basically, I've been long since fed up of wireless being slow, if not dropping out entirely.  So here follows a few notes to remind myself what I did with the hope it might be useful to someone else out there too.

I started out by planning the route for the cable and researching the kit I would need to buy in order to complete the job.  I decided to channel the cable into the wall where the fire is, using a channel I had the fire fitter make behind the fire when it was fitted.  For the rest of the run along the green wall and into the kitchen the cables are sunk into the back of the skirting board. Having replaced the skirting, I routed out a channel just big enough for 2 Ethernet cables and re-hung the skirting back in place.

The next decision was what type of cable to buy, Cat 5e or Cat6, Solid Core or not?  I already knew I wanted 2 cables, driven mainly by the fact it seemed pointless just to run one and a lot of Ethernet wall face plates have 2 sockets in them so that's what I decided on.  Then I went shopping.

I already had a couple of single metal backing boxes to mount face plates on, the rest of my shopping list consisted of the following for the sum of just under 40 quid:

As you can tell from that little list, I decided to go with shielded Cat 6 and bought 2 cables with ends already attached.  I found it was cheaper to do this than to buy a reel of cable!  It was simple enough to cut the ends off and as you can tell from the rest of the list I decided to use whatever length was remaining to make up some Ethernet cables using the ends and crimper tool.


Tuesday 10 April 2012

Moving to Talkmobile

Having been with Vodafone for the best part of 10 years I decided it was time to move on and go with another network offering much cheaper rates, no idea why it's taken me this long to be honest.  I decided to go with Talkmobile as they came recommended from a colleague, have great PAYG and monthly rates and are based on the Vodafone network so I'll get the same coverage I'm already used to.

That all sounds great, except for the fact that moving to Talkmobile has turned into a nightmare of sorts.  You'd think it would be simple to switch network operators.  Basically, you get a Sim card from the new provider, ask your old provider for a PAC, then call the new provider and they arrange to transfer your number over.  Not so.  Here's my story.

Thu 29th March
Ordered Sim from Talkmobile.

Fri 30th March
Talkmobile Sim arrived.

Sat 31st March
Called Vodafone to get my PAC which was given with much less hassle than I was expecting, basically I just told them I found a better deal on another network and they gave me the PAC with no further questions or persuasion to stay with them.

Called Talkmobile support to arrange for my number to be ported to the new Sim.  I was informed because it was a Saturday the request would be queued and completed on Tuesday, no later than 5pm - 6pm.  No problem for me, I still had some Vodafone credit to use up as I was on PAYG with them.  I was told to call back on Wednesday after the porting was complete in order to register and set up the top up card.

Tue 3rd April
Called Talkmobile support at around 7pm and explained my phone was still responding to the temporary number and that my number porting had not yet completed.  The number I want to use is now disconnected and not responding at all so I'm now disconnected from anyone calling or sending me texts, either with the Talkmobile or Vodafone Sims.

After some time spent with them on the phone and going through the phone numbers and Sim numbers for both the Talkmobile Sim and the Vodafone Sim they concluded that the porting had been performed incorrectly and that the problem was with Vodafone as Talkmobile don't do their own number porting but use Vodafone to do it for them.  The fix for this is to do the porting again so another request was put in to port my number, but being late on Tuesday this would be queued on Wednesday for completion on Thursday.

Thu 5th April
The porting was successful, my number is now reconnected and people can call/text me once again.  I set about explaining to those people who had contacted me via other means that my number was now working again - I have no idea if anybody else tried contacting me during this downtime, there's no way of knowing.

I figured out I was now on a different price plan to the one I had originally purchased the Sim to use.  I had sent some texts to people to tell them my number was working again but realised I was over-charged compared to the text price I thought I should be paying.  I also noticed that when I checked my balance I was informed I had "3 minutes before lower rate" which wasn't something I would expect to see of the plan I wanted to use, the PAYG Essentials plan.

I called Talkmobile support again to get them to switch me back to the Essentials plan, figuring that during porting something must have got switched over.  The guy on the phone was very confused and told me the price plan was wrong (he easily switched me over to the one I wanted) and that mobile Internet connection, Voice Mail and MMS were not enabled for my Sim which he said he'd never seen before.  He was also easily able to add those to my Sim card so I could use those services.

I took the opportunity to register at this point and to connect my top up card to the Sim as I'd been informed I should do when I first called Talkmobile.

Fri 6th April
I decided to test the mobile Internet connection, not something I use a huge amount as I'm normally connected to a wireless network and hence why I don't bother with a monthly contract and go with PAYG.  The connection didn't work.  After a bit of searching around the Internet I found out that I have to manually configure the Access Point Name (APN) on my HTC Desire Android phone.

This was an entirely new concept to me as in all my years on the Vodafone network I'd never had to manually configure anything at all, it always "just worked" and was configured over the air to my phone.  The best Talkmobile can do here, however, is to provide a page of settings for Android users to manually configure their phones.  I followed the instructions to set up the APN on my phone and was unable to connect, getting a "connection failed" message.

I was going away for the weekend to see family and decided to wait until Tuesday after I could check my APN settings with my friend at work who is also with Talkmobile.

Tue 10th April
I checked the APN settings with my friend at work and was still unable to connect to the Internet using the mobile network, still getting a "connection failed" message.  Time to phone Talkmobile once again!

I spoke to the assistant (who are all very good, clear English speakers by the way - I'm getting a lot of experience with them) who directed me to the web page for Android settings.  In informed him that I'd already entered the settings and needed some further assistance.  As you would expect he double-checked the settings I entered with me and we decided they matched what was required.  He did some further debugging and got me to re-register my phone onto the Vodafone network and to reboot the phone but nothing appeared to work.  I was put on hold while he did some further checks, he came back to me and informed me there was nothing wrong with my phone but the Sim card is faulty.

The resolution this time is that I'm being sent a new Talkmobile Sim which I should receive in 2-3 days time and will have to go through a number transfer process once again when it has been received as I still want to keep my old number.

Wed 11th April
The replacement Sim arrived in the post so I called Talkmobile to get the number swapped between the original Talkmobile Sim and the replacement.  As I've become used to, the chap on the end of the phone was very helpful and arranged for the number transfer to take place.  It is, however, going to take another 24-48 hours to transfer the number again due to the fact they have to send spreadsheets of porting information over to Vodafone by email (so he said) for the number swaps to be done.

Fri 13th April
Checked the new Sim and my original Talkmobile Sim and the number transfer has not yet completed by 7pm - it was due by 6pm at the latest today.  I'm now wondering whether I should give Talkmobile more time or if it's time to consider looking at another network entirely as this really is becoming a shambles.

Called Talkmobile yet again.  I was informed that there has been an issue (which was not specified to me) with the number transfer and that this issue has been escalated but they are not able to give me more accurate information than that at this time.  I enquired when the transfer might happen and if I should wait indefinitely for the transfer to happen.  The guy on the end of the phone really had no idea and seemed to think there was an outside chance the transfer might happen on the weekend but thought Monday was more likely.  I was advised to call back on Monday for further status.

Tue 17th April
Called Talkmobile for a status update of the number transfer as it's still not completed 1 week after I initiated the process.  I'm told now that there is a backlog of number transfers outstanding and that my number swap is in the queue awaiting processing and there is some hope that it might be done by tomorrow.  Talkmobile offered a £5 credit to my PAYG account for the inconvenience.  I'm assured that one of the managers has my issue in hand and is going to encourage the number swapping team to complete the request as quickly as possible.

Wed 18th April
The good, my number has finally been moved across to the new Sim.  The bad, I still can't connect to the Internet - the very problem the new Sim was supposed to solve!  So, guess what?

A familiar call to Talkmobile to ask what's going on.  I'm told that now my number has been moved across I no longer have GPRS or MMS facilities enabled on my Sim card, so I'm back to the same place I was at on Thursday 5th/Friday 6th April when I couldn't connect to the Internet because my Sim is not enabled.  Talkmobile have to once again escalate the problem back to Vodafone, so it's yet another 24-48 hour wait for the turnaround of my issue.  At least this time the Talkmobile contact on the phone appeared to be taking some ownership of the issue and has said she will contact me either tomorrow or on Saturday with an update on the issue.

I feel sorry for Talkmobile in a way, they appear to be attempting to do everything they can to help and get the issue(s) sorted, but on the other hand I really have no idea where the incompetence lies throughout this whole story.  Once again, I'm left wondering how long it will be before I'll be asking Talkmobile for my PAC?  I've come this far so figure I may as well stick with it a little longer, but if it's not sorted fairly quick-smart from now I think I'll be taking another close look at GiffGaff.

Thu 19th April
I received a call from Talkmobile tonight.  Yes, it seems someone really has taken ownership of my problem and I'm finally getting some good customer service in addition to the polite customer service I've been receiving all along.  I was told the problem(s) should now all be fixed and I should try a manual roam on my phone to disconnect from the network and reconnect again before I should attempt a GPRS connection.  I was informed if that didn't work I should try a couple of times just to be sure.  I confirmed with the person on the phone just what that meant and what I should do on my phone.  We agreed to part company there and I would attempt the instructions with another call-back expected on Saturday to check how I got on.

So, I set about having dinner first of all (I was in the middle of cooking when the call came through).  But not entirely eagerly after that I turned on mobile data on my phone, no worky.  I did a manual roam, still no worky.  I did another manual roam, still no worky.  By this time I'm thinking there's still an issue but decided to allow a little time and try again later.

A couple of hours later and I've just done another couple of manual roam cycles and still got nothing.  I decided a to do a full reboot of the phone and when it came back up, bingo.... IT'S FIXED!!!

I'd like to thank the unnamed (unnamed on this post anyway) person on the end of the phone for finally taking some ownership and getting things sorted out.  I'm sure I'll find the network perfectly ok now I've moved across and you've saved your company a disgruntled customer - I'm merely just an annoyed customer now for the length of time and number of calls I've spent speaking to your service personnel.

In conclusion then, it took me eight calls to Talkmobile, one call-back and exactly 3 weeks to the day to transfer satisfactorily from Vodafone while keeping my number.

Fri 20th April
It's not over, until... well no fat ladies here I don't think!  However, with the saga behind us (I say "us" because I'm certainly glad it's sorted and think Talkmobile are too) I received a call this afternoon from one of the managers.  He was very gracious and first of all apologised for all the troubles I've documented above.  We went on to have a nice chat about the situation, whether there was anything else he could do for me, thanked me for my feedback both here and on Twitter, and he was appreciative of both my patience and for the lack of ranting (ranting is not my style) and good humour on this post.  I gratefully accepted a further £20 credit to my account in addition to the £5 I was offered on the 17th.  We parted company on good terms at the end of the call and I pointed out that all my calls have been courteous (as I've documented) and while my problems weren't always sorted out I appreciate having someone easy to understand (UK call centre) and polite on the other end of the phone, and I asked him to thank his staff.

Going forwards, as I've said before I'm really not expecting to have any problems.  My better half has initiated her transfer to Talkmobile, so unknown to them until now, they were in danger of losing 2 customers.  I wish her better luck than I had, but I really suspect my experience is a rare one-off kind of a situation.

So, the big final question is really whether I would recommend Talkmobile.  I'm hesitant either way at the time of writing.  Based purely on my experience joining it would probably be a negative, however, given the way the situation has been turned around, the likelihood it's a one-off, and assuming normal life with my mobile is resumed over the next month or two I would gladly make a positive recommendation.  So essentially it's a "well done" to Talkmobile, you could have properly screwed up and lost custom, but you didn't and I'm grateful for that.