Showing posts with label hpc. Show all posts
Showing posts with label hpc. Show all posts

Monday 17 November 2008

Super Computing Project Ends

Not blogged in a while, will spare the details/excuses but back in September I had the opportunity to get back into Super Computing for a one-off project. It's this that has kept me so busy throughout October and into November where I was really very well submerged into work for an extended period. Normally I like to keep balanced in my work-life balance. However, this project demanded a lot of time and attention and fortunately my wife, Beth, was away for a couple of weeks on business too so I really had the opportunity to get stuck in.

As I suspected (from much experience in the area) in my previous post there really was a lot of information missing at the beginning of the project. This is to be expected, the customer cannot be expected to know 100% what they want, they may not even know what is available, possible or on offer. To compound this, even the best sales team can't look into minute details when proposing a solution or making bid. So we had a productive kick-off meeting, made a lot of good decisions and recorded these to concrete the design details.

What happened next you can never be prepared for, but we've probably all experienced it. Yes, the iron fist of the bean counters barged their ugly way into the project. The previously neatly agreed deadlines and design proposals were lobbed into the air with such careless abandon it was almost humorous. The promise at the start of the project from my ex-manager that "this one would be different", and we "wouldn't have to work silly hours or cut corners" was about to be viciously broken into pieces. (Wow, I can sound like such a drama queen). Project deadlines were brought forward by give-or-take 50% simply to meet a financial deadline (of one of the parties involved not IBM I should add) way out of my control. When you're met with this kind of single-minded decision making as the technical leader of a project it's very frustrating but you know there is nothing more to be done except save time where you can and work your butt off. Anybody who knows me and my work will know I hate to deliver anything less than the very best solution I think is right. So I openly request to Mr Financial Man, whoever and wherever you are to listen to people, who like yourselves, are considered the expert in their area.

This was a Linux cluster, which for the uninitiated consists of a number of computers set up to be joined to work together on a single large problem that would take any one of them an unrealistic amount of time to compute. Using the example raised by OzBeefer on my last post, climate modelling algorithms are quite large and complex taking a single computer a long time to run, so loads of them need to work together in order to predict the weather in the future before it becomes the past! Fortunately, the design of this cluster wasn't rocket science and it was quite small. Things progressed very smoothly without any potential problems that might otherwise have occurred and with some very long working hours (read 12+ hours/day) the cluster was eventually delivered to the new schedule at the customer location.

Well I could ramble on about Linux, clustering, algorithms, networks, storage, optimisations and the projects I've worked on for hours yet, which gives me an idea... for those people reading this in the Hursley community answers on the back of a postcard, comment, e-mail or whatever if you think hearing my ramblings about this stuff might be vaguely interesting some time over lunch, tea/coffee, beers, etc?

Wednesday 17 September 2008

Dipping my toe back into super computing

I've been out of university and working in IT for a good few years now and held several different roles along the way. When I started work I was doing an internal support role after which I moved to my current department, Emerging Technologies, where I've held several roles including being an emerging tech specialist and working for IBM's Linux Integration Centre.

I've used the attention-grabbing title for the work I very much enjoy doing but due to certain circumstances thought I had left behind in my career. Super computing sounds very grand and conjures up all sorts of ideas along with other grand titles for the type of work I do such as Beowulf Clustering. The term I prefer, but can also be misleading, is High Performance Computing (HPC). There are all sorts of misconceptions about HPC but easily my favourite is when people pretend to joke, but expecting a serious answer ask something like “Wow, how many frames per second can you get on Quake with that?”.

The opportunity has come up for me to go back to this area for a one-off project, working with some of my ex-colleagues. I'm very much looking forward getting stuck in as the work is usually interesting and my old team are fantastic. As ever with these projects, there are an enormous number of unknown's at the start of the project. I already feel at home knowing the list of things we don't know yet – where the hardware is right now (delivery due soon), what the software stack will be, firmware levels, network layout and design, naming conventions, management and monitoring required, storage requirements, job scheduling, operating system, tweaks and configuration, etc. That's all part of the fun though, I get to work things out along the way and fill in the gaps for areas that, for no fault of their own, people just don't think about answering until implementation time. The kick-off meeting is due soon now so I'm looking forward to getting people to think about all the tiny details I'll need in order to supply our customer with the best suited system I can.