Free Software & Me
Since Stuart posted the previous post, and having read through the comments, I thought tonight an apt time to describe my view on FOSS - though I hope people don't care.
When I first came across Linux in 2004, I was enthralled. I spent hours with a friend of mine trying to get it installed on an old Aspire 1350 laptop I'd just got to go away to University with. I finally got a 'flavour' called College Linux installed. It came with lots of apps relevant to academia - quite cool - but ultimately unusable. I wasn't doing a CS degree, I had no experience of any computing other than a ZX81, and Windows. Oh, I'd once played 'Blood Bath' on some kind of Mac at summer camp.
The idea though was what bit me. The philosophy of everyone throwing their contributions into the hat, and then everyone selling their bit. Some guys might be better at Sales, so they'd just burn & sell the CDs (and hopefully pass some profit back to the project), Developers could pool their talents to create better software, and if a company needed a 'bespoke' application, they could get it developed and passed back to the community in return for having access to other companies' 'bespoke' applications. It was a winning formula. Sure, there were places for it to be abused - but a system that allows no freedom only benefits the person that writes it.
Not long after that, I came across the InGOTs. Whilst I may have gotten the wrong end of the stick at the time, I saw it as a potential extension of that very same philosophy - but to enable educators to pool their resources to help students. It is that, but it's also alot more.
The next 'philosophy' to pass across my desk was that of 'Ubuntu' - "I am who I am because you are who you are." Coming from an African philosophy, this idea is much older than anything a Computer Scientist could have thought up. It's about recognising that your life is intrinsically tied to the rest of your community. We all have certain skills that we can offer, but we also benefit massively from the other people around us. For me, this was a winning formula.
I wanted to use this. I wanted to use this philosophy to do some good. Woo. Go me. The problem I realised was that I was using the philosophy to market software to users - that's missing the point by a significant margin.
In the same way that Developers, Testers, Users and Sales are all tied into the ecosystem of software development, this same ecosystem can be extrapolated up to a higher level to society. We have Doctors, Teachers, Consultants, Athletes, Military, Public Servants all working in the same ecosystem. Do these guys share Ubuntu? Where once there was a respect for the professionals across all walks of life, groups have been fighting against each other and poisoning the inter-group relationships and respect.
As an example, were I to ask people about the Teaching profession - many people would say "Long Summer Holidays," and for Doctors - "Overpaid." We've forgotten the "I am who I am so you can be who you are." Teachers provide us with experiences and frame our learning long after we've left their classrooms. Doctors can do for us things which we will never be able to comprehend, and the Military protect us despite our lack of support.
It was this revelation to me that made me realise that I could no longer spend my time espousing the benefits of Free Software. Coupled with a very poor showing by Richard Stallman at the University of Manchester, when he flagrantly and aggressively dismissed the questions "Should software used to control a Microwave be free?" with the short answer 'No' (yup, go figure). Freedom with your software is what I'd prefer to support - and that includes the Freedom to run Vista if you really really want - no matter how inadvisable it is.
Clustering Approaches
Coming from a non-Computer Science education, there are lots of things which I am ignorant of in the world of System Administration. The things I know about, I can understand (else I'd make a very poor sysadmin), but there are gaps in my knowledge which catch me out.
An article I enjoyed reading today, was one written by the Software Architect of Varnish, and covers the fundamental difference between programming for 2006 versus programming for 1975. Despite my work being limited to higher-level languages, or glorified bash-scripting, it's necessary in my view to have an understanding of the underlying technology, so that it's possible to code effectively.
Much of my knowledge, like most in the hacker community, has been gained by reading through other people's code, and by hacking on it until something usable comes out of it. I was quite pleased that today I managed to write my first python code with an __init__ function and docstrings - without having to consciously go back and work out how to do it properly a second time. The code I wrote wasn't pretty - but I knew I'd done it in a modular way, so that the functions returned the values I needed, and so were reusable.
Moving out of my little world and into the world of work, one of the things that I'm starting to look at now is clustering. This isn't to keep up with the Jones' and have our own 'cloud' - but more to have redundancy and automatic fail-over spread across multiple devices, both physical and virtual. The way I see it, virtualisation strategy is extremely complex beast, in that there are plenty of ways of virtualising. Clustering isn't necessarily analogulous with virtualisation, but virtualised clusters theoretically provides a neat way of expanding clusters, without necessarily having to expand across lots of redundant hardware.
Despite the current threats to the project, one such avenue I'm investigating is MySQL clustering. If I virtualise MySQL servers across virtual machines and link them via replication, it's not clustering. There are downsides and upsides to this approach, but for it to be clustering, you'd need to be using MySQL's cluster mechanism, which is something else to learn. I'd also like to work out a way to do a clustered filesystem across multiple machines, a bit like RAID5 across three disks, but networked clients, rather than just disks wired to the same controller. Lots of constraints seem to pop into my head, like I/O - bandwidth, caching and overwriting keys on databases which go out of sync.
Going back to the article, one of the key points that I brought out from it was that although it's good to have an understanding of the underbelly of what's going on, if you're wanting to do something fancy at the low level, the chances are that the kernel is already doing it. Varnish, the caching project, gets astonishing performance figures, albeit in a non-production environment. It does this by making sure that it doesn't do clever stuff that something else is already doing. In essence, that's my job. I need to make sure that I don't start installing virtualisation and automatic configuration scripts, if someone else has already set something up that'll do it for me.
That's not to say I'm lazy. I'm not. I just want to make sure that the way I'm moving is the most efficient use of my time, the computer's time, and gives my client and employer best value for money. Is there any point in setting up MySQL clustering, if I can run a virtual machine across multiple servers - adding more servers to the computer as I go. Is it worth looking at the clustering options of my distribution. When I installed CentOS 5 (the distribution of choice at the client) - does it make sense to look into what the 'Clustering' group of packaged software provides. Are there other best practises documented out there which give me the answer?
Well I'm not expecting anyone to come in and give me all the answers, but I am hoping that anyone else who's reading this and has the same itch to scratch will either benefit from my work, or offer me their experience. I'll keep the 'blog updated with the technology reviews, but is there a bigger point I'm missing?
If anyone has insight or advice to give, I'm ready to listen.
Network Attached Storage - Open Style
After a few months of pretty much no action on kitting out our home recording-studio with what's required, today the family sat down and we're finally making progress.
My brother is the guy who's sorted out all the recording kit (which includes pro-tools and a mac) so my tech stuff is more or less on the periphery. This suits me, as not being in the country 75% of the year means I probably wouldn't be able to support it anyway.
The only thing I've been asked to do is to create a home NAS, for both the PCs/Macs/Ubuntu boxes around the house, and also for much of the material we're going to be recording in the studio.
I had a little chat in the #ubuntu-uk room, with the idea of building my own server, and just having plenty of diskspace and the ability to expand it. However, with me not being in the country, the idea of letting others shut down the PC and play with a RAID card isn't particularly enticing. Looking at other options, I took a more detailed look at the drobo.
Drobo, you look awesome. In fact, it does look too good to believe - but the main drawbacks I have with it are price and scalability. I'd need a droboshare to put it on the network, and the price of the empty chassis with the droboshare is £388 on amazon - which is pretty borderline. If it extended to more than 4 disks then it'd be a no brainer. I love the simplicity of the device, and the whole family appreciated the ease of use when I made them watch the little promotional video.
I even took a look at the drobopro, but that's £939.12 exc. VAT which is just too much, given that max capacity is 16TB.. will I need more than that once people start recording loads of samples to disk.. I'm not sure. That's why the expandability options of the drobo appeal.
However, I'm a big ubuntu fan (as I hope you know by now) and surely there's got to be a way to do it myself, and use a chassis which enables hotswappable disks - even if I have to log in remotely to add a new disk to the storage. I'm not looking for a budget machine, as this is probably going to hold some pretty important data - but what's the best option.
I'd love to hear a few peoples views on this, so please leave me your feedback in the comments. Andy
Server Monitoring.
For these past few weeks I feel like I've spent most of my time working on a PCI DSS audit. It's the audit which checks whether you've made the standards required by the card processing companies - and gives you the ability to store card numbers. It's an interesting scheme - with many people having mixed views on it's suitaility/workability. I enjoy working to the tough specification, and adding extra bits of security all over.
There's only so many ways in which you can secure a server. Most PCI setups involved using multiple boxes (they have to) - and there's lots of security between the boxes to enhance the security further. However, the PCI audit doesn't just check your procedures and systems to avoid the initial penetration - it also looks to migate the affects should some fortunately soul manage to get in.
Due to my background not being in Computing - there are a few areas that I feel I'm weak on. However, once I can take a look at the problem, it's normally just a case of logic, that 'supa-doopa' programming. In fact, once I'd started programming late in 2008, I realised that the programming was just a tiny part of the process of a programmer - most of the time it was making sure that what was about to be programmed wouldn't create erroneous results.
One program which I've had a love2hate relationship with, that I've now really warmed to, is Samhain. From la-samhna solutions. It's a great program, and kudos goes to the developers for releasing it at open source. I've also really enjoyed using SNORT, ntop, wireshark (formerly ethereal) and the Shorewall firewall. System hardening was an interesting task.. locking the system down to as few users as possible. There's so many different things to take into consideration - you end up with a 3D network of traffic streaming from one server to another.
I imagine this as being a gravity-less environment, with streams of data passing like skycars across the 3D network. I then basically put in roads (the firewall rules) that only allow traffic to be passing
- on ports I know about
- transmitting packets I know about (stateful inspection)
Then, when I turn the firewall on - the gravity gets re-enabled - and any datastreams that aren't supported by the firewall (roads) collapse and are broken.
The most important part of it all though, is the monitoring. Without effective system monitoring - the whole system is useless. You need status colours, and a easy 1 screen display. Events which are expected, such as your developer logging into the server, don't need to trigger a 'critical' error - but can be flagged so the project manager can review them. Sure, it may be easier for a developer to have plaintext lines, spelling out the status - and then expect the Project Manager to read them - but surely it's more fun to have a screen of buttons and colours in front of them. They can immediately see any problems.
But, the main thing I've learnt about all this, is that at the end of the day it's not down to the Sys Admin to secure a system. If the developers don't write their code right - if the company policy regarding logins doesn't prevent changes being made to the live server without PM approval (on pain of death) - then securing it to any level is near pointless.
Needless to say, I'm very happy with what I've been doing the past few weeks - and learned many lessons that I think will stand me in good stead for the future.
Bring on the next PCI audit!
Here I go with Kubuntu
Today I traveled from Macclesfield, Cheshire - to Harlow, Essex. Why? To watch the two teams do battle in the FA Cup, 1st round proper. It was a fairly interesting game to watch, Macc Town finishing 2-0 victors.
However, on the journey, I had my iPod in, and was listening to the #Ubuntu-UK Podcast, with Laura, Alan, Daviey and Tony. They began with a very long discussion on KDE.. and Alan saying that he was going to spend an entire development cycle using Kubuntu.. though he hadn't got round to it yet.
Well...
I use Ubuntu on my work PC, home laptop, and private VPS(s). I love the OS and it works wonders for me. I used to run Xubuntu on my old laptop, but since upgrading I've been getting away with bog-standard Ubuntu. :)
So I've decided to follow Alan's lead... though I think I may have beaten him to it. I intend to run Kubuntu for the entire intrepid release on my laptop. I've started today (9th Nov @ 00.00) and will be running Kubuntu for the foreseeable future. Hopefully I'll be able to submit some bug reports .etc and help out the Kubuntu community in general.
I started in Linux using KDE.. and made the switch to gnome (on Gentoo) when I started my first job. Soon after I was googling for Linux distros that used gnome as default; found Ubuntu; and the rest (as they say) is history.
So here I go on my Kubuntu journey. I'll need to switch stuff like mail over to K mail, Music over to amaroK.. but bear with me.. it could be an interesting ride :)