Archive for August, 2008

London 2012 – The Open Source Approach

August 23rd, 2008

Today Boris Johnson has told the BBC he plans to make sure that the 2012 Olympics is under-budget.  With £9.6 billion currently set-aside, there’s plenty of room to make the Olympics a success.



Whilst only being a small part of the overall budget, Boris should push to ‘Open Source’ the IT development required for the games.  Just think of all the useful things that the Olympic management need to do using their IT system.  Well, Open Source it and you’ve got a legacy there already.  Each of the different ‘modules’ for managing different things can be re-used.  Great!



The other approach is to open source all aspects of the games management.  Blog what works, what doesn’t.  Access a massive pool of ‘free’ talent by sharing solutions and problems.  Make sure it’s not just London who are going to benefit from the olympics.  Create in effect a ‘shared’ knowldgebase, that future countries can use as the basis to their Olympic preparation.  As we’ve learnt from China, we’re probably not going to beat them on the wow-scale.. but we can beat them in what we give as freedoms to other countries.  Something that I’m sure will resonate with the political minded.



And of course, we can do ‘better IT.’

Encrypt a Filesystem within a file.

August 22nd, 2008

In order to use LUKS to encrypt a filesystem that is contained in a file, you actually have to follow the same steps as when encrypting a physical partition, plus two. These include:


  • The creation of a file that will contain the encrypted partition

  • Set up an association between this file and a free loop device, so that it can be used by cryptsetup as a block device. At the moment, cryptsetup cannot use a file as a block device directly. That’s why this step is needed.


So, let’s create the file. The following command creates an 100MB file, named “container1“, which is full of random data:

dd if=/dev/urandom of=container1 bs=1024 count=100000


To create a mapping between this file and a free loop device, we’ll use losetup (part of util-linux). Check which loop device is free in your system with the command:

losetup -f


For me it was /dev/loop0. So, I map the “container1″ file to /dev/loop0. As root:

# losetup /dev/loop0 /path/to/container1


From now on, the steps are exactly the same as before. We just use /dev/loop0 instead of the ZIP disk:

# cryptsetup --verbose --cipher "aes-cbc-essiv:sha256" --key-size 256 --verify-passphrase luksFormat /dev/loop0
# cryptsetup luksOpen /dev/loop0 encr-container1
# mkfs.ext3 /dev/mapper/encr-container1
# mount -t ext3 -o rw,defaults /dev/mapper/encr-container1 /mnt/tmp/


We can now copy some files to our encrypted partition, like on a regular disk partition. We unmount it and delete the device mappings with the following commands:

# umount /mnt/tmp/
# cryptsetup luksClose encr-container1
# losetup -d /dev/loop0


So, to mount a LUKS encrypted filesystem within a file you need to create two device mappings before you mount it for use. Of course some automation can be achieved using scripts, but you will still have to supply the passphrase in order to use the encrypted partitions.

Originally published: Here

How can a BIG company contribute to Open Source Software

August 21st, 2008

I had a very interesting conversation with my father at the weekend.  He’s the Technology Integration Director for a large Consultancy in the UK.  Over the few years that I’ve been an ‘advocate’ of Open Source software, he’s always been chipping in with little bits of development/contribution that his company has done.  However, he’s also come across some stumbling blocks which are preventing them from contributing more.  I’ll go through them in this post.  Please provide comments, and advice, where applicable.


  1.  

    1. FSF neo-free’rs  -  The first thing that put him off contributing are all those who slate and bash the big IT companies for things they’ve done in the past.  There are articles there claiming that the big companies will only contribute to open source projects for their own ends, and they’ll mess up the licensing and make unrealistic demands .etc and basically poison the code.  This type of person is also very much against anything that isn’t 100% free software.  “Open Source, it’s just a fad.  FSF and free software is where it’s at.” 

      The annoying thing about this type of person is that they’re often technically gifted, produce a very “professional looking website/blog” and can SEO so their site is the first to be reached.  They have these gifts, and are using them destructively, rather than constructively.

      SOLUTION : IGNORE

    2. The second problem he came across are company email addresses.  After doing some work using a fairly popular open source CMS, his company wanted to contribute back to the projects, and submitted some patches.  Those that were submitted by someone using their company address were ignored, yet those submitted using personal email addresses and gmail accounts were accepted, and thanked for by the team.

      How can we change the culture of Open Source so that the project teams start to look favourably on “BIG COMPANY” contributions to Open Source Software.  I welcome the new developers to the Open Source Software table.  They’re exactly what we need to start progressing.  Companies aren’t going to recommend Open Source if they’re knocked down by the developers all the time.  These are the channels through which more and more people are going to be using open source.  Embrace them, don’t piss them off! :)

    3. Sponsorship – supposing the company would want to sponsor some open source development or a particular conference.  What’s the best thing to sponsor?  It’s such a murky and political world for a n00b, that sponsoring a project takes time to understand developer opinion of doing so.  Will the sponsorship look like a take-over attempt of that project/conference, or a way to exert undue political influence.  It’s hard work.

    4. Changing the landscape drastically.
      Most projects mosey along and don’t really have much in the way of development ‘spikes;’ that is, a short period whereby contributions go up massively, and then get cut off almost as fast as they appeared.  However, with the input of a commercial group of developers working full time on a particular project until it fits their needs, this is something that could quite clearly happen.  Imagine if you’re a developer working on something that you’d like to move from point a to c… and then some external well-meaning development group subtly changes the direction of the project within a few weeks.  This isn’t forking (and that may well be the only solution to the problem, which is a pain).. this is when a small design decision is made which has a massive effect on something else.  How can commercial development groups be sure that when they jump on a project, they stick to the groups development architecture, whilst also fulfilling their own needs.  Especially when that development group may be completely non-commercial.




Anyhow, there is a list of things that have bugged me since the weekend.  I’d love to know people opinons, as to whether it’s something that needs proactively addressing, or something that is engrained in the Open Source culture that we simply have to leave to evolve.

What recommendations would you make to a large commercial organisation wanting to use/contribute to your project?

I’m going to do some number work!

August 11th, 2008

I’ve been playing about with the RadioHead’s House of Cards video on-and-off for the last few weeks.  They released the data to the video as a number of csv files, each one containing the 3D position, and ‘strength’ of each of the pixels.  It was a neat idea, and the code and data are freely available at:

http://code.google.com/creative/radiohead/

I’ve not really been programming long, and have yet to ‘settle’ on a language.  For this project, I began in python before realising that I’d have to do something much lower level.  I ended up generating images using ‘R’ – an open source version of S-Plus programming language, and ‘Octave’ – MATLAB’s open source equivalent.

It’s great that such powerful software can be found for free, and greater still that I could install it in Ubuntu by typing ‘apt-get install octave.’

It’s things like this that are really going to ‘revolutionise’ learning, and things like this that show how the utopian vision of the world wide web can come into fruition.  I was just disappointed that I didn’t have the mathematical background to utilise the tools.

However, I was always quite good at Maths, getting an A at GCSE (which was the last time I studied it).  So now, after a 6 year break, I’m attempting to do some more maths.  I feel like I need to do it now I’m working in computing; it’s a skill that’s required and one I intend to hone.  I’ve been recommended “Engineering Mathematics” by K.A. Stroud – which looks quite daunting – but should provide me with both a nice refresher course in its early chapters, followed by some chapters in which I’ll no doubt be calling on the open source community to help me get through.

The ony disapointment was that the book comes with a interactive learning CD – which only works on Windows.  I’ve emailed the publishers to see if they’d be interested in converting it into an online-tutor.  The benefits of setting up a community around such a popular book (practically every person I’ve seeked guidance for on choice of textbook has recommended it) are limitless.

It was an open letter, and was published on my blog in the last post.  If you too have a view on whether or not they should create an online version of the tutor (and thus an online community for collaboration, question exchanges .etc) please put your comments in that post, so that on the off-chance they read it, they’ll see the community support.  Please blog about it too if you’re a user of the book!

Palgrave.com – Engineering Mathematics

August 11th, 2008

Dear Sir/Madam,

Having recently purchase ‘Engineering Mathematics’ – 5th Edition, I was disappointed to find that the companion CD contained software that would only work with Windows, and not on my Ubuntu Laptop, or Mac Desktop.

Despite Microsoft having held a seemingly unassailable market share over the past 20 years on the Desktop, current trends show that more and more people are using Mac OSX and Linux on the desktop – with Linux being particularly strong in certain academic fields.

I’d like to suggest that palgrave do not release a ‘Mac OSX version’, nor an ‘Ubuntu version’ of the personal tutor; but anticipate the current market trends and put the personal tutor ‘in the cloud.’  This would enable you to centrally-manage updates and revisions to the tutor, rather than having to ‘push’ updates to your customers.  It would also enable you to leverage the intelligence of the ‘community’ of ‘Engineering Mathematics’ readers since its first publication in 1970.

I won’t preach toward you about the benefits of choosing “open source software” as your platform (although do take a look at moodle for your Virtual Learning Environment).  However, I’d like to place emphasis on the potential of the ‘community’ that such a website would create.   Instead of having students as independent satellites, you’d bring them together online to encourage collaboration and solutions.

These are the skills that the academic community are trying to encourage.  It was Tim Berners-Lee, of the academic community, who started the World Wide Web at CERN.  Please consider contributing yourselves.

Regards,

andylockran

digg story

Centos5 and ‘yum update’

August 7th, 2008

I came across a ‘small’ issue today when writing the policy for our CentOS servers.  It was about how to safely run yum upgrade.. and what happens if something goes wrong.

I have daily backups of the server, so should the worse come to the worst, it’s easy enough to recover… but I was looking for a smarter way.

RPM Rollback seemed to be the answer, however, for the life of my I couldn’t get it to work on my CentOS system.. and as it should have been running since the start, it wasn’t really an option.

What I needed was some kind of system that wrapped up the current rpm binaries, with the configs, and timestamped them so that I’d be ready to revert back to a certain timestamp.  RPM rollback would handle this theoretically.

Luckily, I have multiple levels of redundancy built into the systems, so even if ‘yum update’ does bring down my machine, it’ll have little impact other than irritation, and frustration at a waste of time.

Therefore, for now I’ve simply backed up all the rpms from /var/cache/rpm/*/packages/ into /opt/packages.070808/ ; then ran ‘yum update.’

There’s got to be a better way; please let me know if you know it!

Time to switch to Django

August 5th, 2008

Over the last week, I’ve been playing with http://po.zrmt.com.

It scratches a very small itch for my uncles company – and is going to be quietly expanded to manage more and more things for him. However, the current state of the program means that it’ll work well enough ‘as is’ – but I really need a framework for developing the ideas further.

I’m already learning and seeing better ways of doing what I’ve already done. I guess it’s just the learning process kicking in… but there are lots of things that I’m not entirely happy with. Feel free to take a look at the code :) I guess it’s time to switch to Django.

I’ve tried a couple of times to switch to Django.. but I never really managed to get off the ground. However, I think this project required that I use a framework, and will also help me make the code scalable. With the potential for this same project to be used by at least a handful of companies around Birmingham.. it’s not something I should turn my nose up at (although it’s unlikely to make me a millionaire).

I’ve always used php before for my web-coding. It’s one of those things that just seemed nice at home to me; sitting embedded in HTML. I basically used it as variables for HTML processing.. without a loop or an array in sight! I can see now where I can actually begin to use the programming to make things simpler.. not more complicated. Ooh it’s exciting.

The other thing is that it means applications I create are more likely to be used by other people. Django applications (as far as I understand it) are a collection of lots of small apps based around a solid framework. It means there’s probably going to be less work for me, and less work for other people. The comments that I made in “Using the Long Tail” are massively relevant. The more applications there are around a common base – the more random and unique the connection between two applcations.

So soon I shall be starting my move to Django. Hopefully there’ll be a few readers that’ll follow my trials and tribulations (and also help me with any potential problems). The first thing I need to do is install it on a remote ubuntu server.. if there are any gotchas with doing that – please let me know.

The Cathedral and the Bazaar

August 4th, 2008

It’s an essay which most people in the Open Source Software World have read. Yet it’s one that I’ve never got round to reading – that is, until tonight.

It’s alot easier than I expected it to be. Non-technical people should have no worries in attempting to read it (though the motivation for doing so may be reduced by at least an order of magnitude). I’d thoroughly recommend it to anyone with 30 minutes free.

The best thing about it, for me, is that it’s simple. It is in effect an empirical review of Linux’s development through the eyes of the ‘fetchmail’ project.

For those of you new to ubuntu (reading this on planet.ubuntu-uk.org) – I’d thoroughly recommend reading it if you’re a non-technical user.  It’s a nice explanation as to why the effort you’re making in switching to Ubuntu is a step in the right direction.

Philosophy

August 4th, 2008

I am a free software developer. I help develop open source software, because I believe it can be the best software. Whilst I subscribe to many (but not all) the ideas of the Free Software Foundation; I choose open source development because I think the process of open development produces the best, and most powerful software. It also promotes digital freedom – something that I think is lacking in our current transitionary phase to a ‘digitised world’.

My main gripe is not with Steve Jobs, Bill Gates, or other Software Oligarchs – it is with the lack of progress being made on the definition of “digital rights.” One has to imagine the digital world as a direct replica of our own – a parallel digital universe (if you will). If our rights are not correctly defined in this new universe, then the potential growth and benefit of this newly created digital world will be stifled.

There are a number of individuals who have fought to protect our freedoms; however, it will soon be up to the masses to decided upon our digital freedom. Unless liberated from the stranglehold of proprietary vendors, many consumers will have little relevant experience to make a choice. If one only ever experiences one reality – one can only believe/accept one reality. By giving the consumer an alternate reality to believe in, we can empower the consumer to make a free choice.

I don’t want to force people to not use proprietary vendors – I just want to make them aware of the choice, and help them regain their freedom.