By Steve Crouch.

You've put in the hard work, developed your software masterpiece and it's finally ready. How should you go about releasing it into the wild? In the rush to release your software it's easy to forget a few things. A few very important things! Let's take a look at five things you should consider prior to release...

1. Always release software with a licence

Your hard work in developing your software is just that: it's yours. Without specifying the rights you want to convey to your users, and under what conditions, they could do whatever they want with it - including making a tidy sum by selling it! And without being clear on liability, users could pursue you for damages if something goes wrong when they use it. The onus is on you to protect your interests and the rights to your software.

For an introduction to copyright and licencing, see our guide on adopting an open-source licence.

read more

It’s such a joyous time becoming a parent! Your software’s been conceived and is now going through an exciting and healthy period of development. You’ve fought your way through bouts of sickness and the odd craving (I’ll write it in… Malbolge*!!!), but everything’s on schedule. But wait… the delivery date is next week! Stay calm, remember your breathing and follow these top tips for delivering something wonderful into the world.

1 – Always release software with a licence – without a birth certificate, no one will know who the parents are.

2 – Waiting isn’t going to make the delivery any better – a late delivery can be as difficult as one that is premature. If things are taking too long, it can be best to induce labour.

3 – Version your software releases – things are going to get complicated if you don’t distinguish between who’s Sr. and who’s Jr. (Michael Jackson’s a bit of a visionary in this field, having included a version number in his son’s name “Michael II”).

4 – Provide contact information – make sure your software has a name tag just in case there’s a mix up at the repository.

5 – Download and test your release – on the big day, you should make a test run so that you’re happy that there’ll be no complications (pack an overnight bag).

For a slightly more, er… considered… take on this subject, check out my Top 5 Tips for Releasing Software article on the Institute’s blog.

* not recommended if you wish to stay sane. It’s named after the eighth circle of hell in Dante’s Inferno for a reason

This week, Newcastle University played host to a Software Carpentry boot camp run by the Digital Institute at Newcastle University, SoundSoftware and The Software Sustainability Institute. This the first boot camp to be delivered entirely by UK tutors, independent of Greg Wilson's team in Canada.

Neil, Steve and Mike joined the Digital Institute's Steve McGough and SoundSoftware's Chris Cannam for the daunting task of running the boot camp, having attended the first UK boot camp at UCL just 2 weeks ago! The attendees' comments were very positive and there was a number of valuable suggestions for future improvements - courses, like software, can benefit from iterative development.

Software Carpentry aims to teach scientists how to quickly build the high-quality software they need, and so maximise the impact of their research. The format is a workshop, or boot camp, followed by 4-8 weeks of self-paced online instruction.

Further boot camps are planned for later in the year at locations across the UK. Keep tuned to our blog or Twitter account and the Software Carpentry website for updates.

read more

By Mike Jackson.

During my work with the MAUS project, Chris Tunnell pointed me at a recent article in IEEE Software by Nicholas Barnes and David Jones on "Clear Climate Code: Rewriting Legacy Science Software for Clarity". The article describes the authors' experiences in completely rewriting an important climate code called GISTEMP.

GISTEMP produces global surface temperature datasets and was subject to criticism, because it was not publicly available and, when it was made available, because it had obvious bugs and could not be run. The authors' decided to completely rewrite GISTEMP to deliver a version that could be accessed, downloaded, inspected and run by any interested party. To this end, they set up the Clear Climate Code project on Google Code to produce and encourage the production of clear climate science software and contribute to increasing public confidence in the results.

The Clear Climate Code paper gives a concise and highly-readable account of the challenges faced by the authors during the refactoring of their code, and describes the research that has been mad possible by the new version of GISTEMP. I'd recommend it to anyone who is interested in scientific software development, or anyone who wants to know why scientific software should be accessible, clear, maintainable and portable (on that note, it is, perhaps unfortunate, that you may not be able to actually read the article unless your institution has access to IEEE papers online, but that's a subject for another blog article).

read more

Two free training events announced in the space of 2 weeks?  Don't say we're not good to you!

Incase you missed it, last week I announced that we were accepting applications for our "Using e-infrastructure for Research" summer school which will be held in August.  It's completely totally and utterly fully funded so there is absolutely no cost to the participants.  More information can be found on our website.

However the latest event I have to tell you about today is being organised by one of our Community Champions who is funded through the same project as the above summer school - Supporting e-Infrastructure Uptake through Community Champions for Research (SeIUCCR) funded by EPSRC.

Dr Pamela Greenwell, who is based at the University of Westminster, has organised a 3 day training event entitled "Biobytes", a molecular modelling event for bioscientists.  It's a 3 day event held at Westminster consisting of breakout groups, seminars, demos and practical workshops.

For more information and details on how to register, see the event page on the NGS website.  Don't delay as spaces are limited!

By Stefan Freitag, European Grid Infrastructure.

At the Mobile World Congress 2012, Andy Rubin (Senior Vice President, Mobile and Digital Content, Google) presented a speech on the rapid growth of the Android operating system. He explained that Google sees 850,000 activations every day, and the total number of activated Android devices exceeds 300 million. This gives you an impression of the extremely large target audience of the Android operating system (OS). As this audience consists of a heterogeneous mass of users, you need to consider a few things before you write an Android app.

read more

By Mike Jackson.

At the "what makes good code good and peer software reviewing" session we ran at Dev8D in February, there was discussion as to the potentially thorny issue of solo developers who may not want others looking at their code. Some people can be sensitive, defensive, or even aggressive at the suggestion that their code should be reviewed or changed, or even fixed!

From a technical perspective, sharing code is a solved problem. A reluctance to share code is a problem of psychology, team working and attitude. After all, programming is a social activity (contrary to the popular myth of the lone developers hunched in a dark room bathed in the light of a cathode-ray tube).

Encouraging developers to share code earlier rather than later, is a challenge not just within research but in software development in general (see, for example, these blog articles on "Don't go dark" and "Programmer insecurity"). In this article, various approaches to encouraging code sharing and communal ownership within a project are discussed.

read more

So we haven't been blogging a great deal since December and for good reason. We found ourselves in the exciting position of being given additional funding to enhance our network capability and also we had additional equipment to install into the cluster.

First things first however, as you may have read we have had no end of issues with the older network equipment. We had a multi-vendor environment which, while adequate for 800 analysis jobs and 1200 production jobs, wasn't quite up to cutting the mustard as we couldn't expand from there.

The main reason was the 20 Gig link between the two computing rooms which was having real capacity issues. Also, add in issues between the Dell and Nortel LAG and associated back flow problems, sprinkled with a buffer memory issue on the 5510s and you get the picture. In addition to this we were running out of 10 Gig ports and therefore couldn't get much bigger without some investment.

Therefore, the grant award was a welcome attempt to fix this issue. After going to tender we decided upon equipment from Extreme Networks. The proposed solution allowed for a vast 160 Gigabit interconnect between the rooms broken into two resilient link bundles in the Core and an 80 Gigabit Edge layer. In addition to this connection we also installed a 32 core OM4 grade fiber optic network for the cluster which will carry us into the realms of 100 Gigabit connections, when it becomes available and cheap enough to deploy sensibly.

We now have 40 x 40 gigabit port, 208 x 10 gigabit ports and 576 1 x Gigabit ports available for the Cluster.


The new deployment utilises X670s  in the Core and X460s at the Edge.

The magic of the new Extreme Network is that it uses EAPS, so bye bye Spanning Tree and good riddance as well as MLAG which allows us to load share traffic across the two rooms so having 10 Gigabit connections for disk servers in one room is no longer an issue.

Then it got a bit better. Due to the Extreme OS we can now write scripts to handle events within the network which ties in with the longer term plan for a Cluster Expert System (ARCTURUS) which we are currently designing for test deployment. More on this after August.

Finally, it even comes with its own event monitoring software, Ridgeline which gives a GUI interface to the whole deployment.

We stripped out the old network installed the new one and after some initial problems with the configuration, which were fixed in a most awesome fashion by Extreme got the new one up and running. What we can say is that the network isn't a problem anymore, at all.

This has allowed us to start to concentrate upon other issues within the Cluster and look at the finalised deployment of the IPV6 test cluster which has benefited in terms of hardware from the new network install. Again, more on this soon.

Right, so now to the rest of the upgrade we have also extended our cold isle enclosure to 12 racks, have a secondary 10 Gig link onto the Campus being installed and have a UPS. In Addition to this we refreshed our storage using Dell R510s and M1200s as well as buying 5 Interlagos boxes to augment the worker node deployment.


We also invested in an experimental user access system with wi-fi and will be trying this out in the test cluster to see if a wi-fi mesh environment can support a limited number of grid jobs.  As you do.

In addition to this we improved connectivity for the research community in PPE at Glasgow and across the Campus as a whole, with part of the award being used to deliver the resilient second link and associated switching fabrics.

It hasn't been the most straight forward process as the decommissioning and deployment work was complex and very time consuming in an attempt to keep the cluster up and running as long as possible and to minimise down times.

We didn't quite manage this as well as expected due to the configuration issues on the new network but we have now upgraded the entire network and have removed multiple older servers from the cluster to allow us to enhance the entire batch system for the next 24 - 48 months.

As we continue to implement additional upgrades to the cluster we will keep you informed.
For now it is back to the computer rooms.

This news article appeared on the GridPP website and is worth reposting to our blog as it gives an overview of the collaborations efforts to date within the WLCG and with the Non High Energy Physics (HEP) communities.
GridPP At The Top Of Europe

Generally, we don't repost news items on the blog but this BBC article gives a good indication of the changes underway globally for implementing IPv6. currently the Glasgow Scotgrid test cluster is being revamped post our last spending cycle and we are embarking on a full test programme of IPv6 specifically around running Grid services. As this work progresses we will regularly update the blog.

We were in attendance at the LHCONE  meeting at KTH in Stockholm last week. The purpose of this collaboration is to investigate the efficient use of networks globally for LHC research. As usual it was an excellent meeting where the technical mechanisms for current and future network deployments were discussed and considered.

The agenda can be found here. Some of the highlights of the meeting included an excellent presentation by Erwin Laure on the Swedish and Scandinavian Super Computing and Grid computing infrastructure, Joe Mambretti's presentation on the GLORIAD global research network, Mike O'Connor's discussion on the technical configurations required to avoid asymmetric routing issues between the LHCONE and the current production networks and  Domenico Vicinanza's presentation on Perfsonar MDM.

In addition to these presentations technical discussions surrounding various technologies surrounding bandwidth reservation, ultra high speed networking and  Open Flow technologies were held. As these discussions develop through the network architecture groups we will keep you up to date. 


Also, the weather in Stockholm was exceptional and the KTH Campus is worth a visit for its architecture alone. I would like to thank our hosts and all the other attendees for making this such an enjoyable and informative couple of days.




 

We mention things you should do when developing software quite a bit. But we were asked an interesting question at the Software Sustainability Institute’s Collaboration Workshop this year: what things shouldn’t you do when developing software?

Come on, there has to be some. And there are – many! But let’s focus on five of the big ones…

1. Don’t develop code you can’t maintain

This has got to be high on the list. Code can turn into spaghetti from out of nowhere, and it’s always worth avoiding. Best to get into good habits early on in the project!

2. Don’t make your software difficult to build and install

We’ve all experienced this with other people’s software. If user’s can’t install it, they’ll move on – perhaps to a piece of software that has inferior capabilities. Why not make it easy, and simplify (or even automate) the build and install processes that are so often fraught with peril?

3. Don’t keep the source code to yourself

If you hold the source code in only one place – your development machine – and you lose it, you only have yourself to blame. If your development machine is your laptop, it’s even easier to lose. Avoid the pain and use a suitable source code repository from the outset!

4. Don’t forget documentation

Writing documentation is boring, but it is necessary. It’s your primary means of communicating your users what the software does and how to do it to. As with difficult build and install processes, you risk disenfranchisement if users can’t find out what they need to know.

5. Don’t overlook testing

Features, features, features. But if you neglect testing your software, you risk losing users, users, users. In the rush to implement and release a really handy new feature, ending up with a release that doesn’t work will not instil confidence in your product. And including a means for developers to run a solid set of automated tests and implement their own is very useful as a fail-fast development environment when they want to modify it themselves.

Well, that’s my five. You’ll notice I haven’t covered any software release “don’ts”, but that’s because I’m currently putting together a related top 5 list of software release “Dos” So these are just scoped to software development. If you’re interested, you can check out these in more detail in in my post on the Software Sustainability Institute’s blog.

Maybe you disagree with my list above, in which case let me know what you think are the big software development don’ts.

Yes it is that time of year again.  I've spent this morning opening registration for the 2012 SeIUCCR e-infrastructure summer school.  Why a whole morning you may ask?

Well by the time you've double and triple checked the registration form, put the web page live, sorted out the Facebook event page, written the advertising blurb, put together the news bulletin containing the announcement and tidied up another 101 loose ends, it takes a while!

The summer school is taking a similar format as last years successful event.  It will run from lunchtime Monday to lunchtime Thursday with a mix of presentations, hands on and consultation sessions.  It will cover cloud, grid and other e-infrastructures to ensure that attendees gain the widest possible knowledge of e-infrastructure in the UK.

The summer school is primarily aimed at UK engineering and physical science PhD students and post docs but researchers from other disciplines can also apply.  The school is fully funded including meals, accommodation and travel - all you have to do is tell us of a problem or issue in your research that could potentially be tackled by the application of e-infrastructure!

For more information and to apply for a place visit the event webpage.

By Aleksandra Pawlik, Agent and PhD student at the Open University.

1. Think about your audience

Depending on the nature of your software you will need different documentation for different audiences.

The first group, which you always have to address, are your users. Who are they? If you make assumptions about the level of their knowledge, both science- and IT-related, make sure that you list these assumptions in your documentation.

The second group are users-developers, or just developers. The former are scientists who not only use your software, but also develop new modules in order to progress their research. The latter may be software engineers who support a group of scientists that use your software.

The final group are people who maintain and provide IT infrastructure at research institutions. If your software needs to be installed on a server or a cluster, you should provide documentation that enables an administrator to set it up and support its use.

read more

By Mike Jackson, the Software Sustainability Institute.

On Monday and Tuesday this week, members of the Software Sustainability Institute joined over 40 scientists for a Software Carpentry boot camp at University College London. Software Carpentry aims to teach scientists how to quickly build the high-quality software they need, and so maximise the impact of their research. The format is a workshop, or boot camp, followed by 4-8 weeks of self-paced online instruction.

read more

By Quanbin Sun, Research Student at the University of Salford.

On 21-22 March, the Collaboration Workshop 2012 took place at The Queen’s College, University of Oxford. The workshop mainly focused on software development in academic projects and attracted sixty researchers and developers. Thirty two topics were raised and discussed during the two day event and more than twenty lightning talks were presented.

Among these discussions and topics, I enjoyed the ones that were related to the collaboration between scientific researchers and software developers and a possible new species for academic research projects - the scientific software developer, who plays a dual role in the research.

read more

I first heard about “code smells” during a session I was chairing at Dev8D 2012 on “what makes good code good?”: Ian Bayley, from Oxford Unviersity, suggested “no ‘code smells’”. I’d never heard of this term so I turned to my trusty friend Google to see what was what. Although I hadn’t heard the term before, it turns out that I knew what code smells are… source code that just looks “odd” or doesn’t feel quite right, which are signs that suggests to a developer that refactoring might be in order.

Not only were many of the smells familiar but the “deodorants” were too. For example, replacing arrays that are used as records with objects,

String[] row = new String[3];

row [0] = "Liverpool";

row [1] = "15";

can be replaced by,

Performance row = new Performance();

row.setName("Liverpool");

row.setWins("15");

Or, replacing nested conditionals with guard clauses e.g.:

double getPayAmount() {

    double result;

    if (_isDead) result = deadAmount();

    else {

        if (_isSeparated) result = separatedAmount();

        else {

            if (_isRetired) result = retiredAmount();

            else result = normalPayAmount();

        };

    }

    return result;

};

can be replaced by

double getPayAmount() {

    if (_isDead) return deadAmount();

    if (_isSeparated) return separatedAmount();

    if (_isRetired) return retiredAmount();

    return normalPayAmount();

};

The term “code smell” is attributed to Kent Beck in Martin Fowler’s book Refactoring, Improving the Design of Existing Code (Addison-Wesley, 1999, ISBN 0-201-48567-2). There are lots of online resources that will teach you how to spot smelly code and help with the deodorising so that your code ends up as pure as an Alpine breeze. As a starting point you could try Martin Fowler’s own “catalog of code smells and refactorings“, which lists both symptoms and cures, judiciously highlighted with examples. A complementary resource is Mäntylä and Lassenius’s “bad code smells taxonomy”. This groups together bad smells into a useful, recognisable and amusingly named taxonomy. As a couple of examples they have,

• The Bloaters, including long methods, large classes, long parameter lists and data clumps (sets of data like 3 integers for RGB colours which could be encapsulated).
• The Dispensables, anything which can, and should, be removed including lazy classes that don’t do enough, duplicated code, dead code and speculative generality (code which “might possibly be useful someday, maybe” but which incurs maintenance overheads).
• Other classifications are the change preventers, the couplers and the object-orientation abusers.

Another useful resource is SourceMaking’s pages on refactoring which motivates refactoring before describing many code smells and their refactorings.

Static code analysis tools such as CheckStyle or Pylint can also automatically detect (but not fix, that’s your job) code smells, and these might become a useful part of an nightly test system for your software, or part of a continuous integration server.

I hope this answers your question and the resources above will help you to write more fragrant code in future!

At the Software Sustainability Institute I’m often asked - unsurprisingly – to evaluate the sustainability of software. This typically leads to a report for the developers with observations and recommendations for improvement. Wouldn’t it be better if there was some way of evaluating your own software?

There is! Having a third-party assess the state of your software in some way, be it a colleague testing the install process and documentation to provide feedback, or having the Institute perform a full evaluation is always useful. However, developing the skill to impartially self-assess your own software is invaluable. Adopting an objective ‘green’ user or developer perspective – removing your inner assumptions and knowledge of the software from the equation – can only help your software to become better.

Not only are the Institute’s processes for evaluation available for you to use yourselves, but there is now a very helpful, lightweight and quick sustainability evaluation you can do on your own. You just fill in a simple web form with details about your software, and it returns a list of recommendations (with helpful links) on how you can improve your software. Simple! It investigates a number of key areas related to the sustainability of your software, including the processes for building and installation, documentation, availability, support, licences and source code structure, amongst many others.

You can check out this nifty evaluation resource.

You're about to embark on the development of a new piece of software. Of course, there's a whole host of things you should do. But let's look at the flip side of the coin: what shouldn't you do?

1. Don't develop code that you can't maintain

It's all too easy to fall into bad habits, especially if you're up against a tight deadline. In the long run, good coding practices will save you a great deal of time and stress. It will also make growing the software a far more enjoyable process. After all, who likes fighting their way through an unclear spaghetti of code just to make a basic modification?

At the start of a project it's not always clear who else will become involved in developing the software. If other developers join your project, they will need to understand the code before they can develop it themselves. Good code can avoid a lot of problems (and embarassment when you show other developers!).

Writing readable, commented, easily understood and well-structured code doesn't take as much effort as you'd think, and it's even easier with the Institute's handy guide on developing maintainable software. You should also code defensibly - avoiding unnecessary and complex dependencies and deprecated interfaces - which is covered in our defending your code guide.

read more

Hopefully you'll have already seen the announcement on one of our many communication channels such as our website, Facebook page or Twitter feed but if not then read on.

Many of you will remember the changes we brought in last year in April 2011.  Due to funding restrictions, we had to reduce the CPU allocation of all users to a maximum of 2000 free cpu hours in one year.  You can read the original announcement on our website.  As we are now a year on, all NGS users can apply for another free 2000 cpu hours.

If you are looking for some proof of concept computing, a "sand pit" area for your PhD students or to test concepts before purcahsing more hours etc then this is an ideal opportunity.

If you have any queries at all then don't hesitate to contact the NGS helpdesk.

By Ilian Todorov, Advanced Research Computing Group, STFC.

This article represents my personal point of view. It is related to Dirk Gorissen’s blog post “The researcher programmer, a new species?” and discussions from the “Scientific Software Development and Management” group page of LinkedIn, which started after the Software Sustainability Institute’s Collaborations Workshop 2012 (CW12). These discussions pertain to why the software engineer in academia needs recognition.

Software has become a technique of choice for many scientists. It is often considered to be free, but this often means “free to academia”. Somewhere down the line, someone has paid for it. Someone has invested their labour in writing code instructions to implement a scientific methodology of some kind. In most cases, this was a postgraduate (PG) student and/or post-doctoral (PD) researcher attempting to automate and simplify the workflow of their research routines.

Times have changed enormously in the last 20 years. However, for the researcher in academia, one thing has remained constant: their career progression is based on their research performance, as measured by the impact of their research papers in peer-reviewed journals. More papers in high impact journals leads to more success and recognition, and better chances when applying for funding or academic jobs. In contrast, software development has diversified. It has become a well-defined profession with many sub-fields and computer languages. This is not surprising for an industry that governs our lives at home – PCs, games, smart devices, apps – at work and anywhere we go – databases, financial transactions, GPS, industry. It has also become a discipline in its own right in higher education as “informatics” or “computer science”.

read more

You've decided to make the leap and move your software to an open-source repository. You may be doing this for reasons of openness - to encourage a community of developers to evolve around your software - or economics - to outsource the maintenance of your source code repository, email lists, wiki, blog and other services to a third-party who offers these for free.

Before you plunge into this brave new world, we invite you to consider our five top tips for moving your software to an open-source repository, to ensure that your migration is a pleasurable one.

1. Get buy-in from your developers

As your developers will be using the repository day-to-day, they need to be on-board from the outset. You do not want to take the time to create a project in a repository only to have it sit there unused, because your developers don't like its revision control system (or can't see the point of it).

Talk to your developers and let them know why you're moving to a repository and explain the implications and benefits for them. If you can convince your developers that the move is necessary, your migration to a repository is far more likely to be successful.

read more

Hectic and fascinating are the words that best describe the Ocean Sciences Meeting 2012. It's a week-long conference attended by more than 3000 participants, including biologists, chemists, physicists, mathematical modellers, oceanographers, engineers and educators. There was a large number of poster presentations, which I found to be a great opportunity to interact with people and hear what they had to say about my research. In this blog, I'll take a look at the software-related aspects of the nearshore processes and data-management sessions (if you want more information, read my conference report).

Nearshore Processes

Nearshore processes are ocean and beach processes that relate to the nearshore portion of the continental shelf. They shape the coast and include waves, winds, tides, currents, the seafloor and the coast. These processes are very important, because they are significantly affected by climate change. Indeed, climate change has been increasing the frequency and strength of extreme storms, which in turn cause erosion and flooding of cliffs, lowlands and estuaries. This has important impacts on coastal and estuarine ecosystems and on fisheries, shipping and tourism.

read more

My office-mate, Mario Antonioletti, forwarded a blog post by Randall Degges on Successful GitHub development. The post proposes best practice in programming on GitHub-hosted open-source projects. Randall's recommendations overlap with those proposed by other open development advocates (e.g. OSS Watch and ourselves!), but these are always worth restating!

For maintainers, the emphasis is on providing official documentation, using separate stable and development code branches in the repository, publishing test runs and using an issue tracker. Contributors are, in turn, recommended to read the documentation, look  at the issue tracker (before making changes), comply to style guidelines and ask if unsure. Though Randall focuses on GitHub, his guidelines are applicable any open-source project.

read more