Even voorstellen

Onze nieuwe kracht is Geertje van den Brink – Bronkhorst. Zij is per 1 maart Operationeel Manager bij Seecr. Afkomstig uit de ingenieurswereld, neemt ze 15 jaar ervaring in projectbeheersing mee. IMG_8196

In haar vrije tijd zet ze zich in voor de leefbaarheid van haar dorp. Geertje volgt ook nog een opleiding coaching, waarbij paarden als waarnemer van menselijk gedrag worden ingezet.

 

Wat Botswana en Nederland gemeenschappelijk hebben…

Zo aan het eind van het jaar is het een goed moment om terug te kijken. Kijken we wat langer terug, dan zien we dat er verandering komt in een slepende situatie: in Nederland investeren overheid en non-profit organisaties jaarlijks voor miljoenen in kostbaar onderzoek. De research output is echter zelden beschikbaar voor het publieke domein, want het wordt gecopyright door uitgeverijen. U vindt de opgedolven wijsheid dus niet voor iedereen toegankelijk in de daarvoor bedoelde Universiteitsbibliotheken. Ze wordt daardoor onderbenut voor het algemeen belang. Ook in een land als Botswana, dat net zo goed kennis hard nodig heeft, speelt dit probleem. Maar er lijkt een doorbraak te zijn om deze ‘tragedy of the commons’ te doorbreken.

Subsidie onder voorwaarde

Sinds deze maand geeft NWO in Nederland alleen nog subsidies aan onderzoeken, die hun publicatie openbaar toegankelijk maken. Dat helpt. Wie de leiding in Botswana neemt om de tragedie te doorbreken, kan ik u zo even niet vertellen, behalve dat ook zij al heel lang behoefte aan toegang tot informatie hebben voor onderwijs en research, om een duurzame ontwikkeling te realiseren.

Narcis

In 2008 besloten wij om die reden onder andere dit land via de non-profit organisatie Eifl (Electronic Information for Libraries) aan een systeem te helpen, dat lijkt op Darenet. In Nederland werd dit de voorloper van Narcis. In Narcis is goed te zien welke publicaties ‘open access’ zijn.

Blog illustratie Botswana Nederland

December is een goede maand om te bedenken dat wij, terugkijkend, misschien een steentje bijdroegen aan de groei van de ‘open access’ beweging, in Nederland, èn in Botswana.

Mijn eerste werkweek @SEECR

Altijd spannend, beginnen bij een nieuw bedrijf. Per augustus ben ik deel van het team van SEECR. Ergens starten voelt als opnieuw leren zwemmen. Als de eerste slagen op natuurijs. Is het water warm genoeg? Het ijs sterk?

Ik wind er geen doekjes om: wat ik hier zie, heb ik nog nergens meegemaakt. Op vaste momenten op de dag, in een natuurlijk ritme, wordt met stand-ups op een transparante manier onderling kennis gedeeld. Teamsgewijs werken, elkaar aanvullen en helpen is hier volkomen natuurlijk. Ik verkeer niet nabij koud of diep water, maar in een biotoop die natuurlijke groei mogelijk maakt. Behalve efficiënt en slim, komt het mij gezond en menselijk over. Ik ben reuze benieuwd hoe hier wordt gewerkt aan met code relevant maken van Big Data voor zoekers. Daar word ik vrolijk van!

About scalability, efficiency, pairs and time

At Seecr we continuously both scale up and scale out our systems, but we also improve efficiency continuously.  Here is why and how we do it.

Scalability versus Efficiency

Quite often, people think that scalability is everything. But scaling an inefficient system, if at all possible, is going to be expensive and might even stop you completely. It certainly looks nice when Amazon adds 100 extra machines to your system in an instant, but it might just as well be a gross waste of resources. And as long as money is not a problem, the problems related to inefficient systems can remain hidden for a long time.

Why is Inefficiency a Problem

We are now living in an era where more and more people need no explanation as to why inefficiency is a bad thing. Much to my delight, the mere idea of wasting something is becoming increasingly sufficient to let people act. So I won’t go into that direction. There is another problem however.

Inefficiency is caused by something. And when the time comes that you do want to improve it, you need to address the causes. And then it might turn out that you are a little too late….

Here are two significant causes we have observed frequently:

  1. Programming negligence
  2. Wrong algorithm

1. Programming negligence.

Programming is quite a difficult task and each problem has different aspects that need careful attention. There are the matters of primary design, testability, consequences for other features, refactoring, selection of libraries, code style, readability and intention, integration with other parts, packaging, upgrading and data conversion and on goes the list, endlessly. That’s the nature of software.

Efficiency is definitely somewhere on that list of aspects. It is all too natural that, once code functionally works, everyone is glad and moves on to the next task. But at that time, some aspects might not have received the attention they require, and efficiency is often among them.  If this goes on for some time, you’ll end up with many places needing small fixes.

But you can’t fix it later.  The reason for that is as profound as it is simple: there are just too many small things, each of which contributes only little to the problem. It is the power of the many. Addressing each of these problems requires you take time to delve into them again, with only little reward: each single problem solved improves efficiency only a little. You’ll have to work through them all to see results.

If only you would have made different choices in the first place, when you were initially writing it…

2. Wrong algorithm. 

A problem can often be solved with very different solutions. Naturally, you first pick a solution based on your experience and understanding of the problem, then work it out. Often it becomes clear during the process that another solution is a better fit. This is a direct result of an increased understanding of the problem while working on it. Deep understanding of the dynamics that arise when the problem and the solution interact might also arrive later. For example when you run tests with fully populated data sets and unforeseen usage patterns that do not appear in testing environments. It turns out that you will need a (completely) different algorithm to solve the problem.  Back to the drawing board. That’s the nature of software too.

Dead in your tracks

Both problems, many small inefficiencies and wrong algorithm, are not just a pair of non-optimalities of your system.  They both have the ability to simply place required throughput and responsiveness beyond your capabilities and budget.  Because both problems require a complete rethinking of your system: go over all the details again, and improve them, or go over the main design again and change it. This costs a lot of time, and, most importantly, it takes the time of the most specialized people.  If you could only have made other decisions when the system was first created….

What are solutions?

Let me get this straight first: getting better programmers, smarter architects or more elaborate processes including a lot of quality assurance does not solve the problem. While some people do learn faster or can have more things on their minds, and while some elaborate processes do catch more problems, they basically only ameliorate the situation marginally.  They do not address the limitations fundamentally.

So what is the fundamental problem then?

The fundamental problem is that:

  1. people are given too many things to worry about at the same time.
  2. people are given too little time to learn and understand well.

In short, they suffer from Continuous partial attentionNow, the solution becomes almost evident: use more people and give them more time.

In Shock?

You would probably say: “but I can’t spend more people and more time, you know that budgets are tight and your solution is not realistic.”  Well, if you really think that, stop here.

If you think you are in control and you can change things for the better: continue.

First: Pair Programming

The most profound impact on software quality (that’s what we’re talking about) I have ever seen is the working in pairs.  Pairs have double the capacity to deal with al the things-to-worry-about.  But that’s only the beginning.  The dynamics that play between pairs is really what pays off.  Each one has his or her own set of aspects of interest, and stimulates the thinking process about the other.

Pairs have the advantage to easily mix.  Depending on the task at hand, on personal preferences, on tiredness even, one person might switch with another.  This new combination will pay attention to other aspects with a fresh mind.  Also, there is knowledge exchange happening automatically.  Purposely shuffling with pairs is a strong tool in the hand of a team.  It allows you to increase the number of aspects that deserve attention.

Second: Time

But having more brains at the task is only half the solution.  The other half is giving these brains enough time to learn and understand the problem.  So if one pair is functionally done, prepare to let a third person replace one of them.  There will be points left over for him or her to think about. While the first pair focussed on getting it functionally right, the second pair looks at other aspects such as efficiency.  In fact, it is a simple application of Make-It-Work-Make-It-Right.

Conclusion

It all comes down to the careful allocation of people and time. Look at their skill, interests and allocate enough time. I can’t stress this enough: proper allocation of time means saving time. Full stop.  When you rush, you are going to regret it later; and someone is going to pay for it.  It is therefore an act of will to allocate time in the beginning, right when you are solving the problem.

The only way to build scalable systems is by first making efficient systems. And for that you need to allocate enough time and people before you scale up.