Stephan Schwab - training tag

The connection between accounting and sub-standard tools for developers

Sat, 05 Apr 2008 18:16:30 GMT

First of all I have to say that I'm really surprised by the feedback I'm getting. The post What's wrong with software development in large corporations has been the most read post on my blog ever. More than 3,000 people read it on the first day alone.

The other night when I wrote the post I could have gone on for a while. I didn't because it would have been far too much. So allow me to add a few things in this second installment.

Electricians with only a screwdriver and no other tools

Would you accept an electrician who shows up at your house with only a screwdriver in his hand? You probably expect such a professional to carry a number of specialized tools, some basic material and other things needed to perform the work - wouldn't you?

Well, I can report that one and a half year back we had exactly that happen to us in Panama City, Panama. We had some very strange things going on with the electric installation in our rental apartment (current on wires that should have none) and the administrator of the building called an electrician. After a three hours wait a man in his fourties arrived by taxi and from his bulky pants he pulled a single big screwdriver as his only tool. He then started to loosen and tighten some screws holding wires in place and never managed to solved the problem. I had the opportunity to see for myself a good part of the building's electrical wiring and I have to say that it looked to me as it were at least 50 years old. You can imagine that I was quite surprised when they told me the installation were only 10 years old. The explanation was that, yes, the materials used were taken from another building that has been torn down.

Now you might say that you can understand that because it's Latin America, Panama were a third-world country, etc. I'm willing to accept that - to a certain extent.

The connection between accounting and sub-standard tools for developers

But how do you explain why companies of any size let software developers work with inadequate equipment? By inadequate I mean slow CPU, few memory and a small screen. Some would probably say that I'm a big sucker for screen real estate as I had a 21" monitor on my desk back in 1995. But on the other hand I always paid the same amount for my computers through all the years. There is this saying that a software developers needs a new system every year and a half and see that exactly corresponds to Moore's law. Every 18 months you get twice the power or can get the same for half the price.

Software development is expensive and it takes a while to get it right. The professionals who possess the knowledge and long-term experience charge a premium and to me it doesn't make sense to save a few Dollars or Euros on equipment. I don't want to start talking about salaries here, but I think it's safe to say that a high-end, state of the art workstation with the biggest screen you can get is still far less than a month's salary. The question is why do so many companies hire expensive developers or pay expensive consultants just to slow them down artificially by letting them work on 15" laptops with a locked-down Windows XP and as little as 512 MB of memory? With that kind of system you start eg. Eclipse and Firefox to try out the webapp you are supposed to develop and then have to plan ahead of time where you are going to click ... Could be seen as some elaborate form of torture ;-)

Looking a bit deeper one will find that budget constraints are usually to blame for that kind of nonsense. The project gets funded out of one department's budget and the hardware is usually already there or provided by a central services department. And the services department usually charges an outragous amount - although it's kind of a fictious charge, as it is only ledger adjustments - for sub-standard hardware. So it's not that the company doesn't have the money. It is simply not available where it should be.

But that is only one part of the equation. In some cases one may want to ask why a department that is charged with software development doesn't have adequate hardware at all in the first place. I can't tell for sure how the taxation rules in the US are with regard to this, but in Germany the problem has always been that companies can only deduct expenses for new computers over a three years term. I remember it being five years and that has been reduced after a lot of controversy.

Actually it should not matter much how the taxation works. Software development is a big investment, the professionals doing it should be supported by the best tools money can buy in order to create the best software possible for the company they work for. So in the end it comes down to a leadership issue.

There are companies that get it

There are some companies that really get it. For example Matt Raible reports that LinkedIn is giving even contractors new MacBook Pros for their work and other companies buy their developers Macs with 30" screens at the workplace and at home. That truly expressed how they value and appreciate the work these highly skilled individuals are doing for them. Should be common but sadly it isn't.

What's wrong with software development in large corporations

Thu, 03 Apr 2008 04:28:13 GMT

How much time does it take to become a knowledgeable professional? 2 years, 4 years, 10 years or half a lifetime? How much time does it take to become a good artist such as painter, musician or composer? What about the saying that good artists are born? That somehow implicitly means that either you are a good artist by birth or you aren't and you can try as hard as you wish, you'll never become a good artist even if you had an eternity.

What has that all to do with software development? If work in the IT department of a larger corporation or as a consultant helping them, you'll certainly can tell by now what I'm about to write next. It doesn't matter where you look, you'll get to know over time hundreds of people working as programmers for those corporations and it's very, very rare to encounter someone who really has a clue about what he's doing. Think about something as common as object-oriented programming. How many people do you know who really got it and are able to come up instantly with a good OOP design with good separation of concern, all those little collaborator objects talking to each other and so forth? I guess you will have to think hard. Sure there has been that one guy or gal but that's about it - isn't it? But you certainly can easily remember a fair number of people who call themselves software architect and the only contribution to the project you were working on where a lot of blockage and repeating stuff that sounded awfully similar to some vendor's sales talk to praise their rigid waterfall'ish tools.

But why is that? I think it has something to do with education and the expected behavior good employees should show. Let's start with the latter.

Today's corporations are still created after the organizational model of the military. Any military organization has a clear chain of command where orders flow down from high rank to lower rank and the lower rank is expected to execute the order given by the higher rank without questioning it. That works great in a battle situation - at least that's what you usually hear about that. But let's ask for a moment what that battle situation actually is. It's quite simple and cruel at the same time. Any battle ends with a large number of dead people and low ranking soldiers are disposable "units". Nobody really cares if they die. The less the know about the overall picture and the more they believe in what you want them to believe the easier it is to make them accept the risk, which nobody would ever accept under normal circumstances. But let's not slip into talking too much about that topic. The point is that in the military it is not common to not inform lower ranks about the overall picture, they get some strong motivation to do a stupid thing (put themselves up for being shot) and they are made to follow orders without questioning - and of course there are sanctions put in place for those who do question and not follow an order.

Now let's look at any corporation. The chief in command is the CEO. He has a number of officers (CFO, CIO, etc)., which command other officers at the departamental level and so on until it comes to the ranks of sergeant, corporal leading groups of privates. The sergeants and corporals are the low ranking line managers and the privates are the common workers. So in the world of corporate software development we can identify the programmer and tester as privates and their team lead as some kind of corporal. The project manager might be some kind of sergeant. As you can see that all so important project team is not so important after all. At least from the vantage point of the commander in chief, the CEO commanding the corporation.

How much time does it take to become a good soldier? Most armies of the world train their lowest ranking members for something betwen a 12 or 18 months. In the beginning they perform some tests to determine whether a person is better suited for less complex work, that is usually Army or manufacturing in most corporations, or for more complex stuff, which is usually Navy/Air Force or administration/support/development. Then basic training starts and the most important part of that basic training is to learn the rules. That is to make people understand that they have to obey orders or suffer consequences. In the corporate world the common consequences are reassignment or you simply get fired. And as people's well-being depends usually on their job, everybody understands all too well that it's a bad idea to show too much of an independent mind.

So... What is software development in its core? Is it some other form of manufacturing? Or is it research or art or what? Can you construct software or is it more that you create software?

It depends on how you answer these questions whether you want corporate soldiers (sergeants, corporals and private) to work on projects or not.

In my opinion software development is something between research and art. After all the word development implicates that something gets created for a purpose and before you develop you will have to know what the purpose is (the problem you want to solve; what you want to improve), what the environment looks like, who will use it, etc. So that includes a very large research component as well. I think one can describe software development as some kind of exploration with the goal to create something with a certain level of quality to solve a problem that is well understood.

That doesn't sound very much like a battle situation - does it? One may think that a good part of software development is about learning, reflection, experimentation and other activities usually associated with scientists. And when it comes to write the code then skills and attitudes, which musicians are known to possess, will be helpful. It needs an awful amount of time to practice test-driven development and be disciplined enough to implemented only what's needed to solve a problem described by a user story.

What's the situation of most corporate programmers? They get hired based on their resume and then put to work on a project. Typically the HR department decides upon hiring or not and they do some kind of pattern matching by looking at the keywords. They get a shopping list from the department and if it says Java, Struts and Junior Programmer, they will easily find someone with less than 5 years experience. Now that poor soul arrives and gets assigned some tasks by the team lead (sent into battle) and there we go. Will that person be creative? Will she be part of a group analyzing the business problem the team wants to solve? Usually not, because everything has already been done and now it's about to write the code based on a specification, which is kind of the battle plan.

Now here is a suggestion. Wouldn't it be great to employ the master/apprentice model known since the middle ages? Why do we allow inexperienced people to mess around with the most important thing in software, which is the code? I think a well motivated apprentice working alongside with a good master will evolve into a true master himself over the years. He will take more and more load off his master as his skills evolve. He will understand why the master does employ certain techniques and why he doesn't use X or Y in different situations. That takes time. But the quality will be higher and so will be the capacity of the whole team, because the team will be comprised of more masters as time goes on. That's an investment and corporation leaders would be wise to invest into the future of the corporation they serve instead of only stare at the short-term shareholder value created in any given quarter.

How long does it take to become a master? In my home country Germany plumpers, carpenters, painters, electricians and others spend three years as apprentices and easily up to eight years as journeymen before they are allowed to lead their own shop as masters. So that old saying "learn how to program in ten years" isn't that wrong after all. Between 4 - 6 years formal study at the university followed by an apprenticeship of maybe 2 years means one learns for at least 6 - 8 years. The next few years are then to hone one's skills and after 10 years one will know the whys and hows of software development.

Thoughts on the ideal project team

Tue, 05 Feb 2008 21:03:35 GMT

Update:

When I posted Thoughts on the ideal project team a few days ago I received a comment to which I'd like to respond in more detail here. I really appreciate that kind of comment. Who ever is xcosyns: Thank you!

Commenter xcosyns in Thoughts on the ideal project team at my blog:
Often you will need someone handling the infrastructure, uat, itt should match the real production environment. And from experience this is not always the case, and not that easy to achieve when multiple applications have to interact, one way or the other, together? And we are not yet talking about backups, db replication, san drives, authentification systems, load balancing…

Development infrastructure should be as simply as it makes sense. And it's the developers who should have control over it. For example in our case mostly I do that and when you have some experience it's not that hard to do. I encourage my fellow team members to not limit themselves to the "code monkey" role, but learn how to administer their own systems and team systems. My belief is that every software developer needs to be able to do sysadmin duties - at the very least for those systems he works with on a daily basis.

By that I don't mean to say that SAN drives and load balancing in a production environment should be maintained by the developers of a system. That's the job of an operations group. But developers who can do this for their little development environment have a much better understanding of these things and know what to do and what not to do when creating an application for such an environment.

Also, developers can really benefit of a DB expert, often java developers really sucks when it comes to db performance, and most developers are rarely aware about what their db can really do. A good DB expert can speed up legacy applications with no or minor changes.

You are right. Often Java developers really suck when it comes to DB performance. Why is that? Probably because their education is too shallow. One thing is to merely understand the language and a few common libraries/frameworks. And another thing is to understand software development in a holistic way. That's the result of experience and natural curiosity. For example in my own case I have been forced to learn Unix system administration, networking (up to managing a larger network for an ISP I co-founded; think of the whole zoo of routers, access concentrators and the various protocols such as BGP, OSPF, etc.) plus several high and low level programming languages including all the, literally, zillions of libraries and frameworks for those. It took a while, but I consider this the difference between an apprentice and a master. The ideal project team practicing agile development should be comprised of masters. There is a room for apprentices. You can easily assign to each master an apprentice and delegate some tasks to those.

Some documentation should be written, the business analysts can do the functional part, developers or the architect can do the technical part. But when the project/projects/teams scales up it becomes a time-consuming task and it can partially be delegated.

Agreed. If you have a need for more extensive documentation, then you certainly can add a tech writer. I see one question that remains. If you make the tech writer part of the team and follow the very good practice of "done done" for your user stories, that would mean that a story is only "done done" when the documention related to the story is completed as well. In this case you add another constraint besides the programmers. Probably it depends on the type of application and the target market whether that makes sense.

Also someone needs to follow up the development process, a project manager or a team leader. Someone that can prioritize the tasks in function of the business needs and business gains, someone that has a global view of the project and the company. And managing the budget, logistics, recruiting, etc…

For that there is the role of the Product Owner. His job is to prioritize stories based on business value. The customer is the only one who can really know the business value of stories so he should be the only one providing that indicator. Budgeting, logistics and recruiting are unrelated to the work the team is supposed to perform. You form the team before you get started - obviously ;-) - and then that's your team. I would not move people in and out of a team, because that negatively affects the accumulated domain knowledge and slows down.

The original post:

As I'm preparing some material for an upcoming event, I thought I could as well share this little piece here on the blog. Basically it is about the ideal software development team, the roles people play and the skills each team member needs to possess to be able to make meaningful contributions.

Customer

The customer is not that company or that guy who pays. The customer actually drives the project and is part of the team. It is important to engage the customer in a constant dialog or at the very least to give him a means to collaborate and respond to questions easily.

Programmer

Programmers write the code and are supported by Testing Programmers. The programmers are the constraint on the team. Their number needs to grow in order to be able to handle a larger project. I share the common opinion that when you add two programmers, you should add one testing programmer as well. It seems to be a good idea to add programmers in pairs, because that allows them to pair on difficult tasks.

Skills required: Expert knowledge in the choosen programming language, tools and other technologies used for the project.

Testing Programmer

The Testing Programmer is just an ordinary programmer, but in that role he looks out of test coverage, performs manual and automated tests of the integrated software system and checks for completenes of the implemented solution based on user stories and in cooperation with the Business Analyst acting as Product Owner.

Skills required: Expert knowledge in the choosen programming language, tools and other technologies used for the project plus strong testing skills and expert knowledge in test automation.

Information Architect

Almost every system has some kind of user interface. The Information Architect designs the user interface, creates wireframe models for communication purposes, works with the Business Analytics and the Customer and helps the programmers when they implemented the user interface. He creates graphical elements for the user interface.

Skills required: feeling for good user interface design, graphic design, communication abilities

Business Analyst

The Business Analytics acts as Scrum Product Owner for the team and maintains a constant dialog with the customer. His job is to understand the customer's goal and expectations for the project. He collaborates with the customer to create user stories.

Skills required: Strong analytical skills and ability to gain expert knowledge in the customer's business in short time. Very good communication skills oral and in writing. Expert in writing user stories.