DeveloperLife

Tuesday, May 8, 2012

Developer's Fantasies

The moment I became a proud father I almost lost all of my free time. Since then I became a proud father two times more. To make a long story short – I have three kids. This basically means no free time at all. Every now and then when I have five minutes to think clearly I fantasize about things I would have done if only I got the time. This includes learning how to play the guitar, teach high school kids math, open my own pizzeria and much more. Even when I do find free time I do not spend it on my fantasies because there is simply no point perusing any of them if I have no plan to stick with it.

As a software developer we also have fantasies (job related - other fantasies are out of scope). Each of us has his own ideas about a great change which will defiantly bring improvement. Those ideas are not realized because they have no economical direct incentive. Such ideas range from developing a smart text box which will use natural processing language and understands user requests to replacing our existing old annoying build technology with a new cool one. However, no one asked for a smart text box and the old annoying build system works just fine.

What happens to these fantasies in the agile era? Agile development teaches us to pursue costumer value. Tasks are wrapped as user stories to ensure there is no waste. Said stories are prioritized by value so at all time we work on the most important and valuable story. Technical debt and technical user stories are dealt with extra care. We only work on the most painful ones. We have been taught to adopt a new technology only if it helps us solve a problem, not entertain us or keep us up to date.

The problem is that besides the fact that it killed some of the fun in development it also keeps our solutions close to where we found them.

Several algorithms in computer science such as the Genetic Algorithm and the Simulated Annealing Algorithm use randomization in order to escape minimum locality. In a nutshell these algorithms are based on the following methodology: They start with an initial solution to the problem. Then iteratively use 'smart' assumptions to improve the solution by making small modifications to it. The more iterations they undergo the better the solution they will find. The problem with this approach is that the initial solution might be surrounded with bad solution (minimum locality). In such scenario the algorithm will not perform well and eventually return something close to the initial solution. How do we escape such minimum locality? One suggestion is once in a while to change your solution randomly without using your brain and smart assumptions. The idea is that sometimes such a bounce will upgrade your solution so well that it will compensate for other times where it did not help at all.

How can we escape minimum locality in our development process? Even in the agile era we should spend some time on our fantasies. Such random bounces in our iteration should be integrated in the agile methodology. If we will stick to it long enough we will eventually find better solutions for our problems. One thing we can promise. It will be much more fun.

Tuesday, April 10, 2012

Performance Parameters - What should I expose?

This post begins with a short poll. Did you ever let your end-user determine your application's thread pool size? Well if you didn't then you are a better man then us, because we did and we were wrong doing so. Thread pool size is just an example to a variety of performance parameters we tend to publish as configuration. We don't mean to but do so at some point of our application life cycle.

Like any good fairy tale at the beginning everyone are happy with our new application. Then slowly but steady complaints start coming stating it is too slow. To make it faster we use every trick we now. We start optimizing SQL's, parallel what we can, use threads and refactor the code to be more efficient. Everyone's happy? Not Yet! Now we get complaints that although the job is done fast, the application takes too many resources. "No problem", we say and immediately publish every performance parameter we have in the code as configuration. That includes stuff like batch size, poll interval, sleep time and naturally thread pool size.

This turns out to be a bad idea. It is a bad idea because the user has no way of knowing which values are adequate. Moreover the values probably depend on the hardware used, which as we know changes rapidly. It is a bad idea because it uncovers implementation details. But it is a bad idea mostly because it is not part of the user domain language.

Performance parameters should be handled just like any other feature in the software. One should think of the user story behind it and specify configuration accordingly. Some time it is only a terminology change. Instead of using "thread pool size" you use "number of client served simultaneously". Sometimes the mapping from the configuration parameter to an inner parameter will involve some computation and heuristics. So you might configure "Number of items processed per minute" which will be translated to "batch size", "poll interval" and so on....

So keep in mind. Performance parameters are not second-class citizens. It should be specified with the same abstraction layer as the rest of the software. It should hide any inner implementation details and if possible it should be hardware independent.

Thursday, March 22, 2012

6 things they didn't teach me at the academy - Prepare yourself better for a developer's life

Looking for your first position in the software world? Want to know more about developer's life? Improve your chances of getting hired and blend in better? Most cases you just finished your undergrad studies. The years you spent in the academy are valuable and helpful in many ways, but don’t necessarily prepare you for a developer's life. Now, this post is not about whether undergrad studies are a necessary and sufficient condition for being a good programmer, nor about which university prepares you better, but it is about pointing out what might be overlooked during your study, what can be improved, and how one can have a head start when diving into this exciting world. We present you with 6 essential points that will help you in your upcoming venture:

Understanding the structure of a software company. Although this can vary form company to company, in most cases there will be a distinction between Developers, QA's/Testers, Support, Product and IT. Understanding this structure beforehand will help a candidate understand the job description he is interviewing for. A new employee will blend in better, adopt business processes faster and utilize all the companies' capabilities.
Development Methodologies. Whether it's Waterfall or Scrum, Agile or not, understanding the different approaches in Development Methodologies can give you a since of how your life will look like as a programmer. It could even help you choose a better suited company.
Code Management tools and processes.

You can finish your degree with knowing slim to none about code repository tools such as SVN, GIT, etc, which are used extensively.
Continuous Build as a quality control process.
Integration Test as a phase in the development process.
Issue tracking systems such as Trac and Jira used to handle development process as a whole.

Coding guidelines. No need to be alarmed if you haven't coded much during your study. However - they say it takes 100 repeats to learn how to bench-press correctly, but if you learn it the wrong way then it takes 10000 to correct yourself. Teach yourself how to write good code from the very start. A great book to get you started is Code Complete, Second Edition.
IDE usage.

IDE's are great - Don't you think? If you haven't mastered one during your studies - that's a shame. They really make life easier and supply easy access to new technologies.
However, on the downside - if you adopted one too fast - you might be baffled when someone will ask you to run something from the command line or mention the word classpath. Be aware that there is a world outside your IDE.

Utilizing the power of open source. Let's say you have a great idea for a new project. Most chances are that a large portion of your project's infrastructure is already available as open source. Moreover, choosing the right technology can save you an enormous amount of time. A new developer must understand that open source is a powerful tool that can be leveraged- there is no need to invent the wheel every time.

As a final note, and from our point of view, the best tip that we can give a starting developer is joining an open source code initiative. This can certainly introduce him to most of the concepts mentioned above, get him a feel of the software world, and make the difference between him and other job seeking developers.

Monday, February 27, 2012

Junior Developers - Anthropological Review

Junior Developers...You can't live with them and it is illegal to shoot them. Well, except maybe for our North Korean readers. So how the hell can you pass software development knowledge to someone who thinks that Maven is a city in Eastern Europe?

In his book 'Software Development as a Cooperative Game', Alistair Cockburn describes the three levels of learning in Aikido. They are Shu, Ha and Ri which roughly translate into learn, detach and transcend.

A worrier in the Shu level learns techniques from his master and copies them. This level is also called ‘Follow the Master’.
In the Ha level the worrier learns the strengths and limitations of a technique. He learns other techniques and can now choose which one to use according to his position.
In the last level Ri the worrier creates a blend of techniques he knows, while also inventing new ones.

These three levels of learning exist in any craft, software development included. The Junior is naturally at Shu level in most aspects of software development. Consider Object Oriented Design for example. A Junior will adequately follow the rules he just learned at the CRC Object Oriented Design course. He is the one saying "We should use smaller cards. In the course they told us to use 6 inch cards...”

Let’s consider the communication between two people in different levels. If we deduce it to our world then it is between a Senior and a 'Shu' level worrier/developer. A Junior will always prefer to receive a set of instructions/rules for every task he needs to accomplish.

A Junior joining a software development team has a lot of catching up to do. He doesn't know the product. He doesn't know the business domain. He might not know all the tools and technologies you are using. He definitely has no familiarity with software development idioms such as Dependency Injections, Continues Build, etc. Make sure his assigned tasks are with no more then one unknown domain. Moreover, make sure he understands that one of the task goals is to better understand that unknown domain. Taking a different approach will lead the Junior to the mood of "the world is too complicated to understand", rushing to you with problem descriptions like "That red thing is not talking to the green part of my screen anymore. I tried to restart it but it doesn't help". Remember that a software developer must be inspired to understand what is happening under the hood. Throwing to much complexity at once might generate the exact opposite.

Embracing new technologies is a good practice – depends when and where. Google-Guice home page describes how using the ‘new’ operator is old-school. In the Git home page they tell you that using a client-server source control is old news - Distributed Source Control is Da - Bomb. When visiting the Scala home page they will convince you that using critical section and ‘synchronize’ is ancient history - STM (software transactional memory) is buzzing. The Junior will believe them all. One day he will call you all excited for a simple dialog application demo. It will be on a Git branch, while all objects will be created using Guice, and instances will only communicate using an inner event bus that uses weird syntax of embedded Scala code... you get the idea. So please don't let a Junior embrace new technologies without your review.

The Junior only recently graduated from the University. He already gain experience in problem solving. In the University solutions are always elegant yet complicated where in the real world most chances the best solution is naive yet simple. Once more he will come to you showing off his latest coding. He was instructed to read some configuration from a database and save it in-memory. You are desperately trying to keep up with his explanation on why he had to use a red-black tree (something about reducing the complexity from O(nlogn) to O(lognlogn)) while also trying to overcome the sharp pain you suddenly feel in the chest. So stop mumbling "...but there were only 20 rows in that table..." and remember that you must change his approach to the well known "keep it simple".

Stating the above, we cant ignore the benefits of Juniors. Their ability to adopt new technologies, highly motivated and hard working (usually they don't have 3 mouths to feed...)

Wednesday, February 22, 2012

Extending the Task Service in Activiti Engine

As we described in our latest post, we have been trying out Activiti lately. A challenge we came across was to create our own Human Task Service of some sort, as we had used our own Identity Management. In a typical BPM system the Process Engine and Human Task Service are decoupled from one another. Moreover, a standard called WS-HumanTask Service exists for establishing a protocol between the last two. However, in Activiti's case the Human Task Service is embedded inside the engine.

So we took a different approach as we wanted to use our own human task server. We thought that if we will be notified about changes occurring inside Activiti's engine concerning user tasks (e.g. create task event), we could propagate those notifications to our Human Task Service. Unfortunately, we found out that Activiti's engine does not have a notification mechanism as we hoped (at least in 5.8 - there are talking about an event bus in future releases), so we had to use another mechanism to achieve this task.

Apparently Activiti does have a notification mechanism while parsing a BPMN process definition XML. It allows one to plug in a listener which will be notified in several events. One of those events is when a user task is being parsed. This was exactly what we needed. We plugged our own listener implementation and every time a user task was parsed we added a TaskListenr to the user task in the process definition. This listener was invoked by the engine when this task was created. Obviously the listener impl is to simply send a notification as "Task Created" to our own Human Task Service. Note: you will also have to manage the task life-cycle and update Activiti's engine using its services.

Now for the technical details. As in our latest post, this requires:

adding a Java class implementing BpmnParseListener. This is the interface you want to implement to plug in your own listener.
injecting the Java class using the Spring xml config file.

BpmnParseListener Class
We implement a TaskListener for each of the events we are interested in:

CompleteTaskListenerImpl
AssigneTaskListenerImpl
CreateTaskListenerImpl

We Override the parseUserTask method so that in each time a parse event occurs our TaskListeners will add some logic on our WS-HumanTask Service/Task Service. The rest of the parse events will be ignored.

package my.great.company.com.businessprocessengine.listeners;
import java.util.ArrayList;
  
  
public class MyBpmnParseListener implements BpmnParseListener
{   
    
    private final class CompleteTaskListenerImpl implements TaskListener {
        @Override
        public void notify(DelegateTask delegateTask) {  
                //TODO: this is where your logic should be added
                logger.info("TASK COMPLETE from Activiti " + delegateTask.toString());
        }
    }
  
    private final class AssigneTaskListenerImpl implements TaskListener {
        @Override
        public void notify(DelegateTask delegateTask) {
                //TODO: this is where your logic should be added 
                logger.info("TASK ASSIGN from Activiti " + delegateTask.toString());
        }
    }
  
    private final class CreateTaskListenerImpl implements TaskListener {
        @Override
        public void notify(DelegateTask delegateTask) {
                //TODO: this is where your logic should be added 
               logger.info("TASK CREATE from Activiti " + taskEntity.toString());
        }
    }
    
    private TaskListener            createTaskListener         = new CreateTaskListenerImpl();
    private TaskListener            assigneTaskListener        = new AssigneTaskListenerImpl();
    private TaskListener            completeTaskListener       = new CompleteTaskListenerImpl(); 
    private static final Logger     logger                     = Logger.getLogger(MyBpmnParseListener.class);
    
  
    public MyBpmnParseListener(){}
    
    @Override
    public void parseUserTask(Element userTaskElement, ScopeImpl scope, ActivityImpl activity)
    {
        ActivityBehavior activitybehaviour = activity.getActivityBehavior();
        if (activitybehaviour instanceof UserTaskActivityBehavior)
        {
            UserTaskActivityBehavior userTaskActivity = (UserTaskActivityBehavior) activitybehaviour;
            if (createTaskListener != null)
            {
                userTaskActivity.getTaskDefinition().addTaskListener(TaskListener.EVENTNAME_CREATE, createTaskListener);
            }
            if (assigneTaskListener != null)
            {
                userTaskActivity.getTaskDefinition().addTaskListener(TaskListener.EVENTNAME_ASSIGNMENT,assigneTaskListener);
            }
            if (completeTaskListener != null)
            {
                userTaskActivity.sgetTaskDefinition().addTaskListener(TaskListener.EVENTNAME_COMPLETE,completeTaskListener);
            }
        }
    }
    
    @Override
    public void parseRootElement(Element arg0, List<ProcessDefinitionEntity> arg1)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseProcess(Element processElement, ProcessDefinitionEntity processDefinition)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseStartEvent(Element startEventElement, ScopeImpl scope, ActivityImpl startEventActivity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseExclusiveGateway(Element exclusiveGwElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseParallelGateway(Element parallelGwElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseScriptTask(Element scriptTaskElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseServiceTask(Element serviceTaskElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseTask(Element taskElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseManualTask(Element manualTaskElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseEndEvent(Element endEventElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseBoundaryTimerEventDefinition(Element timerEventDefinition, boolean interrupting,
            ActivityImpl timerActivity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseSubProcess(Element subProcessElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseCallActivity(Element callActivityElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseProperty(Element propertyElement, VariableDeclaration variableDeclaration, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseSequenceFlow(Element sequenceFlowElement, ScopeImpl scopeElement, TransitionImpl transition)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseSendTask(Element sendTaskElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseBusinessRuleTask(Element businessRuleTaskElement, ScopeImpl scope, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseBoundaryErrorEventDefinition(Element errorEventDefinition, boolean interrupting,
            ActivityImpl activity, ActivityImpl nestedErrorEventActivity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseIntermediateTimerEventDefinition(Element timerEventDefinition, ActivityImpl timerActivity)
    {
        // Nothing to do here
    }
    
    @Override
    public void parseMultiInstanceLoopCharacteristics(Element activityElement, 
            Element multiInstanceLoopCharacteristicsElement, ActivityImpl activity)
    {
        // Nothing to do here
    }
  
    @Override
    public void parseInclusiveGateway(Element arg0, ScopeImpl arg1, ActivityImpl arg2) {
        // Nothing to do here
        
    }
}

Spring configuration

The Spring file change will be done in our context XML file:

... 
<bean id="processEngineConfiguration" class="org.activiti.spring.SpringProcessEngineConfiguration"> 
        ...
        <property name="preParseListeners">
            <list>
                <bean class="my.great.company.com.businessprocessengine.listeners.MyBpmnParseListener"></bean>
            </list>
        </property>
 ...
 </bean>...
 

That's it. You are good to go and handle tasks on your own. Good luck.