Process

Agile manifesto

Agile is dead! Really?

Dave Thomas, one among the founders of agile manifesto says so in his post blog, “Agile is Dead (Long Live Agility)“. And lately I have seen a blog post “Agile is Dead, Long Live Continuous Delivery“. Is agile really die and need to be changed? Is Continuous Delivery a replacement of agile?

Background

In his blog post, Dave Thomas indeed say that the word “agile” need to be killed down. But why? That’s because the word “agile” and the “agile” concept has been misguided and become a tool, a process and being industrialized. As Dave said in his blog post:

Once the Manifesto became popular, the word agile became a magnet for anyone with points to espouse, hours to bill, or products to sell. It became a marketing term, coopted to improve sales in the same way that words such as eco and natural are. A word that is abused in this way becomes useless—it stops having meaning as it transitions into a brand.

Get it? The current “agile” is swayed from it’s original meaning and objective. It has become “marketing term”.

Agile is guidance, not methodology

Let’s see what we have at agile manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The first point itself state that agile is not a methodology. The first point of manifesto states that you need to prefer to individuals and interactions over process and tools. If the agile you know focus to methodology and process, it already violates the first manifesto. Agile is just a guidance, to help you (and the team / company) to find the best process and tools for your development activity.

Even scrum, one of the most popular agile adaptation is still only a guidance or development framework. As stated,

“Langkah pertama yang perlu dilakukan untuk dapat memahami apa itu Scrum adalah dengan tidak mengaitkannya dengan sebuah metodologi apa pun juga karena Scrum hanya sebuah kerangka kerja dalam mengembangkan produk kompleks seperti software.” – Partogi, Joshua (2015) Manajemen Modern dengan Scrum, Yogyakarta, penerbit ANDI.

As translated: the first step that need to be done to able to understand what is Scrum is to not connect it with any methodology because Scrum is only a framework to develop complex product like software.

The marketing term agile / scrum

Today’s agile / scrum meaning varies and very misleading. The most popular misleading scrum is that it has “morning meeting” and “no documentation”. “If we have morning meeting, we already do agile”. It’s very misleading. The sole focus to process is already swayed away from agile manifesto.

Now let’s say that in your company you have one PM that manage 3 teams. Each team is doing project in different area. How can you do morning meeting with those setup? Evening e-mail daily reporting is enough for project tracking with those setup.

Continuous delivery is the next agile!

First, continuous delivery is not directly contradict with agile. Agile does not state that you need to develop a non-ready program. Moreover, agile state that you need to “responding to change over following a plan”, in which more or less is aligned with continuous delivery. And as stated above, agile is a guidance, not a methodology. We can say that continuous delivery is a more advanced implementation for agile.

Then it is the next agile! Hold on. As stated in agile manifesto, individuals and interactions over processes and tools, you cannot directly implement continuous delivery (CD) and hoping for everything to work well. You still need to consider the capability of team, servers, workflows, division, business decision. And the most important, the kind of software you make. CD relies heavily on good QA, test unit and continuous integration infrastructure. If you don’t have those matured, it’s risky to implement it.

Conclusion

Agile is not dead. It’s just that the term “agile” itself has swayed from it’s original meaning. It is a guidance and not methodology. It’s implementation may be evolved and improved. Continuous delivery is a good methodology and can align well with agile.

I’ve found a bug! Who to blame?

Despite that blame culture serve no benefit to the company, many companies still have this culture. Moreover, it is stupider to put a blame for bugs happened in software. Simply said, that’s because bugs happen, and the possibility of a bug is close to infinite.

Many times, many parties responsible for bugs in production

Let’s say that we use the following SDLC: requirement, analysis, design, develop, test, implement, maintenance. Now let’s see which responsibility can be held in each phases for each parties.

Requirement

The requirement given by user is ambiguous, has multiple meaning or the worst, he/she is wrong when giving the requirement. Next, the meeting PIC (person in charge) is doing mistake during passing the requirement through the analyst and programmer.

Analysis

The analyst doing a mistake during interpreting the user requirement. The analyst is mistaken during communicating the analysis and requirement through the programmer.

Development

Many-many things can happen during development. Wrong logic condition, wrong parameter assignment, etc. In short, the code is wrong.

Testing

The test case is very lacking compared to the complexity of the project and environment deployed. The case is testing wrong things. The test case resulting in false positive.

Implementation

Incorrect publish parameter. Incorrect deployment tools configuration. Production environment not tested beforehand. Production environment (OS, etc) is different with testing environment, and the code has environment-dependencies. Worse if the publishing done manually.

Maintenance

Internal support team inserting data with format that is not tested (whitespace characters). Some script being executed without tested. No change management for software correction, which causing the developers can directly push file to server. Some maintenance activity (backup, index rebuilding) causing anomaly and incorrect results. User trying to break the system.User input data with unsupported format.

Other causes

Requirement change mid-development. Some requirement break / not compatible with other requirement. Development time being shortened to meet earlier shipping date. Natural disaster causing loss in development time, but no schedule adjustment.

Bugs caused by many factors

Despite code / logic error, many other things can cause bug to an application. For example:

  • compiler / interpreter error – very rare cases though
  • flawed server configuration / installation
  • OS-level bug
  • Bug caused by different configuration in different environment
  • Error caused by cosmic rays
  • Y2k error – year 2000 problem
  • Network-related error
  • Incorrect data / format
  • Not-printable characters input

And there are many-many more.

Conclusion – now, who I can blame for that?

It’s best to not blame anyone for bugs / error happened in a software. Entire team / company should take responsibility for the error (even user as well). What you need to do is, to find the root cause, fix the software and find a way so the same mistake / error won’t happen in the future.

Custom apps IT Solution vs Product Developer

Last night I was being interviewed at one IT solution. The interview goes normal, with questions technically and personally. However I feel that the topic of question is different from the last time I had interview (around 2 weeks ago). Long story short, this interview asks more about technical skills and ability of using tools, while at the last interview the topic is leaning more to design, architecture, security and performance.

Custom IT solution is doing the hard work

I’m sorry to say and if this phrase is very harsh to you, but if you are working at custom IT solution, it means that you are being expected to do hard works. The client does not expecting you a good quality product. They expect you to work under their order and develop what they want. You are being valued by how hard you work, not by the product you produced (well the product also being valued, but not as much as your hard work).

I may sound narrow minded, but that is what happen in majority of custom IT Solution companies. Maybe not all (if any), but at least most of them do like that, at least at my country. And no, I don’t say that the practice is bad, or delivered bad-quality product is a sin for those company. It is just a nature of the company, enhanced by clueless leader about IT world. Moreover, the business nature of custom IT solution tend to bring the company into that path.

Faster delivery = more revenue

It is true for all IT solution that faster delivery = more revenue, because faster delivery means more project to handle and it means more income. However the way they do to improve delivery time is wrong. Usually they cut testing time and encourage overtime (worse, unpaid). It resulting in fast, low quality product. Fast, cheap, good, pick two. Usually they pick the fast and cheap one.

Less self improvement

Self improvement (better design, better framework, better developed component, code review, test units, etc) is third priority inside those company. Product delivery takes number one, while customer support takes second. It is because self improvement is seen as expense (manpower and time) and has invisible/indirect benefit. Training and conference does not fall into self improvement though, because it is tightly required to development. Try to learn kaizen.

Even if the company apply self-improvement, it will be driven and handled from top levels, rendering your involvement useless.

Bad documentation

Because you can support and train the customer directly in such big time span (morning to night), it makes documentation less required. Moreover writing a good documentation, and update it every changes will require big effort.

No idea will bloom

Any sprouting idea won’t bloom. Because everything is made by customer request, the company does not need good, fresh idea. They just need a good worker and constructor. Moreover, idea will be seen as hindrance, costly and too much effort to apply.

What about product developer?

If the company developed a product and live by it, the mind set will be different. I don’t say all product developer like that, but at least the successful one does.

Higher quality = more revenue & less cost

Am I drunk that I write something like that? No, I’m not drunk. In product developer company, higher quality product will increase customer satisfaction, resulting in good testimony and recommendation, then resulting in increased sales, means more revenue.

But isn’t higher quality means longer development time and higher cost? Why it can become less cost? It is indeed higher in short time period. However because you will maintain the product for long time (years, at least), fewer bugs and good administration system will help you cutting the cost. The longer the product being maintained, the higher the cost is saved.

Documentation!

A good product must have a good manual, and documented. The better the documentation is, the lesser effort needed to support the customer. It may not apply in all case, but at least some are. The more customer using this product, the more documentation is needed.

Idea!

Idea is everything for the lifetime of the product. A stagnant product lack of new features will become obsolete, and lose competition with new products. The terms is coined as product lifecycle. A good idea from the company member has higher chance to be applied.

Conclusion

The business flow of custom IT solution and product developer is different. Their revenue source and cost is also different. How they are managing the development and support is following each business process. A custom IT solution focus more on development time, while product developer focus more into quality. Not every company in each type follow the same rules, There is also custom IT solution who focus on quality, and product developer who focus on development time.

Why using CMS/third party plugin?

As a software developer myself, I previously hate to use CMS or plugin (let’s call both with CMS) for anything related with my works. Even worse, I try to avoid any plugins-related such as bootstrap. Well, that is me before I’m being involved in professional world. I think that I can make anything, better than any existing tools out there. Presently, I that way of think is wrong, but not when it was at the time before.

It’s Time

In the past

One main reason the inconsistent whether it is right or wrong is one: time. In my past, I have a lot of free time and not involved in any money-making task/job. It is very suitable for the past me to explore the website world, making the prototype myself and trying to compete with existing one. I even developed C# winform foundation myself (the foundation consist of login management, user controls and many utilities such as printing).

It is good, and right, at least when I was in the past. It is my phase in life to learn and train, crafting my skills better. If right now you are thinking about lyric from Linkin Park, “In the end, it doesn’t even matter”, I will say no, it is matter. It does not matter whether what you are trying to build will be used or not. It is matter however, about knowing how to build it and how complex the existing tools is. You will be able to differentiate between good and bad tools, will be fluent and better at using it.

Presently

Now I have been involved in professional work. The time is very limited, so does any works that I can produce within those limited time. I can choose between a vast types of products with shallow quality or very specific type of products with deep quality. I chose the later, resulting in me very specialized at enterprise back-end system, architecturally. Then, if in case I want to produce some general product like company profile, I will use CMS.

Standard and supported

CMS is used by many people worldwide. It has standard interface that everybody knows. Moreover it is supported, meaning that some important feature requested by user will be implemented later, and the bug will be fixed. It is also support external source integration to some point, such as SEO and API. It will be better if you use their paid version or service, meaning you can get support directly from the provider. You can consider this practice as “outsourcing”.

Documented and discussed

A good CMS is used by many people. Many people using the CMS will create discussion. That discussion, when too frequently happen, will one day become FAQ or even included in the documentation. A good documented product will be used by more people, resulting in the cyclic flow of improvement. It is however, different with self-developed system. A good documented and discussed component will make new developer faster to catch up with development.

But it’s limited

Limitation caused by CMS / plugin usually become problem to you. However currently, any good third-party component will provide some way to extend and customize the current behavior. It can be achieved with DI, overriding (CSS and javascript for example), to the extreme way of customizing the source code. The last is not recommended, because after you do so, you cannot expect any update from the provider, rendering the additional feature and bugfix useless.

Every tools out there have limitation. There is one tool that good at one side, meanwhile other good at other side. My knowledge (or yours) received from trying to develop third party tools and people review will enable you to choose which one is suitable for your current needs.

Conclusion

CMS is good if it fulfill your requirements and align with it. If you want some other things that is vastly different with the CMS provided, you need to either look for another or develop it yourself. Customizing CMS too far out of original purporse (ex: using wordpress for stock exchange) or even modifying the source code is not good, since it will prevent you to get further support.

Big or Small Company

Last night I’ve met with my past co-worker at a big company. Before working here at a small company, I’ve worked at a big company for some time beforehand. We talked a lot. Based on their story, the work situation there currently is not far off from last time I was there, or maybe even worse. Same company size, worse workload. It makes me think, how different is it at the big company instead with one in a small company?

Decision maker

In small company, most of strategic decision lies at the hand of the owner. The success factor or future of company lies at the sole hand of the owner.

Meanwhile in big company, The strategic decision are separated by levels. More critical and strategic the decision, the higher the position that make the decision.

Position competency

In big company, a position have clear definition about competencies and required skills. In small company, strategic position usually filled with relations (family or friend), or subjective competency from the owner.

Reward and punishment

In big company, reward and punishment are defined clearly. In small company, it is changing from time to time and only based from oral norms.

Strategic plan duration

Strategic plan in small company usually can be executed faster and clear faster. Usually the strategic plan is planned shorter (days/ weeks/ months) before execution. Strategic plan in big company need more factors to reconsider, has bigger duration and cannot be easily executed. Strategic plan also planned further (years) in big company.

Law

Usually, small company are tend to violate the law more than big company. Especially for tax and employment law. It is basically due to the government can monitor big company more than small company, and big company can easily be exposed when violating law,

Performance

Big company usually has more stable performance. The same performance can be expected with the branches in same area. Small company’s performance are vary and unstable. It is usually based on company owner’s performance, or some skillful’s manpower. When the company lost those skillful manpower, usually the performance go down a lot.

Conclusion

Big company tend to be more rigid and stiff. They don’t change much, but they have clear plans, rewards and rules. In small company, the rules are more subjective based on owner’s view, but they are more agile and flexible.

Hardest : 3rd party system integration

This is the first part of explanation from Hardest : back end enterprise system development article. In this post we will dig down about how hard is it to use 3rd party system into our system.

3rd party system can be either using hardware with drivers, plug ins, or different programming tools such as Microsoft BizTalk and SharePoint. Using 3rd party system in your system sometimes can be easy, or hard. But usually, it is harder than the system development itself.

Different bridging support

This is one of the hardest point in third party system integration. Talking about hardware, usually it comes with built-in driver from the same vendor. However, not every language are supported by the driver. For example, the driver library may not happen to come in python or Lisp language. In worst case, you will need to execute command prompt for bridging with the hardware.

Talking about other programming tools/system, they either provide a supporting language library, or bridging protocol such as tcp/ip, http or command prompt. Different system and different vendor provide different library and bridging method.

Different process

Even in same product type (barcode scanner for example), they can have different syntax, and even different process structure. For example: one use event-driven while the other use threading / polling and the rest use procedural result. The different process structure make it harder to abstract the driver and integrate it with your system. Sometimes it forces your system to change following the process structure.

The same happen when talking about software integration. Most of them have different process and operations. It is hard to abstract them. Moreover, usually they are not supporting transaction processing, making it harder to rollback when error happens.

System error

In some extremely rare side cases, some error can happen in the driver / library or even the component itself. It will makes it harder for the support to identify the error, and the repair will be even harder. In worst case, you need to have the vendor repair the component to even fix the problem, and no workaround can be made.

System Change

Now we get the above problems. Now in some case, the vendor is no longer sustainable and we are forced to change vendor, or the item produced is discontinued and changed with a new one. The can be very costly and time-consuming as well. Change management need to be defined in order to keep business running during the transition, and we don’t have any choice but to do it.

Conclusion

3rd party system integration is harder due to some limitations and lack of control over the component itself. It makes harder to develop, abstract and worse at debugging.

Edit 2015/05/28: 3rd party hardware integration’s scope is too small and is similar with 3rd party system integration. Hence the modification.

Hardest : back-end enterprise system development

What is the hardest thing during developing back-end enterprise system? As a programmer, I have experienced full cycle of building a system in a company from start until being used. Based on my experience, when I try to order them into priorities, this is how I will do it. Please note that this list is different while developing framework/foundation (maybe i’ll list them later):

  1. Handle exception and negative case
  2. Server administration (uptime, backup, re-index, disaster management)
  3. Implementation / publishing / change management
  4. Business requirement gathering
  5. System Documentation
  6. 3rd party system integration
  7. Programming

Of course the priority can be changed based on what system to develop, how good is the company / enterprise with planning the system, etc. And this list can be biased only by my experience alone. But I think the adjustment cannot be shifted very different. If you are a programmer, do you agree with the list? Why is programming / developing the code is the easiest in developing back-end enterprise system? What is my priority based on?

Edit 2015/05/28: 3rd party hardware integration’s scope is too small and is similar with 3rd party system integration. Hence the modification.