Process

Methodology / design pattern / development driven apa yang paling bagus?

Saat sedang melihat-lihat group programming di facebook, saya pernah melihat beberapa job opportunity dengan slogan seperti berikut: “Perusahaan kami menerapkan agile dan scrum!”. Banyak juga bahasan mengenai “agile” lebih baik dari waterfall, adalah metodologi terbaik. Tidak sedikit pula junior programmer atau analyst atau project manager baru di kantor-kantor yang bersikukuh bahwa team harus mulai menerapkan agile dan scrum. Hal yang sama juga berlaku pada TDD (Test Driven Development). Benarkah scrum + TDD adalah metodologi yang terbaik?

Sebelum saya melanjutkan dengan pembahasan yang lebih detail, akan menekankan hal yang menurut saya pribadi paling penting dalam pengembangan applikasi / system:

Pergunakan tools / cara apapun yang dirasa terbaik untuk menghasilkan applikasi yang bekerja dengan baik, dan mudah diubah. “Make it works, and changeable!”

Individuals and interactions over processes and tools

Salah satu manifesto yang cukup penting dan sering dilupakan dalam agile adalah “individuals and interactions over processes and tools“, atau bisa disingkat sebagai “people over process“. Sebagai programmer / developer / analyst, terkecuali kamu adalah ConcernedApe yang men-develop indie game “Stardew Valley”seorang diri dulunya, kamu akan bekerja dalam team, berinteraksi dan membuat keputusan-keputusan bersama.

Preferensi setiap orang berbeda-beda, dan metolodogi tertentu bisa bekerja di satu kelompok orang, bisa juga tidak berfungsi di kelompok orang lainnya. Bagaimana kamu dan team developer bisa bekerja dengan baik adalah yang terpenting, proses / metodologi adalah “alat bantu” yang bisa digunakan untuk mencapainya. “Scrum user story“, “Kanban board” tidak lebih dari hanya alat bantu untuk bekerja dalam team.

Bila dalam beberapa situasi tools tersebut tidak dapat digunakan dalam team, misalnya mayoritas tidak mengerti cara pakainya, atau tidak merasa ada manfaatnya, atau misalnya projectnya cukup kecil sehingga tidak diperlukan, maka carilah alternatif tools yang lebih dapat bermanfaat bagi team. Misalnya post-it notes di cubicle kerja masing-masing, atau minutes of meeting yang di-share dalam email, atau bug tracker.

Pernah saya berdiskusi dengan seorang project manager yang cukup berpengalaman, mengapa beliau tidak menerapkan daily standup meeting, yang umumnya adalah senjata pemungkas di scrum. Beliau menjelaskan bahwa teamnya bekerja secara remote di berbagai daerah yang berbeda, sehingga daily standup meeting tidak dapat dilakukan. Selain itu, tidak selalu ada hal yang dapat di-share dalam daily meeting, sehingga beliau menerapkan rule weekly report, dan contact langsung apabila ada (laptop, mouse rusak misal atau approval) yang diperlukan. Itu adalah contoh “people over process“.

Management juga tidak akan serta merta mengubah metodologi yang sedang berjalan secara tiba-tiba, untuk mengadopsi scrum secara mendadak. Perubahan itu terlalu beresiko, apabila team tidak terbiasa dan banyak yang tidak mengerti, maka perubahan hanya akan membawa musibah daripada manfaat. Terkecuali memang ada kebutuhan untuk meningkatkan metodologi development, jangan memaksa management untuk berubah demi ego sendiri atau “hanya karena scrum lebih baik”.

Metodologi sebaik apapun tidak akan berguna bila team tidak dapat menghasilkan applikasi yang berfungsi dengan baik

Scale it!

Salah satu hal yang menarik yang saya cermati adalah tidak banyak orang yang menyadari bahwa metodologi yang berbeda bisa diterapkan dalam skala yang berbeda pula, dalam satu organisasi. Misalnya project manager dan customer menggunakan iterative waterfall untuk memecah-mecah feature development ke dalam project-project, sebagai team bisa saja development dilakukan secara agile / scrum.

Atau hingga dalam skala personal sebagai programmer, kamu bisa saja memecah task list yang diberikan menjadi iterasi2 yang bisa menghasilkan feedback dalam 2 minggu (iterasi standar scrum), dan melaporkannya ke project manager / team lead secepatnya.

Don’t forget to make it changeable!

Saya berani bertaruh, tidak ada development plan / requirement yang tidak berubah saat development. Perubahan requirement adalah sangat wajar dan sangat mungkin terjadi. Di luar negosiasi finansial yang memang bukan ranah developer, mengembangkan applikasi agar bisa mudah diubah-ubah sesuai dengan perubahan requirement adalah penting. Applikasi yang mudah diubah juga penting agar dapat menambahkan fitur dengan mudah di kemudian hari. Hal ini juga tersirat dalam agile manifesto, “Responding to change over following a plan“.

Lalu bagaimana dapat mengembangkan applikasi yang mudah dikembangkan / diubah? Salah satunya adalah dengan membuat “low coupling, high cohesion” modul (class / function). Dan TDD dapat membantu mengembangkan kode yang low coupling tersebut. Namun dengan efek samping development time yang meningkat hingga 2x dari biasa dan kompleksitas dalam unit test, tidak semua team dapat menggunakan TDD (namun pastikan menggunakan TDD bila memungkinkan).

Ada 2 pemahaman lain yang juga dapat membantu mengembangkan kode “low coupling high cohesion”, yaitu Single Responsibility Principle, dan Dependency Injection. Keduanya adalah bagian dari SOLID principle, yang menurut saya lebih bermanfaat dari ke-3 pemahaman lain dari SOLID principle.

Conclusion

Pergunakan development method yang paling cocok dan bermanfaat untuk team. Kembangkan applikasi yang berfungsi, bisa digunakan dan mudah untuk diubah. Kembangkan kode yang “Low coupling High cohesion” agar applikasi dapat diubah dengan lebih mudah.

Advertisements

Is Artificial Intelligence can become dangerous?

This article is purely my opinion about dangers present in Artificial Intelligence, based on arguments between two amazing person in computer fields, Mark Zuckerberg and Elon Musk about it. In short, Elon says that there is danger present in non-regularized A.I. development, while Mark says not (one example article).

I don’t want to discuss or guessing who is right or wrong, and considering that application and fields in Artifical Intelligence is very wide, we cannot decide anything based on several fields. Furthermore my understanding about A.I. is far more limited than theirs are, so saying that one is right and another isn’t, is nonsense. But let’s say that I agree with Elon that in some cases, AI can be devastating, and it is because our lack of control and security concern.

One of A.I. design is to mimic us

The above video is MarI/O, machine learning (a part of Artificial Intelligence) for classic game Super Mario World in SNES (oh, how nostalgic). It shows how it learns to beat a stage using experience gained from losing each time. It is seems like a very simple application of AI, though at current technology, it’s kind of amazing.

Now what if we can somehow develop the same system with purpose of bypassing captcha? Kind of similar with virus vs antivirus, it is a game of cat and mouse, captcha vs bot is a game of cat and mouse. Captcha must evolve to keep able to detect bots, and bots need to evolve to keep able to fool Captcha. It’s a constant battle.

Assuming that bot can be evolved using supercomputer and machine learning and it’s so advanced so it can mimic us, humans flawlessly, then Captcha is fighting a losing battle. At worst, captcha will even prevent human from access.

Now what will happen if that resource is focused towards development of hacking tools, powered by supercomputer? That hacking tools are being armed with skills from clever hackers and improved overtime from experience and trial and error. One of the existing application of programmed hacking, by brute-force attack, is the example. Fortunately, brute-force attack is being greatly mitigated with account locking and captcha.

Now that when the tools are ready, it then mounted on hundreds of supercomputers to exploit the vulnerabilities existing in internet. How much outrage will happen at that time? Many sites will be hacked at same time, many other will be down. Considering our credit card and banking information exists there, it is also at risk to be blown over. Combine with Ransomware like WannaCry, it’ll make matters worse. What’ll happen if GitHub fall over because of that?

Lack of Control

Soon, we will have autonomous driving car in the streets. They’ll become common. Now what will happen if for one time there occurs a system failure? Maybe due to short circuit, deteriorating hardware or even cosmic ray. Usually it’s being handled by changing the control to manual mode, but how if at the same time, due to how advanced the A.I. has become, that the A.I. decided to not give away control? Or more realistically, if the passenger / driver is too unaware or distracted to act on specific time. It’ll be bad.

In a current, simpler case will be when a gas pedal is stuck at automatic car. In manual car, pressing the clutch will definitely cancel the acceleration. Put the gear to natural for further safety, then no danger will present. However in automatic car, it needs the gear to change to neutral, still with a chance of error. It’s not much, but the more we lose control over something, the more dangerous it will be.

Even in today’s lifestyle, companies will try to use A.I. to sell you more goods. Sometimes you don’t realize, but for they who don’t notice it, they can be powerless in front of those companies, only to burn more bucks for them.

Smart encryption

So let’s say that terrorist already develop an evolving-tightly-encrypted, anti-spy messaging service via internet. They can use that platform to communicate each other, freely, without able to be tracked by government or officials. Worse if they actually can hack into officials (police or army) communication line, they can get the security hole to perform some action.

No robots?

AI robots situation like Terminator or I, Robot maybe possible in distant future, but not in near future. Limitation of energy supply and processing power is one of the big cause they cannot be realized soon. Furthermore the lacking of robot-shaped humans hinders them in place / specific area. As xkcd has explained, it’s very unlikely for it to happen.

Conclusion

The way A.I. can be dangerous is not in a physical form that is popular in movies, like robot apocalypse. It is more related to our daily life, our interaction with computers and security concern.

Agile manifesto

Agile is dead! Really?

Dave Thomas, one among the founders of agile manifesto says so in his post blog, “Agile is Dead (Long Live Agility)“. And lately I have seen a blog post “Agile is Dead, Long Live Continuous Delivery“. Is agile really die and need to be changed? Is Continuous Delivery a replacement of agile?

Background

In his blog post, Dave Thomas indeed say that the word “agile” need to be killed down. But why? That’s because the word “agile” and the “agile” concept has been misguided and become a tool, a process and being industrialized. As Dave said in his blog post:

Once the Manifesto became popular, the word agile became a magnet for anyone with points to espouse, hours to bill, or products to sell. It became a marketing term, coopted to improve sales in the same way that words such as eco and natural are. A word that is abused in this way becomes useless—it stops having meaning as it transitions into a brand.

Get it? The current “agile” is swayed from it’s original meaning and objective. It has become “marketing term”.

Agile is guidance, not methodology

Let’s see what we have at agile manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The first point itself state that agile is not a methodology. The first point of manifesto states that you need to prefer to individuals and interactions over process and tools. If the agile you know focus to methodology and process, it already violates the first manifesto. Agile is just a guidance, to help you (and the team / company) to find the best process and tools for your development activity.

Even scrum, one of the most popular agile adaptation is still only a guidance or development framework. As stated,

“Langkah pertama yang perlu dilakukan untuk dapat memahami apa itu Scrum adalah dengan tidak mengaitkannya dengan sebuah metodologi apa pun juga karena Scrum hanya sebuah kerangka kerja dalam mengembangkan produk kompleks seperti software.” – Partogi, Joshua (2015) Manajemen Modern dengan Scrum, Yogyakarta, penerbit ANDI.

As translated: the first step that need to be done to able to understand what is Scrum is to not connect it with any methodology because Scrum is only a framework to develop complex product like software.

The marketing term agile / scrum

Today’s agile / scrum meaning varies and very misleading. The most popular misleading scrum is that it has “morning meeting” and “no documentation”. “If we have morning meeting, we already do agile”. It’s very misleading. The sole focus to process is already swayed away from agile manifesto.

Now let’s say that in your company you have one PM that manage 3 teams. Each team is doing project in different area. How can you do morning meeting with those setup? Evening e-mail daily reporting is enough for project tracking with those setup.

Continuous delivery is the next agile!

First, continuous delivery is not directly contradict with agile. Agile does not state that you need to develop a non-ready program. Moreover, agile state that you need to “responding to change over following a plan”, in which more or less is aligned with continuous delivery. And as stated above, agile is a guidance, not a methodology. We can say that continuous delivery is a more advanced implementation for agile.

Then it is the next agile! Hold on. As stated in agile manifesto, individuals and interactions over processes and tools, you cannot directly implement continuous delivery (CD) and hoping for everything to work well. You still need to consider the capability of team, servers, workflows, division, business decision. And the most important, the kind of software you make. CD relies heavily on good QA, test unit and continuous integration infrastructure. If you don’t have those matured, it’s risky to implement it.

Conclusion

Agile is not dead. It’s just that the term “agile” itself has swayed from it’s original meaning. It is a guidance and not methodology. It’s implementation may be evolved and improved. Continuous delivery is a good methodology and can align well with agile.

I’ve found a bug! Who to blame?

Despite that blame culture serve no benefit to the company, many companies still have this culture. Moreover, it is stupider to put a blame for bugs happened in software. Simply said, that’s because bugs happen, and the possibility of a bug is close to infinite.

Many times, many parties responsible for bugs in production

Let’s say that we use the following SDLC: requirement, analysis, design, develop, test, implement, maintenance. Now let’s see which responsibility can be held in each phases for each parties.

Requirement

The requirement given by user is ambiguous, has multiple meaning or the worst, he/she is wrong when giving the requirement. Next, the meeting PIC (person in charge) is doing mistake during passing the requirement through the analyst and programmer.

Analysis

The analyst doing a mistake during interpreting the user requirement. The analyst is mistaken during communicating the analysis and requirement through the programmer.

Development

Many-many things can happen during development. Wrong logic condition, wrong parameter assignment, etc. In short, the code is wrong.

Testing

The test case is very lacking compared to the complexity of the project and environment deployed. The case is testing wrong things. The test case resulting in false positive.

Implementation

Incorrect publish parameter. Incorrect deployment tools configuration. Production environment not tested beforehand. Production environment (OS, etc) is different with testing environment, and the code has environment-dependencies. Worse if the publishing done manually.

Maintenance

Internal support team inserting data with format that is not tested (whitespace characters). Some script being executed without tested. No change management for software correction, which causing the developers can directly push file to server. Some maintenance activity (backup, index rebuilding) causing anomaly and incorrect results. User trying to break the system.User input data with unsupported format.

Other causes

Requirement change mid-development. Some requirement break / not compatible with other requirement. Development time being shortened to meet earlier shipping date. Natural disaster causing loss in development time, but no schedule adjustment.

Bugs caused by many factors

Despite code / logic error, many other things can cause bug to an application. For example:

  • compiler / interpreter error – very rare cases though
  • flawed server configuration / installation
  • OS-level bug
  • Bug caused by different configuration in different environment
  • Error caused by cosmic rays
  • Y2k error – year 2000 problem
  • Network-related error
  • Incorrect data / format
  • Not-printable characters input

And there are many-many more.

Conclusion – now, who I can blame for that?

It’s best to not blame anyone for bugs / error happened in a software. Entire team / company should take responsibility for the error (even user as well). What you need to do is, to find the root cause, fix the software and find a way so the same mistake / error won’t happen in the future.

Custom apps IT Solution vs Product Developer

Last night I was being interviewed at one IT solution. The interview goes normal, with questions technically and personally. However I feel that the topic of question is different from the last time I had interview (around 2 weeks ago). Long story short, this interview asks more about technical skills and ability of using tools, while at the last interview the topic is leaning more to design, architecture, security and performance.

Custom IT solution is doing the hard work

I’m sorry to say and if this phrase is very harsh to you, but if you are working at custom IT solution, it means that you are being expected to do hard works. The client does not expecting you a good quality product. They expect you to work under their order and develop what they want. You are being valued by how hard you work, not by the product you produced (well the product also being valued, but not as much as your hard work).

I may sound narrow minded, but that is what happen in majority of custom IT Solution companies. Maybe not all (if any), but at least most of them do like that, at least at my country. And no, I don’t say that the practice is bad, or delivered bad-quality product is a sin for those company. It is just a nature of the company, enhanced by clueless leader about IT world. Moreover, the business nature of custom IT solution tend to bring the company into that path.

Faster delivery = more revenue

It is true for all IT solution that faster delivery = more revenue, because faster delivery means more project to handle and it means more income. However the way they do to improve delivery time is wrong. Usually they cut testing time and encourage overtime (worse, unpaid). It resulting in fast, low quality product. Fast, cheap, good, pick two. Usually they pick the fast and cheap one.

Less self improvement

Self improvement (better design, better framework, better developed component, code review, test units, etc) is third priority inside those company. Product delivery takes number one, while customer support takes second. It is because self improvement is seen as expense (manpower and time) and has invisible/indirect benefit. Training and conference does not fall into self improvement though, because it is tightly required to development. Try to learn kaizen.

Even if the company apply self-improvement, it will be driven and handled from top levels, rendering your involvement useless.

Bad documentation

Because you can support and train the customer directly in such big time span (morning to night), it makes documentation less required. Moreover writing a good documentation, and update it every changes will require big effort.

No idea will bloom

Any sprouting idea won’t bloom. Because everything is made by customer request, the company does not need good, fresh idea. They just need a good worker and constructor. Moreover, idea will be seen as hindrance, costly and too much effort to apply.

What about product developer?

If the company developed a product and live by it, the mind set will be different. I don’t say all product developer like that, but at least the successful one does.

Higher quality = more revenue & less cost

Am I drunk that I write something like that? No, I’m not drunk. In product developer company, higher quality product will increase customer satisfaction, resulting in good testimony and recommendation, then resulting in increased sales, means more revenue.

But isn’t higher quality means longer development time and higher cost? Why it can become less cost? It is indeed higher in short time period. However because you will maintain the product for long time (years, at least), fewer bugs and good administration system will help you cutting the cost. The longer the product being maintained, the higher the cost is saved.

Documentation!

A good product must have a good manual, and documented. The better the documentation is, the lesser effort needed to support the customer. It may not apply in all case, but at least some are. The more customer using this product, the more documentation is needed.

Idea!

Idea is everything for the lifetime of the product. A stagnant product lack of new features will become obsolete, and lose competition with new products. The terms is coined as product lifecycle. A good idea from the company member has higher chance to be applied.

Conclusion

The business flow of custom IT solution and product developer is different. Their revenue source and cost is also different. How they are managing the development and support is following each business process. A custom IT solution focus more on development time, while product developer focus more into quality. Not every company in each type follow the same rules, There is also custom IT solution who focus on quality, and product developer who focus on development time.

Why using CMS/third party plugin?

As a software developer myself, I previously hate to use CMS or plugin (let’s call both with CMS) for anything related with my works. Even worse, I try to avoid any plugins-related such as bootstrap. Well, that is me before I’m being involved in professional world. I think that I can make anything, better than any existing tools out there. Presently, I that way of think is wrong, but not when it was at the time before.

It’s Time

In the past

One main reason the inconsistent whether it is right or wrong is one: time. In my past, I have a lot of free time and not involved in any money-making task/job. It is very suitable for the past me to explore the website world, making the prototype myself and trying to compete with existing one. I even developed C# winform foundation myself (the foundation consist of login management, user controls and many utilities such as printing).

It is good, and right, at least when I was in the past. It is my phase in life to learn and train, crafting my skills better. If right now you are thinking about lyric from Linkin Park, “In the end, it doesn’t even matter”, I will say no, it is matter. It does not matter whether what you are trying to build will be used or not. It is matter however, about knowing how to build it and how complex the existing tools is. You will be able to differentiate between good and bad tools, will be fluent and better at using it.

Presently

Now I have been involved in professional work. The time is very limited, so does any works that I can produce within those limited time. I can choose between a vast types of products with shallow quality or very specific type of products with deep quality. I chose the later, resulting in me very specialized at enterprise back-end system, architecturally. Then, if in case I want to produce some general product like company profile, I will use CMS.

Standard and supported

CMS is used by many people worldwide. It has standard interface that everybody knows. Moreover it is supported, meaning that some important feature requested by user will be implemented later, and the bug will be fixed. It is also support external source integration to some point, such as SEO and API. It will be better if you use their paid version or service, meaning you can get support directly from the provider. You can consider this practice as “outsourcing”.

Documented and discussed

A good CMS is used by many people. Many people using the CMS will create discussion. That discussion, when too frequently happen, will one day become FAQ or even included in the documentation. A good documented product will be used by more people, resulting in the cyclic flow of improvement. It is however, different with self-developed system. A good documented and discussed component will make new developer faster to catch up with development.

But it’s limited

Limitation caused by CMS / plugin usually become problem to you. However currently, any good third-party component will provide some way to extend and customize the current behavior. It can be achieved with DI, overriding (CSS and javascript for example), to the extreme way of customizing the source code. The last is not recommended, because after you do so, you cannot expect any update from the provider, rendering the additional feature and bugfix useless.

Every tools out there have limitation. There is one tool that good at one side, meanwhile other good at other side. My knowledge (or yours) received from trying to develop third party tools and people review will enable you to choose which one is suitable for your current needs.

Conclusion

CMS is good if it fulfill your requirements and align with it. If you want some other things that is vastly different with the CMS provided, you need to either look for another or develop it yourself. Customizing CMS too far out of original purporse (ex: using wordpress for stock exchange) or even modifying the source code is not good, since it will prevent you to get further support.