Problem Decomposition

Introduction#

The process in this article is for a one-person company. Even though there is no communication between people, I still need to communicate with myself in the future. I also need to have daily tasks that I need to work on ready before I begin work.

Topological Sort#

Recently, I read a book called “Learn AI-Assisted Python Programming With GitHub Copilot and ChatGPT”. The diagrams in the book made me to think about Topological Sorting of tasks. In Topological Sorting, you start by identifying the leaf nodes that have no dependencies and work your way backwards through the graph.

The diagrams in the book showed the problem decomposition approach. The tasks were checked off from the end, working from right to left of the problem decomposition diagram. The rightmost nodes had no dependencies. If you draw the graph from top to bottom. You will start with the bottom and work your way to the top.

Tracking Tasks#

I was creating tasks both on paper and markdown files as part of the project documentation, but I struggled to make consistent progress towards my objectives, which included creating a custom image from a base image and provisioning a server for a customer. I found myself randomly jumping between tasks, designing experiments, creating prototypes and documenting the working steps. The tasks document grew very big and it was becoming difficult to review and refine the tasks.

Task Granularity#

After reading the book, I decided to try a new approach. The question is: when do you stop decomposition? Until recently, the answer was: until someone says, “I know how to do that.” However, with the AI tools, the answer has become: until the task is specific and clear with a single purpose. This clarity makes for an effective prompt to generate the code.

Task Sequence#

Here are the steps I followed using blank index cards:

1. Write down the objective on the first card.
2. Start with the first task. Write down the task on the front of the card.
3. Arrange tasks in an approximate sequence (you can easily rearrange the cards).
4. Find any subtasks for every task (some tasks may not have any subtasks).
5. Write down the subtasks on the back of the card.

I wrote my objective on the first card and then documented the tasks in the order I thought they would work. For tasks that can be further decomposed, I wrote the subtasks on the back of the card. This approach allowed me to consolidate all the tasks that were previously scribbled on paper and in the readme into one place.

The process to some extent resembles the Technical Spike in Agile software development. In a techncial spike the time is fixed. For me the time was not fixed. The task descriptions were at a high level, similar to the level of Packer and Terraform Templates.

Going reverse from the implementation to the tasks made it easier to document the tasks. Initially I used the top down approach to find the tasks and implemented them using Packer and Terraform. Once the unknowns were addressed, I went bottom up to document the tasks.

With this method, the tasks became more manageable, and tracking progress was easier. No more just staring at the computer screen and not knowing what to work on next.

Choosing Analog#

The small size of index cards is actually a good thing because it forces you to define concise and focused tasks. You won’t be able to cram too many details or sub-tasks onto a single card, which keeps the tasks at the right level of abstraction.

Staying analog with index cards also makes it easy to iterate on the sequence of tasks and add new tasks anywhere within the sequence. As you learn more about the domain, you can easily insert new cards or rearrange existing ones to reflect updates to your workflow. This flexibility allows you to adapt your plan as needed without losing sight of the big picture.

Design Experiments#

Although my previous approach of learning concepts, designing small experiments and creating prototypes may seem like a waste of time, it helped me find the unknowns and document all the tasks needed to complete the project. However, it lacked the right sequence, which is difficult to determine when there are many unknowns.

Build Prototypes#

It is not possible to foresee all the tasks needed ahead of time. The prototype I built using Packer and Terraform had declarative code in the templates, which made extracting tasks and subtasks easier than if the prototype code had been imperative. I also wanted to minimize the code I have to write and just use the existing tools to build the prototype quickly.

However, not every task is included in the Packer template. Some of the things that Packer does internally, such as creating a temporary key pair for remote SSH connection, need to be explicitly done in boto3 code. We must also define tasks such as whether to use the default VPC or not during custom image creation.

Prototype + MVP#

As a developer, I find it difficult to throw away code. I built the first version of HiveGrid that combines a prototype and a MVP. I did not realize it at that time. Packer and Terraform templates are the throw away parts. The throw away of the prototype part was motivated to some extent by the license changes by Hashicorp. The heavy lifting is done by the Ansible playbooks. The playbooks are task specific and reusuable. Ansible uses boto3 internally to implement its functionality. By eliminating Packer and Terraform, I’ve reduced the number of tools I rely on. This makes the MVP simpler.

Cooking vs Coding#

Coders can learn a lot from how professional chefs work. Before I start coding, I use the simple process explained in this article to have every tasks clearly defined for the day. When I begin to work, I can start using the AI tools to generate the code. Verify it works and keep iterating until the task is completed. Otherwise I will waste time on things like looking for sample code that I already saved in some document.