Building an Org-mode Workflow: Epic Organization
Not long after my last Org-mode post, we started planning for a new release cycle at work. Previously, I was tracking individual issues or small features as level-1 headlines with action items immediately underneath as level-2 headlines with the TODO keyword. Now though, I was going to need to keep track of an entire collection of interrelated features (an 'epic' in the Agile terminology), not all of which would be things I would work on. Clearly I needed to adjust some things to keep a handle on it all.
Usage
The New Structure
This process added a couple of layers of complexity, so I started with deeper nested headlines.
Roughly speaking, level-1 headlines now each correspond to a high-level epic, which would then contain multiple level-2 story or individual feature headlines. For a while I also created level-2 headlines that would capture notes and so forth, but as things proceeded I moved away from that (more on this in the next sub-section).
* Generalized solution for adding/removing execution capacity - https://issues.redhat.com/browse/AAP-2238 - https://issues.redhat.com/browse/AAP-493 (pushed to 4.4) - https://issues.redhat.com/browse/AAP-3717 parent feature ** Add a state field to the Instance and InstanceLink models... ** Add POST access to /instances endpoint... ** Create new Receptor Ansible collection to provision Receptor node(s)...
Each level-2 story then might contain multiple level-3 headlines, sometimes a feature, sometimes just an action item. These would usually be something assigned to an individual engineer, though not always me. These would also only get the TODO keyword if they were a simple enough action item. I still wanted to track these features even if I wasn't assigned, since work that I was doing would often depend on them. Similarly, it was important that they be longer-lived than I would normal treat things, so I avoided archiving items that were technically complete.
** DONE Add the ability to add/remove nodes CLOSED: [2022-09-13 Tue 11:33] - https://github.com/ansible/tower/issues/5972 - https://github.com/ansible/awx/issues/12847 - https://github.com/ansible/awx/pull/12623 (Alex; merged) - https://github.com/ansible/awx/pull/12655 (Alex; merged) *** DONE Instances List: Add a new Instance CLOSED: [2022-08-08 Mon 17:13] - https://github.com/ansible/tower/issues/6036 *** DONE Instance Detail: Add new fields and Peers tab CLOSED: [2022-08-26 Fri 10:09] - https://github.com/ansible/tower/issues/6038 *** DONE review 12623 CLOSED: [2022-08-09 Tue 13:26] *** DONE review 12655 CLOSED: [2022-08-23 Tue 09:59] *** fail to put node in deprovisioning state when removing from UI - https://github.com/ansible/awx/issues/12847 - https://github.com/ansible/awx/pull/12855 (Alex; merged) *** DONE Update Instances list view "Status" column to reflect new states CLOSED: [2022-08-31 Wed 11:00] - https://github.com/ansible/tower/issues/6035 - https://github.com/ansible/awx/pull/12773 (Kia; merged) *** DONE Remove hop nodes from Add Instance form CLOSED: [2022-09-09 Fri 14:51] - https://github.com/ansible/awx/pull/12774 (Kia; merged) *** DONE Allow k8s to create Instance Groups CLOSED: [2022-09-09 Fri 14:52] - https://github.com/ansible/awx/pull/12781 (Kia; merged) *** Instances List: Edit, Sync, Remove Instance - https://github.com/ansible/tower/issues/6041 - https://github.com/ansible/awx/pull/12784 (Alex; merged)
Note that while I wasn't typically giving level-2 headlines a TODO keyword, I did give it a DONE keyword when everything underneath of it was also done.
Finally, each level-3 feature (if I didn't already consider it to be a TODO item) might then contain level-4 action items. These corresponded to the same things I was previously marking as TODO items, such as "Experiment with some ideas", "Write a PR", or "Schedule a meeting", if it was a feature that was assigned to me, or things like "Review PR 1234" if it was someone else's work.
*** Instances List: Edit, Sync, Remove Instance - https://github.com/ansible/tower/issues/6041 - https://github.com/ansible/awx/pull/12784 (Alex; merged) **** DONE review 12784 CLOSED: [2022-09-09 Fri 14:52]
Most of these headlines would have links associated with them. Level-1 and -2 would often have Jira tickets, and level-2 and -3 might have Github issues or pull requests. These wind up as a bulleted list immediately under the headline (as in my examples above).
For a while I would include every link associated with a given item in the same bullet list, but that grew to be unwieldy as we accumulated dozens of planning docs and meeting recordings and digital whiteboard drawings and other such miscellany. So in addition to capturing the new layers of complexity, I also needed a way to better control what was visible.
Hiding Things
Key amongst the Org tools for controlling visibility of things are drawers. These are blocks that can easily be folded and unfolded without having to be their own headline item, allowing me to hide things out of the way until I need them.
I have come to use these for lists of links that aren't the ticket or PR links that ought to remain visible. I also use them for long-lived notes around planning each feature, and also org checklists to keep track that all of the acceptance criteria are met.
** DONE Add a state field to the Instance and InstanceLink models
CLOSED: [2022-08-25 Thu 13:20]
- https://github.com/ansible/tower/issues/5966
- https://github.com/ansible/awx/pull/12587 (me; unified PR; merged)
:notes:
- [X] update the management commands
- [X] pack the link info into summary_fields or whatever, to avoid
a call to a related endpoint
- [X] the UI needs to be made aware of the healthy -> ready change
- [X] open question: is the enabled toggle orthogonal to the state?
- answer: yes
- disabled nodes shouldn't transition into unavailable from a check
- disabled being a state implies that the user can set disabled
and ready states under some circumstances
:END:
Another thing that I wanted to control the visibility of was those items of work that had already been completed. It didn't make sense to tuck them into a drawer since they were headlines, but it was still desirable to reduce the amount of lines that they typically took up in the Emacs buffer. For this case, I started using the property :VISIBILITY: folded.
To add a new property to a headline, use C-c C-x p. You will then be prompted for the name of the property, which can be tab-completed if it is one of the built-in properties, and then you will be prompted for the value. The result will be added to a :PROPERTIES: drawer, which will be created if one didn't already exist.
*** DONE develop implementation plan (sprint 1) [11/11]
CLOSED: [2022-06-30 Thu 11:48] DEADLINE: <2022-07-01 Fri 17:00>
:PROPERTIES:
:VISIBILITY: folded
:END:
In order to make sure that both drawers and items with :VISIBILITY: folded are folded by default when the file is first opened, I had to configure the org-startup-folded setting to be 'showall instead of its default value of 'showeverything.
Tags
The final piece of the puzzle is that I started to use tags to mark up items in my Org files. Mainly, I used these to mark the release (:release4_2:, :release4_3:), the component or components the work affects (:api:, :ui:, :cli:, :collection:, :operator:, :qe:, :docs:), or the expected "sprint" that the work was going to be done in (:sprint2:, etc.).
** DONE Add a state field to the Instance and InstanceLink models :api:sprint2:
Tags are added by using C-c C-q, and then filling in the name in the prompt. This can be tab-completed if the tag exists elsewhere in the file, or somewhere in the Org configuration.
Once an items are tagged, I can then use C-c a m to search for agenda items with a given tag, or C-c a M for those specifically that are also TODO items, though it was often enough that I could take in at a glance what the tags were for an item. Tags do get inherited by child headlines, so the search will pick them up.
To make it easier to use tags, allowing them to be tab-completed, I added this configuration to the top of my devel.org file:
#+TAGS: api ui cli collection operator qe docs sprint2 sprint3 sprint4 release4_2 release4_3
Settings
- org-startup-folded: Determines the visibility of the contents of org files when you first visit them. The default value of this setting is showeverything, which shows the entire contents of the file unfolded. I want drawers and things marked as folded to not be visible initially, so I've changed this to showall.
After this change, my org-mode section in .emacs looks like this:
(global-set-key (kbd "C-c l") #'org-store-link)
(global-set-key (kbd "C-c a") #'org-agenda)
(global-set-key (kbd "C-c c") #'org-capture)
(setq org-startup-folded 'showall
org-agenda-files (list "~/org/")
org-refile-targets '((org-agenda-files :maxlevel . 5))
org-refile-use-outline-path 'file
org-log-into-drawer t
org-log-done 'time
org-log-redeadline 'time
org-log-reschedule 'time
)
Key Bindings Learned
- TAB: cycle through the visibility options for any item, even if
it is folded by default (i.e. drawers or subtrees with
:VISIBILITY: folded set)
- C-u C-u TAB: switch back to the default visibility for everything in the buffer.
Tags:
- C-c C-q: set the tags for the headline.
- C-c a m: search for headlines with a given tag
- C-c a M: search for TODO headlines with a given tag
Drawers:
- C-c C-x d: create a new drawer, prompting for the name.
Properties:
- C-c C-x p: create a new property, prompting for the name and then the value. So far this was always VISIBILITY and then folded for me.
Checkboxes:
- C-c #: updates the progress cookie for a checklist when you are in the checklist but not directly on the cookie.
- C-u C-c #: updates all of the progress cookies within the file.
Next Steps
With the increase in the number of things being tracked, I also wanted a way to let more important tasks rise to the top of my agenda. So next time, I'll talk about priority cookies.