The dangers of refactoring to an idea

A self retrospective

Recently I started a new position. After a couple of (I hope) successful additions to de codebase, I was entrusted with a bug fix that sparked in me an idea for a potential refactor.

I discussed my idea with some of my new colleagues and got some buy-in and time to execute the refactor.

I took about a week tinkering and delivered my PR, all the tests passed, and all seems okay.

And still, I have a nagging feeling of dissatisfaction with the work I produced. It works, I believe, fine enough and the solution is, if I might self-congratulate, elegant enough.

So, why do I have this feeling of dissatisfaction?

Please embark with me, dear reader, on a small introspective journey while I try to make sense of these feelings. With some luck, we will all learn something from the exercise (at the very least I hope I will).

The unease begins.

When reflecting on what created my nascent unease with my work, I believe I can pinpoint it to two things. One of them concrete, one of them more intangible.

On the concrete side of things, is the file changes count and the lines +-. I changed about 45 files and added ~2000 to remove ~200 lines.

This is A LOT, and made me pause when I realized it.

It also, I believe, contributes to the intangible part, explaining and self-reviewing my changes in writing felt very hard, not so much on a conceptual level, as on the sheer amount of changes.

Red flags(?)

Okay, so those things made me feel awkward, but are they red flags?

On a vacuum, yes, certainly.

But are the changes I made justified or am I justifying them to myself due to the sunk cost fallacy?

Goals accomplished (?)

I think, to make sense of all of this I need to separate the actual PR (and what I think the goals of a PR should be) from the refactor (and what I think the goals of a refactor should be)

Goals of a PR

In my opinion, a PR should accomplish the following goals:

  • Be cohesive (success)

  • Be easy to review (failure)

  • Have a small diff (failure)

1 out of 3, and in my opinion not the most important success, so here I clearly failed (I'll reflect below on what I could have done differently)

Goals of a refactor:

  • Make future changes easier (success)

  • Make the code easier to parse (success)

In this, I feel like I succeeded, I believe the code is easier to parse and easier to change later.

Where did I go wrong?

I started the refactoring by presenting an idea to the team, which is not a bad thing in and of itself, but I believe I fleshed out the idea too much.

It also prompted me to start the refactoring by my 'wanted' result rather than step by step.

This had two consequences:

  • My thinking was less flexible

  • I branched out the oath by inserting my 'final' class and broke the tests for a week.

What I should have done differently

Giving a high-level idea of the changes is fine, but I fleshed it out too much.

More importantly, I should have done the refactoring step by step and starting from the existing code, making sure the relevant specs pass all the way.

This would have provided me with natural stopping points where I could have submitted several PRs instead of the monstrosity I submitted.