Skip to main content

My biggest technical mistakes (so far)

We all make mistakes in the technical planning process that bubble through to execution. Looking back over the past 10 years or so, these are the two biggest tech & development mistakes that I think I have made and the lessons that I’ve learned from them.

Choosing Flow over TypeScript

Back in about 2016, typing JavaScript was still coming into the scene. At Twitter, we were evaluating both TypeScript and Flow. We wanted to add type-safety because it was too easy to accidentally miss required props across components that ensured ads were attributed as such. Without that information, Twitter wouldn’t get paid. The fact is, it happened and we lost out on a lot of money across a few days.

We ended up choosing Flow because:

  • We had many Babel plugins we needed to use and Babel was not TypeScript-compatible. Switching to TypeScript would mean using tsc and ts-loader with Webpack and we would have to rewrite our entire compilation setup and change any Babel plugins into Webpack plugins. This would have been a huge undertaking.
  • React was written using Flow and thus was poised to be a better support model.
  • TypeScript either did not have a strict-enough mode in it. There were too many examples at the time that would just pass through, despite best intentions by developers to produce strictness.
  • TypeScript was slow across large codebases.
    • Flow as fast. We were able to get checking across an entire monorepo with about 10,000 Flow-typed files to run in about 20 seconds.

Flow ended up being a mistake because:

  • Adoption lagged. While we jumped on Flow, everyone else started jumping to TypeScript.
  • The core team at Facebook changed directions multiple times and eventually stated publicly that they didn’t really intend to continue interfacing with the open source community, but would continue publishing releases.

Unfortunately, before we realized that TypeScript was taking over and would have been a better choice, we were somewhere like 75-85% Flow coverage. Interop between Flow and TS is not possible and so another slow migration over to TS would not be possible.

Lesson for type safety

I’m not sure that at the time we could have seen it coming that Flow would essentially be dead for anyone external to Facebook/Meta. Next time something like this comes up, I will try to do more due-diligence in checking with the open source teams in understanding their roadmap as well as watch the community’s traction more closely. I would also try to make sure we have an ejection plan if things go sour with the choice we make.

Using a multi-language monorepo

This has actually happened multiple times. The first was at Twitter, which historically had a single “source” repo. Everything was in this repo: from Scala, Python, Ruby, some PHP, and yes, JavaScript. While there was a team dedicated to ensuring that the CI pipelines and tooling were all fast, JavaScript teams were small and spread, without much agreement on stacks. Because of that, the DX team prioritized Scala over anything else. While things “worked” for JavaScript projects, everything was unacceptably slow or poorly linked (eg, 30-60 seconds to switch git branches locally and 30+ minute pull-request CI runs).

After many months of planning, campaigning, and meetings with the DX team, I was able to get my team to build our own monorepo that was for meant for JavaScript only. The results were staggeringly awesome. We were able to cut the amount of time developers in our codebases wasted just waiting around for slow process by about 2 hours per developer per week. Across about 75 contributors weekly, this was a very material gain. On top of that, I received quite a number of kind words of appreciation and happiness for this work 💙.

The second time I made the mistake of using a multi-language repository was at Microsoft for Startups. While our tech stack was small across just Python and Node/JavaScript, it was still really troublesome to feel confident and safe with changes across the different languages. Because of the weak connection between the two stacks, changes needed to be tested carefully and it was always confusing what command you ran to check each individual piece.

Lesson for monorepos

My lesson for Monorepos is that they are actually good, but they only scale as long as you’re using a single language and every workspace internally has some connection to the same graph. As soon as you have a workspace or project that doesn’t use any of the same stack, tools, or language, a new repo is likely the best course forward.