The ROI on following best practices has fundamentally changed

Following best practices has always paid off — if you do a lot of something, there are clean returns to doing it the right way. But there was always a cost that came out of the main thing: learning them, setting them up, maintaining them. What’s changed is both sides of that equation at once. AI made best practices dramatically cheaper to adopt, and — because AI is far more useful on top of clean logs, organized data, and good tooling — following them pays back more than it ever has.

AI as a linear transformation of the cost-payoff plane payoff old break-even — payoff = cost new break-even — payoff = (α/β) · cost everything in here just became worth it always worth it still skip cost to adopt x-axis squashed — AI does the setup, so every cost gets ÷ α y-axis stretched — AI uses the output, so every payoff gets × β

Examples of best practices in software development

  • If you have access to observability and performance tooling like Datadog or Grafana, actually using them and putting in the legwork to create the right metrics, alerts, and profiling
    • E.g. setting up session replay on critical routes, creating the right custom metrics, setting up performance profiling in a way that won’t nuke your spend
    • Change: AI agents can learn all the DSLs you don’t care about and use them to their fullest in order to make actually helpful alerts and dashboards
  • Remove PII, secrets, and anything else that shouldn’t be there from application logs. Actually logging things in a way where “errors” are problematic or ideally unexpected behavior
    • Change: you can now have AI do lots of audits on your code and make sure only the right things are getting logged, and once everything is clean you can in good faith point AI agents at your logs in order to debug issues
  • Have a serious test suite that can actually replicate most production errors
    • If you are running a standard backend API most backend APIs can reproduce most errors with a database that matches prod schema, realistic DB state, and mocking all network calls
    • Change: probably still needs some guiding, but AI agents can set up a lot of the infrastructure for these, and then once they are actually good and in a place where tests can just be added, you can just mock production errors, confirm root cause, and have a big test suite built over time.

Best practices in personal information management

I realized that personally and professionally using best practices is more important than ever. Here are some examples:

  • Keep everything on your calendar, and sync all your calendars to one calendar
    • You can hook your AI agents into your calendar and get more personalized results. E.g. “give me some ideas for how to spend my afternoon” and it may be able to tell that evening I am going to a dinner in a certain neighborhood in NYC so it makes sense to go to a famous sports bar nearby and watch the World Cup game
    • AI can take a look at your calendar and do research on important events. E.g. if you are meeting someone about a job opportunity it can prepare notes for you
  • For anything that is your “source of truth”, prioritize services that let you actually pull the data out
    • E.g. Strava only lets you pull your data out if you are a premium member, ideally make a running watch your source of truth and then pull the data from there
    • Change: it was always nice to have access to your data, but I was always going to be too lazy to take advantage of it

Some examples of myself using this lately

  • Datadog logs our % of time on an outbound request heavy service. On a service that does ~2 million a day of requests we were spending about 20% of total time on the Python requests package doing requests.get, requests.post.
    • Ended up moving our service to IaC, parameterizing the core items around concurrency and instance numbers
    • Being able to tie those changes in VCS to our Datadog led to reducing costs by > 50% and better performance
  • I had Claude do an audit of how we were using Datadog - cleared out tons of stale alerts, fixed bugs in our core dashboard, and started tracking metrics in a way that was actually useful to the business
  • I synced all my calendars to one, including external event services like Partiful, so that I can get personalized reminders on events I have coming up.
    • For example I have a potluck coming up, so I will get some alerts from my agent to actually cook some food for that before it
    • I have an OpenClaw type agent that runs at the beginning of the day and tells me if I have anything to do before these meetings
  • I finally moved all my emails to one, so I can easily interface with one email account with my agent

AI or not, these were all best practices to begin with

Software logs are supposed to be squeaky clean, your work and life should be organized, you should use industry best practices like observability etc. The difference is that now these are the difference between AI being moderately useful and incredibly helpful in organizing your life and work.

Best practices are often what determines if you actually get leverage from AI

If you aren’t using best practices — your data is accessible, safe to access, organized right — you just won’t get a lot of return from AI. These kinds of things were previously arduous to set up and maintain correctly, and now it is much easier. Previously the benefits existed but were often diffuse. Now you can directly get benefits.

Implications

There is a lot of talk of whether or not AI will actually lead to productivity increases since so far it has been hard to measure. I wonder if in information related fields we will see a rise for no other reason than all production Python code will end up getting type hints properly or issues surfaced by logging might actually get read. People can generally ask “how to make my code more reliable” and get told they need to account for requests.ConnectionError and requests.ReadTimeout properly.

For myself the main reason I’m writing this is to remind myself when working on something to step back and be like “Am I leveraging the best practices here? How hard would that be to get to?” and “Is AI able to grok this part of the problem I’m working on or this aspect of my life so that it can assist?”. In the long run hopefully I will be more organized and productive.