Creating World-Class Feedback Loops

August 23, 2024

In the summer of 1995, hackers Paul Graham and Robert Morris decided to start an online business. They wanted to write software that anyone could use to generate an online store. In just weeks, they had a viable product ready for testing. Instead of making a downloadable application, the two entrepreneurs opted to create a web-based software (a revolutionary idea at the time). This would allow them to quickly release new, updated versions of their software without forcing their clients to download clunky applications over and over again.

Being the sole employees of their own company, Paul and Robert became both salespeople and customer support for all of their clients. And as customer support, they were bombarded with complaints about broken and missing features. Because the feedback came straight from the customers to the developers with no stops in between, the rate of getting fixes to customers was unprecedented, and the quality of their product increased at a wild pace.

Three years later, the two hackers sold their company, then named Viaweb, to Yahoo! for $50 million.

As Steven Levitt and Stephen Dubner write in their book Think Like a Freak:

"The key to learning is feedback. It is nearly impossible to learn anything without it."

When we pay attention, the world lets us know how things are going. We get grades in school, make people laugh, and receive traffic citations. These are all forms of feedback generated by our actions.

For better or for worse, we get to choose which sources of feedback to listen to. If we are to purposefully improve and produce, we must learn to use proper feedback to our advantage. Professionals do this through creating their own feedback loop.

A feedback loop is a system that captures, and analyzes the information generated by our actions.

In his first restaurant, Gordon Ramsay inspected every plate that came back to his kitchen. He wanted to see what customers couldn’t get enough of, leaving unfinished, and what they were totally ignoring. Ramsay then adjusted his dishes accordingly, adjusting spices and changing plating ratios to perfect the dining experience for each customer. His feedback loop has earned him 17 Michelin stars, making him the third most decorated chef ever.

You don’t have to be a professional to build a world-class feedback loop. In fact, the opposite is true; creating a world-class feedback loop will make you a professional.

Building a World-Class Feedback Loop

Feedback loops, generally speaking, are made up of the following elements:

• A growth mindset
• A source of information
• Methods for capturing information
• Methods for analyzing information

Just like a malfunctioning heart valve can ruin an otherwise healthy body, a poorly chosen source of information can throw a whole feedback loop out of whack. By understanding the role and function of each element, you can create a world-class feedback loop of your own.

The Growth Mindset vs the Fixed Mindset

The first pitfall of a world-class feedback loop is pride.

In Josh Waitzkin’s book The Art of Learning, he talks about different championship mindsets.

The first is that of a winner. A winner is at the top of their game when they beat their competition. When they lose, however, they are crushed because they are no longer on top. The second mindset is that of a learner. To a learner, winning and losing don’t matter all that much. The thing that matters is improving for the next round of whatever you are doing. In the long run, learners beat out winners every time.

These attitudes are commonly referred to as the fixed and growth mindsets. A fixed mindset sees itself as intelligent and loses itself when it is proven wrong. A growth mindset seeks betterment, and is willing to look stupid in its quest.

A quote by John Ousterhout, a Professor of Computer Science at Stanford University, sums it up:

"If you write a piece of code and it seems simple to you, but other people think it is complex, then it is complex."

The foundational element of any productive feedback loop is humility. If you think you are right and others are wrong, then you won’t listen to feedback and you won’t learn a thing.

Adopt a growth mindset.

Sources of Information

"True words are often unpleasant; pleasant words are often untrue"
- Lao Tzu

When building a feedback loop, you choose which sources to listen to. The trick is identifying sources of honest, objective information.

The great pitfall in choosing which sources to listen to is validation. Because we like to feel like we are doing a good job, we often search for validation. While validation can make us feel good for a short time, it is rarely useful. If you actually want to set up a useful feedback loop, you want objective information, not validation.

You can test a feedback source by giving it something you know is bad. Overcook and undersalt a chicken breast, then serve it to your spouse for dinner. If they don’t complain, or if they complement the meal, they aren’t a valid source of cooking-quality information. If they tell you that the food isn’t good, they are a good source of information. If they tell you that the chicken is overcooked and needs salt, they are a great source of information.

Look for sources of objective information.

Capture

Information doesn’t do you any good unless you can collect it.

Few groups are better at capturing data than professional sports teams. Players, coaches, and analysts alike collect an insane amount of footage and statistics. They see every missed shot and note every dropped pass.

Be thoughtful when choosing tools to capture information.

The astronomer, philosopher, and mathematician Arthur Eddington said that, if you use a net with 3-inch holes to gather and study fish from the ocean, you will come to the conclusion that there are no fish out there smaller than 3 inches wide.

If your tools leave you with incomplete or incorrect information, your feedback loop can become ineffective or counterproductive. Incomplete information makes improvement slow. Incorrect information sends you in the wrong direction.

Information also ceases to be practical when it becomes unprocessable. It is possible to gather too much data. When Gordon Ramsay was collecting data on his dishes, he didn’t learn about the patrons at his restaurants (where they were from, what they did for work, etc.). He focused on what they ate and what they left behind. There is a lot of information you could gather that might be useful, and even more information that isn’t useful at all.

When selecting methods of capture, think about what information is useful to you and which tools will best capture that information.

Analyze

This is where humility comes into play.

Analysis is the process of turning data into useful, actionable information.

When an NFL wide receiver watches tape of himself dropping a pass, he sees his body position, his defender, where the quarterback threw the ball, and how accurately he ran his assigned route. He can then identify his faults and work to fix them.

Analysis begins by asking questions to identify the area(s) that needs improvement:

• Why did I drop the pass?
• Did I run my route poorly?
• Did the quarterback underthrow the ball?
• How can I identify a poor pass when it is thrown?
• What can I do to improve my ability to catch poorly-thrown passes?
• What can I do to improve reaction time and lateral quickness?

Without analyzing the data, the wide receiver might set up a plan to improve running routes. Instead, he sets up a training routine to improve his decision making and lateral agility, thereby increasing his ability to catch poorly-thrown passes.

Boom! The real issue has been found, and an actionable solution is in the works.

This is an important note:

Anecdotes are not data. One data point, no matter how compelling, is an anecdote. Watching the same video of a dropped pass 1000 times is not as useful as watching 100 videos of dropped passes 10 times each.

Find insights in your data by asking and answering honest questions.

Making Our Feedback Loop World Class

Feedback is the key to iterative improvement. Once you have an actionable plan, you start the process over again. You act, capture information from a reliable source, and analyze that data to make a new plan. With each iteration, you improve a bit, either in performance or knowledge or both. Through continually iterating through a well-designed feedback loop, you reach your potential.

But we don’t just want a well-designed loop; we want a world-class one.

There are two levers that separate an amateur feedback loop from a professional-grade system; distance to feedback and rate of feedback.

Distance to Feedback

Paul Graham and Robert Morris received feedback directly from customers, and then made improvements directly to their code. The distance between the source of feedback and the people doing the work was one conversation. Gordon Ramsay didn’t have an intern look at unfinished food and write a report for him on what could be improved; he looked at the plates of food coming back to his kitchen himself and made adjustments accordingly. His distance from feedback was one interaction.

You control how close you are to your sources of information.

Rate of Feedback

If you gather customer reviews of a product via email survey and check the stats once a month, you will be able to run through the improvement process at most once a month. If you receive feedback faster than you can make improvements, much of the effort you put into gathering useful information will be wasted. Capturing data is only one step in a cyclical feedback loop. Trying to gather feedback at blinding speed can be a detriment to analysis.

Strive to gather feedback as often as you can make meaningful improvements.

The Point

Feedback loops are powerful tools for improvement. You can use that tool however you would like.

What is something that you wish to improve in your life?

The quality of your working life? Your athletic prowess or your overall health? Your relationship with your kids?

Find a source of objective information, capture some data, and use that data to inform decisions about what to do next. The more you loop through your feedback system, the better it will become, and the better you will become at using it.

Set up a feedback loop. Make it progressively tighter, faster, and more objective. Make it world-class.

It will change your life for the better.