The practice of studying about a problem at work
Why you should consider reading review papers
At work, it’s common for teams to study new technologies (what platforms or services to use, which libraries to adopt, new programming languages). We’re also very often sniffing through forums trying to wrap our heads around implementation related problems (e.g. “why aren’t my configs working as expected?” or “how come my connections are still hanging open?”).
However, it’s not very common for teams to study about more fundamental problems, that could make a critical difference on how products are conceived or designed, or even in how a problem could be leveraged as an opportunity within the company’s strategy. Here are a few questions that illustrate what I mean by more fundamental problems:
What conversion rates should we expect from this product? What are the most probable main drivers?
What are different pricing approaches to explore and how to experiment with them?
What is the most adequate metric we could be using to track our progress at this point?
Do we have all the data we need in order to check for our process conformance?
For those that went through college, discussing fundamental problems and the frontiers of knowledge around them was routine and there were text books to guide us deep and wide. But, what about work?
Business books typically cover topics at very high levels, so they can reach a wider audience and succeed commercially. They can help you learn concepts, way of thinking, best practices, but rarely at enough depth to map directly into execution.
As it turns out, academic research is a lot about diving into specific problems and publishing to the world about their learnings. Many think that consuming from academic publications is reserved to those dealing with highly complex or cutting edge problems described through puzzling mathematical equations, such as route optimization, natural language processing, etc. However, there’s a lot of business oriented research out there, even in less mathematically intricate fields such as marketing or ecommerce conversion optimization.
A great way to get started on leveraging academic studies at work is to explore review papers when getting started on a new class of problems. Review papers, referred to as secondary sources in the academic work, are studies that explore a field or problem horizontally, collecting and organizing concepts and discussing what advancements other research work on the topic have accomplished.
Studying and discussing relevant review papers with the team on early stages of a project may payoff in several ways (in addition to being cool):
Discovering the relevant concepts and aligning communication
Review papers are typically very good at organizing the relevant concepts related to the problem domain: different categories of the problem, different categories of solution approaches, metrics, solution evaluation criteria, most common input requirements, etc.
This can help the team not only prune the problem space, but also build a common internal jargon that improves communication.
Benchmarking and adjusting expectations
Such works typically make reference to industry standard numbers (conversion rates, latency, precision vs recall, etc depending on your problem). If you’re lucky to find recent enough studies, these numbers can help you and your team set the expectations about how much benefit the considered approach could yield.
Being able to tell early in the process whether an approach can or cannot bring the result standards you’re looking for means a lot of time not wasted.
Maturity of current solutions and solution insights
There’s hardly any fundamental problem known to humans that hasn’t been studied extensively. For some, we’ve found strong and highly applicable solutions. But some times academia hasn’t advanced enough on a problem space in order to propose usable solutions.
Also, it can be rare for your problem to match perfectly to those of published studies and the solutions presented might be useful as a source of insights for solving your own instance.
Prioritization
A better sense of implementation complexity vs potential yields can be very useful for prioritization purposes. You might find out it’s a low hanging fruit that’s worth pursuing or that the problem seems too hard for the benefit and explore other opportunities.
References
Review papers are literature reviews, so they are honed to help you navigate existing studies and prune your search for what are more specific works you could read.
Personal development
For some team members it could be very impacting to take a measure on how deep one could go on exploring a problem or where are the bars set for someone to become an expert on a field.
Studying as a group also fosters collaboration and creates opportunities for team members to step up and share knowledge with other colleagues. A classic approach is for the team to take turns on who will present a paper to the team (and often other guests) for a discussion session.
For others, like me, it could be a striking reminder that smart people have already thought about a problem and it’s often worth it to take some time to learn from them before jumping into problem solving.
I was lucky to meet some great people back in college years that went on to become true friends and workmates. This strong present tie with academia by working with Davi de Castro Reis, Bruno Fonseca, Akio Nakamura and Andre Paim Lemos (all who I hope will contribute to this blog at some point) helps me keep an attitude of exploring problems in depth and learn not only the tricks of the trade, but also what fundamental problems are behind the challenges we face in business, tech or people management.
A few years ago, we kicked off a service quality task force at Loggi and, since I was new to the logistics field, decided to learn more about the basics. So I stumbled upon this amazing review paper titled “A Review and Evaluation of Logistics Metrics” by Caplice and Sheffi (MIT). It defines a list of criteria for evaluating logistics metrics. It was just perfect for that moment when we were looking for ways to measure our progress and spark a corporate culture. Essentially all wisdom I have gathered since then in terms of how to think logistics management was inspired by a single (shorter than a page) section titled “Behavioral Soundness”. I became such a meta-metrics nerd after reading this paper that I’ll probably end up writing more about it.
Here’s a toast to friends, science and learning (and metrics). Cheers.
The practice of studying about a problem at work was originally published in Desirable Difficulty on Medium, where people are continuing the conversation by highlighting and responding to this story.