Learning Materials/Problem Solving

Problem Solving for Engineers

Master systematic approaches to complex challenges.

Problem solving is the core of engineering. But at its heart, problem solving is really about learning - the ability to acquire new knowledge quickly, build accurate mental models, and apply that understanding to novel situations. The best problem solvers aren't those who memorize solutions; they're skilled learners who can adapt their knowledge to whatever challenges emerge.

This guide explores problem solving as a learning discipline, examining the research and frameworks that help engineers develop this meta-skill across all career levels.

The Problem Solver's Mindset

Learning as the Foundation

The connection between problem solving and learning is fundamental. Every bug you encounter is a gap in your understanding. Every architectural challenge reveals the boundaries of your mental models. swyx (Shawn Wang) captured this insight in his influential essay "Learn in Public":

Make the thing you wish you had found when you were learning.

This principle transforms problem solving from a reactive activity into proactive skill development. When you document your debugging journey, explain your solution, or teach a concept to others, you're not just solving today's problem - you're building capacity for tomorrow's challenges.

The fastest way to learn, according to swyx, is to create "learning exhaust" - blogs, tutorials, talks, cheatsheets. This isn't about building an audience; it's about forcing yourself to articulate your understanding clearly enough that others could follow:

Don't judge your results by claps or retweets - just talk to yourself from 3 months ago.

The Feynman Technique

Richard Feynman, the Nobel laureate physicist, developed a learning method that applies directly to problem solving. The technique has four steps:

  1. Select a concept and map your knowledge: Write everything you know about a topic. Use different colors for new information as you learn.
  2. Explain it to a 12-year-old: If you can't explain it simply, you haven't grasped it fully. Complexity and jargon often mask lack of understanding.
  3. Review and refine: Identify weak spots and return to source material. Re-write sections as your understanding improves.
  4. Test and archive: Teach someone else without notes. Archive your explanation for periodic review.

As Mortimer Adler put it: "The person who says he knows what he thinks but cannot express it usually does not know what he thinks."

For engineers, this means that explaining a bug to a rubber duck, writing a postmortem, or documenting an architectural decision aren't just communication exercises - they're learning techniques that deepen understanding.

Systematic Debugging

Julia Evans' Debugging Manifesto

Julia Evans, creator of the "Wizard Zines" on technical topics, distilled years of debugging wisdom into a manifesto:

1. Inspect, don't squash: The natural instinct is to fix bugs as fast as possible. But it's generally more effective to leave the bug in place, figure out exactly what's gone wrong, and then fix it after understanding what happened. Trying to fix without fully understanding usually leaves you more confused.

2. Being stuck is temporary: Debugging can feel demoralizing when progress stalls. Remember: you've fixed bugs before, and you'll fix this one too.

3. Trust nobody and nothing: Sometimes bugs come from surprising sources. Even popular libraries, operating systems, official documentation, or extremely competent coworkers can be wrong. Question everything.

4. It's probably your code: That said, 95% of the time something is wrong with your program because you did something silly. Look for the problem in your own code first before blaming external sources.

5. Don't go it alone: Collaboration accelerates debugging. Coworkers can share debugging tools ("here's how to use GDB to inspect memory"), explain how things work ("can you explain CORS?"), or recall similar past bugs ("I've seen this break in X way before").

6. There's always a reason: Sometimes it feels like things are randomly breaking for no reason. That's never true. Even hardware problems are still reasons.

7. Build your toolkit: To fix bugs you need information about what your program is doing. Sometimes you need to learn new tools (tcpdump, strace, debuggers). Sometimes you need to build better tools (improved test suites, pretty printing, better logging).

8. It can be an adventure: Debugging is an investment in future knowledge. If something is breaking, it's often because there's something wrong in your mental model - an opportunity to learn.

Rubber Duck Debugging

The rubber duck debugging method, popularized in "The Pragmatic Programmer," is deceptively powerful:

  1. Obtain a rubber duck (bathtub variety).
  2. Place it on your desk and inform it you're going to go over some code.
  3. Explain to the duck what your code is supposed to do, then go into detail and explain your code line by line.
  4. At some point you will tell the duck what you're doing next and realize that's not what you're actually doing. The duck will sit there serenely, happy in the knowledge that it has helped.

Note: In a pinch a coworker might substitute for the duck, but it's often preferred to confide mistakes to the duck instead of your coworker.

The power of rubber duck debugging lies in forcing articulation. When you explain code to an external entity - even an inanimate one - you shift from reading to teaching, activating different cognitive processes that reveal gaps in understanding.

Binary Search for Bugs

When facing a bug in a complex system, binary search debugging systematically narrows the problem space:

  1. Identify the boundaries: Where does the bug definitely exist? Where does it definitely not exist?
  2. Split the difference: Add logging or checkpoints at the midpoint.
  3. Determine which half contains the bug: Did the midpoint show correct or incorrect behavior?
  4. Repeat: Continue halving until you've isolated the issue.

This technique works for:

  • Code: Add logging statements at the midpoint of the execution path
  • Time: Use git bisect to find which commit introduced a bug
  • Data: Test with half the input to determine if the issue is data-dependent
  • Configuration: Toggle half the settings to isolate problematic configuration

The key insight: linear searching through possibilities is O(n); binary searching is O(log n). For a bug that could be anywhere in 1000 lines of code, linear investigation might require 1000 checks; binary search needs only 10.

Root Cause Analysis

The Five Whys

The Five Whys technique, developed by Sakichi Toyoda and used within Toyota's production system, explores cause-and-effect relationships by repeatedly asking "why":

Example: Bolts are cross-threading in the engine block.

  1. Why? The threads aren't cut cleanly.
  2. Why? The cutting tool wasn't changed today.
  3. Why? The replacement tool bin was empty.
  4. Why? The bin's contents had fallen and rolled under the shelves.
  5. Why? One of the shelf feet has rusted and failed, making the shelves unstable.

The fifth "why" reveals a root cause that can be addressed - replace the shelf foot. But the true insight goes deeper: the factory needs a process for regularly inspecting shelving units.

For software engineers, apply Five Whys to production incidents:

  1. Why did the service fail? A database query timed out.
  2. Why did the query time out? The table had grown to 10 million rows without indexes.
  3. Why was there no index? The migration adding it was never run in production.
  4. Why wasn't it run? There's no automated migration check in the deploy pipeline.
  5. Why isn't there an automated check? No one owns database migration verification.

The root cause isn't the missing index - it's the missing process ownership.

Limitations of Five Whys

Toyota's own Teruyuki Minoura criticized Five Whys as too basic for deep root cause analysis:

  • Investigators tend to stop at symptoms rather than true root causes
  • You can't find causes you don't already know
  • Different people produce different root causes for the same problem
  • It tends to isolate a single root cause when multiple causes may exist

Use Five Whys as a starting framework, but supplement with techniques like fishbone (Ishikawa) diagrams when problems are complex or multi-causal.

Mental Models for Problem Solving

Charlie Munger famously said: "If you want to be a good thinker, you must develop a mind that can jump jurisdictional boundaries. Just take in the best big ideas from all disciplines."

Mental models are simplified explanations of how things work. Like maps, they highlight key information while ignoring irrelevant details - tools for compressing complexity into manageable chunks.

First Principles Thinking

First principles thinking breaks down complex problems into their fundamental truths, then builds up solutions from there:

Reasoning from first principles allows us to step outside the way things have always been done and instead see what is possible.

In a world focused on incremental improvement, first principles thinking offers competitive advantage because almost no one does it. It requires willingness to challenge the status quo, to start from scratch and build from the ground up.

For engineers: When inheriting legacy code, don't just patch - ask what the system is fundamentally trying to accomplish. Sometimes the right answer is rebuilding rather than extending.

Inversion

Much of success comes from simply avoiding common paths to failure:

Instead of 'How do I solve this?', inversion asks, 'What would guarantee failure?' Rather than 'How can I achieve this?', it asks 'What's preventing me from achieving it?'

When facing a tricky problem, try inverting. Ask how you'd guarantee failure. The answers may surprise you and unlock new solutions.

For debugging: Instead of asking "why doesn't this work?", ask "what would have to be true for this to work?" Then verify each assumption.

The Map is Not the Territory

This model reminds us that our mental models of the world are not the same as the world itself. It cautions against confusing abstractions and representations with complex, ever-shifting reality.

Consider the person with an outstanding résumé who checks all boxes on paper but can't do the job. Updating our maps - reconciling what we want to be true with what is true - is difficult but essential.

For engineers: Your understanding of the codebase is a map. The actual code is the territory. When the behavior surprises you, your map is wrong. Don't argue with the territory.

Occam's Razor

When faced with competing explanations, the correct one is most likely the simplest - the one making the fewest assumptions. This doesn't mean the simplest theory is always true, only that it should be preferred until proven otherwise.

For debugging: Start with simple hypotheses. "The environment variable isn't set" before "there's a race condition in the kernel scheduler."

Hanlon's Razor

Never attribute to malice that which can be adequately explained by incompetence (or simply, mistake).

For code review and debugging: When you find confusing code, assume the author was facing constraints you don't understand, not that they were deliberately writing bad code. This mindset leads to better conversations and more learning.

The Learning Posture

Learn in Public

swyx's "Learn in Public" philosophy transforms problem solving into visible skill development:

Whatever your thing is, make the thing you wish you had found when you were learning. Don't judge your results by claps or retweets - just talk to yourself from 3 months ago.

Key principles:

  • Wear your noobyness on your sleeve: Being obviously a beginner is an advantage. Senior engineers will help you.
  • Try your best to be right, but don't worry when you're wrong: "Repeatedly. If you feel uncomfortable, or like an impostor, good. You're pushing yourself."
  • Pick up what mentors put down: When a respected engineer mentions a concept they wish someone would explain, that's your opportunity.

The teaching-learning connection is powerful: you learn best by teaching others. Not because you're an expert, but because explaining forces precision.

Ultralearning

Scott Young's "Ultralearning" describes aggressive, self-directed learning - taking control of what and how you learn, directing effort toward what works:

Self-directed means taking control; aggressive means directing effort toward what works.

For problem solving, this means deliberately seeking challenges slightly beyond your current capability, using feedback loops to adjust, and focusing on fundamentals rather than surface-level tricks.

Practical Problem-Solving Techniques

Define the Problem Precisely

"It's broken" isn't a problem statement. "Users see 500 errors when submitting forms with special characters" is.

Good problem definitions include:

  • Who is affected
  • What behavior is occurring
  • When it happens (always? under specific conditions?)
  • Where in the system (which service? which endpoint?)
  • How it manifests (error messages, incorrect data, performance degradation)

Reproduce Before Diagnosing

Speculation without reproduction leads nowhere. Before hypothesizing causes:

  1. Reproduce locally if possible
  2. Create a minimal case that exhibits the behavior
  3. Document reproduction steps precisely
  4. Verify the reproduction shows the same symptoms

A bug you can't reproduce is a bug you can't confidently fix. Even if you guess the cause correctly, you can't verify the fix.

Work Backwards from Outcomes

Start with what success looks like. Let desired outcomes guide solution exploration.

For a bug: What should the correct behavior be? Work backwards from correct output to find where divergence occurs.

For design: What does the ideal user experience look like? Work backwards to determine what systems must exist.

Decompose Complexity

Large problems are collections of small problems. Break them down until each piece is tractable.

Techniques:

  • Divide and conquer: Split the problem space in half repeatedly
  • Separation of concerns: Which parts are independent?
  • Time decomposition: Break the problem into phases (before, during, after)
  • Layer decomposition: Which layer is responsible? (network, application, database)

Consider Multiple Approaches

Don't implement the first solution that comes to mind. Generate options, then evaluate tradeoffs.

For any significant problem:

  1. Generate at least three possible approaches
  2. Identify the tradeoffs of each
  3. Consider which constraints are actually immutable
  4. Evaluate second-order consequences

Know When to Step Away

Sometimes the best debugging technique is a walk. Fresh eyes solve stuck problems.

The science: Cognitive load and tunnel vision reduce problem-solving capacity. Breaks allow:

  • Subconscious processing to continue
  • Fixed attention patterns to reset
  • New perspectives to emerge
  • Stress hormones to dissipate

Many engineers report "shower breakthroughs" - solving problems while not actively thinking about them.

Level-Specific Guidance

Junior Engineers

Focus areas:

  • Build systematic debugging habits before developing intuition
  • Document your debugging process - it reinforces learning
  • Learn your tools deeply (debugger, profiler, logging)
  • Practice rubber duck debugging to force articulation
  • Embrace being stuck; it's where learning happens

Growth pattern: You're building the foundation. Every bug you solve adds to your pattern library. The goal isn't just to fix this bug, but to learn from it.

Mid-Level Engineers

Focus areas:

  • Recognize patterns across different problem types
  • Develop specialization in specific debugging domains
  • Mentor juniors through debugging sessions (teaching deepens your understanding)
  • Build team debugging infrastructure (better logging, monitoring, test cases)
  • Start contributing to postmortems with root cause analysis

Growth pattern: Your pattern library is growing. Now work on meta-skills - teaching, tooling, prevention. The best debugging is the debugging you never have to do.

Senior Engineers

Focus areas:

  • Design systems that are debuggable by default
  • Develop frameworks for common problem types
  • Lead blameless postmortems that improve systems
  • Create documentation that helps others debug independently
  • Tackle ambiguous problems where the problem itself needs definition

Growth pattern: Your job is increasingly about preventing problems and building others' problem-solving capacity. Debug the system that produces bugs, not just the bugs themselves.

Staff+ Engineers and Architects

Focus areas:

  • Solve problems that span organizational boundaries
  • Address problems where the first challenge is alignment on what the problem is
  • Build cultures of learning and blameless analysis
  • Create reusable frameworks and mental models for the organization
  • Develop people who develop people

Growth pattern: Your leverage comes from multiplying others' problem-solving abilities. Your debugging happens at the organizational level - why does this class of problem keep recurring?

Further Reading

Books

  • The Pragmatic Programmer - Andy Hunt & Dave Thomas (origin of rubber duck debugging)
  • Debugging - David J. Agans (nine indispensable rules)
  • Ultralearning - Scott Young (aggressive self-directed learning)
  • The Great Mental Models - Shane Parrish (thinking tools)

Articles and Resources

Related Deep Dives

TL;DR

  • Problem solving is fundamentally about learning - the Feynman Technique (explain it to a 12-year-old) reveals gaps; if you can't explain it simply, you don't understand it
  • Debug by inspecting, not squashing - leave the bug in place, understand exactly what went wrong, then fix; premature fixing leads to more confusion
  • Binary search your bugs - add logging at the midpoint, determine which half contains the problem, repeat; O(log n) beats linear investigation
  • Apply the Five Whys carefully - the fifth "why" often reveals missing process or ownership, not just technical fixes; recognize when to stop drilling
  • Know when to step away - "shower breakthroughs" happen because breaks allow fixed attention patterns to reset; being stuck is temporary

Ready to test your knowledge?

Put what you've learned into practice with our assessment.