But when I’m working on large software projects or certain external libraries I tend to encounter bugs or design issues where I realize I made the wrong assumptions about how someone else’s code works in the first place, and a good debugger is very useful in those cases when the problem changes from debugging your own logic to reverse engineering someone else’s.

If your mental model of how the code that you're interfacing with works is wrong, then it won't help you. Your data validation will error out and you might find that it was too much too early. Not only do you have logic based on flawed assumptions but check code as well.

Understanding the code base you're programming against, that's a skill that can be improved but hoping to guard against all misunderstandings is probably unrealistic. Given that, following a trace may help you identify disagreements with your model more quickly.

Not making mistakes certainly saves time.

I often use a debugger to inspect variables and check my assumptions about the application state at that point.

I am saying you don't need a debugger in the scenario above because the only time you need it in that case is if your inputs can't be trusted.

If you control the domain, the only way you can't trust your input is if you fucked up.

The answer to that isn't "oh now I need a debugger" the answer is to go clean your code.

While I'm here, I'd also like to point out this isn't a "I'm good and your shit" thing I'm describing, this is my day job. I make lazy crap code to get things done, then I go in to modify it and find I can't reason about it, so I go and clean up the mess.

Maybe I made it, maybe I didn't, it's besides the point, you have a mess, clean that first, then reach for a debugger.

Hell reach for a debugger to help clean up if you need to, just don't sit there and tell me you need a debugger because code inherintly needs to make assumptions about what is in a variable. It doesn't. If it does, it's a code smell.

> So many errors boil down to assumptions about what is in a value.

Only avoidable errors. They should not be dictating your tools or your language.

Good programmers are humans and make mistakes, of course. However, that doesn't mean good programmers don't exist or that people can't vary a lot in skill or average quality of output.

She is correct that starting with some notion of "good people" is no silver bullet. But the conclusion that "we all suck", I hope that is hyperbole, because it isn't true. (Or maybe: everybody sucks a little bit differently.)

I am the first person to understand developers are far from perfect, my self included. But it's just code, you can refactor and learn from mistakes.

The thing I don't buy is that this stuff is difficult or that you can't expect good behavior from professionals.

Avoiding side effects, or pushing them out to the edge where reasoning about them is clear is something you should have been taught when you got your expensive piece of paper.

Likewise, avoiding, mutation, over abstraction, early optimisation, and variable reuse are all really basic shit you need to understand, if you're not hiring for these basic qualities, what the fuck are you hiring for? Good looks?

An example inconvenient debugging story: https://mjg59.dreamwidth.org/11235.html

(I don't think DMA triggers hardware watchpoints, but if you set a hardware watchpoint and an an address changes value without hitting a watchpoint you know that something funny is going on)

This. A few days ago, I spent a couple of hours trying to get a Jest+Enzyme test to open in chrome inspector, (apparently there's a bug which causes debugger statements in Jest tests to be ignored), because I hit an edge case bug in a method in the Enzyme library. If I had been able to step through it in the debugger, it would have taken me minutes to figure it out, instead I spent a couple of hours going through the codebase and figuring out where to put the log statements.

But when everyone starts taking this view on it, the result is that changes get made tactically based on what makes your task work. With no understanding of the overall vision. The result undermines the integrity of the system, and makes debuggers ever more necessary going forward.

This is a good point to hold in your mind as you read or re-read Programming as Theory Building by Peter Naur: http://pages.cs.wisc.edu/~remzi/Naur.pdf.

I guess the pie in the sky vision is that tactically making your task work somehow becomes harmonized with the overall vision. Extreme Programming was supposed to do this through the high information exchange of pair programming, the constant refactoring, and the practice of there only being 7 or so large scale patterns for the whole of the application.

The way this works in most of the real world, is that it's supposed to work like this, but there's no pair programming, and you never get enough time to refactor.

I rarely actually fire up the debugger, and I try to refrain from gratuitous prints (though I use them in emergencies or if a debugger isn't convenient).

I think this makes me a better programmer, though that's really hard to tell objectively.

You can pry my debugger from my cold-dead hands, but I don't even know how to step through code in my favorite debugger[1].

In my opinion, the only right way to fix a bug is to build a mental model of the code, and a debugger is a massive force multiplier in doing so. For any code that wasn't written by me in the past 6 months or so, the code itself is a mystery and while reading the code is a big aid in understanding how it works, it can also mislead you. In particular there is a class of bugs that are when how the code actually works diverges from how the author thinks the code works. A good author will structure the code to guide a reader into how the code works, but when the author was mistaken, this can be very misleading (in particular I will actively avoid reading comments when I know there is a bug in the code, because comments are the one part of code that are never tested).

Another way of putting it: The source code is very good for telling you how the program is intended to operate, and a debugger is very good for telling you how the program actually operates.

1: I'm sure it's listed how somewhere here, but I've never felt the need for it: https://common-lisp.net/project/slime/doc/html/Debugger.html

1. Instrument the code in one way or another. At the simplest level, many debuggers can log each function call.

2. Change the definition of functions while the system is running. Most highly dynamic languages will let you do this. I'm told that there are C IDEs that can do this too though (modulo inlining anyways).

3. Change the timing characteristics of the program; if a race condition is suspected, ordering can be forced through the use of thread-local breakpoints, for example.

4. Inject specific data. Have a function called under normal operating conditions and modify some or all of its parameters. Think instant unit-test, but no need to mock anything.

If your lisp supports it, pressing 's' in sldb should do it.

Personally I _love_ debuggers when coming to new codebases. I view code in the process of execution as the natural state of a codebase. Instead of hoping around through source code by hand, why not step into functions, continue execution, and let the program flow do it for you? Most code only has small parts that are important and with debuggers I can usually find those parts right away.

I don't blame Torvalds for not wanting to use a debugger. He doesn't like them and doesn't want to support them. That's his choice. But I find it odd to just categorically dismiss them.

One of my favorite bugs I ever introduced was typing 0 instead of O in a variable name. Review your mental model all you want, a debugger is going to be a lot more useful in sussing that kind of thing out. Even just being able to see you have two variables on the stack RockOn and Rock0n will pretty much save you.

I rather suspect that Torvalds' beef with using debuggers is that so many engineers get lazy and begin to use them as a substitute for thinking things through.

I helped found a Mathematics of Finance masters program at my university, teaching a numerical methods course before we could hire specialized staff. The students got stronger each year. My last year at it, three very strong students were trying to implement a research paper, tying together all their work to strengthen their resume. They had an impression of me as hot-shot C programmer from a computer algebra system I had written (everything is relative) and they sought my advice when their code wouldn't work.

I was in deep inner cringe, aware of their hefty tuition, how I truly didn't understand a word of what they were saying. I had to think of something to say that would get them to leave my office, apparently satisfied. Then I heard their voices catch as they apologized for introducing a fractional time step as their algorithm shifted phases.

"No! Always a bad idea! Here's another way to do that. I don't know if that's your bug, but..."

The email that evening was profusely thankful, warning me they'd be back the next day with their new bug. Launder, rinse, repeat.

While I'll probably never hang a shrink shingle in Silicon Valley, as tempting as the idea is, one can also listen to one's own voice catch. Ever notice how everything that can go wrong cooking was actually anticipated and ignored at the time? Mathematical research teaches one to listen to the faintest anomalies. It's the universe whispering truths.

Pike is conflating the diagnosis process with the quality of the chosen solution. In practice, they are often unrelated.

A good debugging tool will not only show you the state of the system when the bug happens, but help you understand the execution path that formed that state. (If your debugger doesn't do that for you, then you should either get better at using it, or find a better debugger.)

A debugger _doesn't_ tell you how a problem should be solved; that's a separate decision which often has many other factors beyond "improve the design to be more future-proof".

The story also implies that it should be enough for the programmer to explore the mental model they currently have. But if I have a problem that needs debugging, and a minute's thought isn't enough to reveal it, then it's usually because that mental model _is wrong_. I've spent enough time dealing with engineers who furiously insist that a system shouldn't be doing something that it clearly _is_ doing (including myself) to know that your mental model can be just as much of a barrier to debugging as it is an aid.

So, as often with arguments about whether you, a smart engineer, can solve all problems with pure reason: No, you can't. There are often things that you don't know, and things that you don't know you don't know, and getting external assistance is often a huge time- and embarrassment-saver.

You've got to turn off the targeting system and start thinking about problems holistically. Programming is not yet a solved problem and the really valuable element in the equation is a human's ability to reason and predict by aggregating massive amounts of information.

1. Write down the problem 2. Think really hard 3. Write down the solution

I often debug in the same way I design software. Step away from the computer and write something down.

It's probably a personal thing, but I solve problems much faster with a pen in my hand as opposed to a keyboard at my fingertips.

Some, things to note. I've worked on code bases that were in the 10s of millions of lines of code (healthcare industry). I still use this technique, but with the understanding I may never have a complete model of a code base that size.

It only makes sense to me, although it does require you to actually understand what the code is (trying) to do.

I do sometimes use the standard debugging techniques, including debuggers, logging, etc. But I end up doing that only when I'm working with code that I don't fully understand, or if I'm totally at a loss as to what's going on. I'd say that covers 10-20% of cases.

Of course you can verify these invariants with a debugger. But when you hit a bug and immediately start digging in like Pike says, that tends not to be what you're thinking about.

Debuggers are great for putting in the minimal hack in a production codebase that doesn't disturb the code. But they're generally bad for developing software, because most (all?) software should have some invariants that allow the Thompson style of debugging.

You step back, think about what invariant could have been violated to cause the bug, put in a print statement somewhere to see if it was violated. Then you take another step back to think about how to fix it, possibly with a fundamental change to the data structures. That is, you design your software differently to avoid the bug, rather than just putting in the minimal hack.

-----

The other problem with debuggers is that once you've used one and modified the code, then everybody who modifies the code later probably has to use one too. The code has become subtler, but not in a good way. This makes development slower.

-----

I think it also relates to another thing Linus said [1]:

Bad programmers worry about the code. Good programmers worry about data structures and their relationships.

It's easier to diagnose problems in data structures (by printing them) than to diagnose problems in control flow (which is greatly aided by a debugger).

One weakness of debuggers is that they don't print things in application-specific formats. You have to write debugger plugins, and those aren't set up for most projects, and they don't behave the same way on all platforms, etc.

So basically having a bug that you can't figure out without a debugger is a sign that you may be using too much CODE and not enough DATA. It's a smell that indicates a problem with the software design.

Of course, in the real world, sometimes you have to swoop in with a debugger and fix something. But I do think that there is something wrong with living in a debugger for months on end (which I have experienced on one dev team.)

[1] https://softwareengineering.stackexchange.com/questions/1631...

https://queue.acm.org/detail.cfm?id=2038036

As you can see, a number of invariants tie the different record fields together. Maintaining such invariants takes real work. You need to document them carefully so you don't trip over them later; you need to write tests to verify the invariants; and you must exercise continuing caution not to break the invariants as the code evolves.

It's also covered here under "make illegal states unrepresentable".

https://blog.janestreet.com/effective-ml-revisited/

The author is advocating OCaml, which allows you to encode such invariants in the type system.

But 99% of state machines are not written in OCaml -- more often it's C or C++. For example, the entire Linux kernel :-/ So basically I'm saying rather than poking and prodding at the code with the debugger and flipping variables, you can think of your whole program as a state machine, and the state machine has invariants like the ones listed:

last_ping_time and last_ping_id are intended to be used as part of a keep-alive protocol. Note that either both of those fields should be present, or neither of them should. Also, they should be present only when state is Connected.

----

For another example, the "lossless syntax tree" data structure I use for my shell has a number of invariants:

http://www.oilshell.org/blog/2017/02/11.html

I use the same data structure for generating errors and translating between two languages. So it really has to have some higher-level properties rather than just setting and getting variables.

In C# they have something called "Red-Green Trees" that are similar:

https://github.com/oilshell/oil/wiki/Lossless-Syntax-Tree-Pa...

In summary, your debugger is not going to show you which nodes are red and green in this sense! It has no idea about such things. IMO it's better to think at a higher level when you can.

(Note that red-green trees are just a name for an application-specific concept they made up. I t has nothing to do with red-black trees, which are a generic data structure. Even so, your debugger also has no idea about the invariants in a red-black tree implementation either!)

----

Maybe a simpler way to put it:

- the assert() statement in C and Python checks invariants at runtime. There's something of an art to doing this; I think it's covered in books like "Code Complete". You can grep a codebase for assert() and see what invariants the author wants to maintain.

- I'm not sure if they still teach "loop invariants" anymore, but it basically means "something that's true at every iteration of the loop", regardless of whether it's the first iteration, last one, in the middle, etc. If you have off-by-one errors and you stick in a -1 to fix it, that's a sign that you could think more rigorously about the loop.)

Invariant means "a thing that's always true, regardless of the values of the variables." It's a higher-level property than what the debugger shows you.

Survival of the fittest sure, but when the requirements to change are hard because of high technical accomplishment and a high standard vs grossly complex are two different things.

I like Linus and his willingness to say what he thinks but sometimes he really is just an asshole from the old guard.

https://www.bbc.com/news/technology-45664640