"Debugging is like being the detective in a crime movie where you are also the murderer."
Filipe Fortes
We all know that debugging takes time. However, there are a few tricks to write our code in such a way, which will save us time debugging. This article will showcase 7 of those techniques. While the examples are in Python, most of the things mentioned here will apply to any programming language. The techniques are ordered by level of complexity and also scope - from changes you can make in your code instantly, to more architectural patterns that help with debugging (and not only).
Separating expressions in different variables
The simplest thing to do to make our code more debugable is to separate different expressions and/or actions into separate variables. For example, we can have the following code:
def calculate_based_on_user_input():
user_input = parse_input(read_input())
return calculate(user_input.number)
Let's imagine we have traced the bug to this function, called calculate_based_on_user_input. What it does, is read some input from the user, parses it, take a part of it (the attribute number), and then returns the result of some calculations on the specific part of the input. Where can the bug be? Is it in the reading of the input or it's parsing? Perhaps the bug is in the calculate function? Could it be in the attribute itself (in Python, attributes can be properties, which can have certain behaviors attached to them - just like functions/methods)?
Assuming we don't have a test to verify this quickly, our next best option is to bust out the debugger and start looking. Or just put print statements here and there. Now, the issue comes that we do multiple things on a single line - even with the debugger on, it's hard to pinpoint which action is the culprit exactly.
I'm not suggesting that this code is impossible to debug - on the contrary, it's debuggable, with the right tools. However, the time spent debugging can be reduced, if we separate each expression (or in a more general sense, each action) into separate lines and separate variables.
Looking at the code again, we have 4 distinct actions - read_input(), parse_input(), user_input.number, and calculate. If we separate each action into a new variable, our code would look something like this:
def calculate_based_on_user_input():
raw_user_input = read_input()
user_input = parse_input(raw_user_input)
user_number = user_input.number
result = calculate(user_number)
return result
We put the results of each action into a separate variable, with a good-sounding name (more on naming in just a bit). Now, we can attach a debugger or just use print statements to figure out where the bug is a lot quicker. Another benefit of this is that our code becomes a bit more understandable, even if we are just reading, not debugging it. A clear flow can be seen. In the words of Robert C. Martin: "We would like a source file to be like a newspaper article". Or like a recipe, for pie.
Clear naming
Another important part of the debugable code is having clear naming of our variables, functions, and classes. But how does having good and clear naming affect debugging?
Let's look at this piece of code, which has a few bugs in it:
def prs(a):
v = ['+', '-', '*', '/']
if a not in v:
return None
return a
a = input()
b = prs(a)
c = input()
d = input()
if a is None:
print("Invalid operation")
sys.exit(0)
if b == '+':
r = c + d
elif b == '-':
r = c - d
elif c == '*':
r = c * d
elif c == '/':
if d == 0:
print("Cannot divide by zero")
sys.exit(0)
r = c / d
print(r)
Did you manage to find the bugs? It's a bit hard, isn't it? Let's change the naming and try again:
def parse_operation(raw_operation):
valid_operations = ['+', '-', '*', '/']
if raw_operation not in valid_operations:
return None
return user_operation
raw_operation = input()
operation = parse_operation(raw_operation)
raw_first_number = input()
raw_second_number = input()
if raw_operation is None:
print("Invalid operation")
sys.exit(0)
if operation == '+':
result = raw_first_number + raw_second_number
elif operation == '-':
result = raw_first_number - raw_second_number
elif raw_first_number == '*':
result = raw_first_number * raw_second_number
elif raw_first_number == '/':
if raw_second_number == 0:
print("Cannot divide by zero")
sys.exit(0)
result = raw_first_number / raw_second_number
print(result)
The only change is the naming, nothing more, nothing less. Now it's much more obvious that this code is not working. There are three big issues:
The first big issue is the if raw_operation is None. Here, we check raw_operation, which contains the raw user input. This means, that whatever the user enters, it will always be a string. So raw_operation can't be None. What we should do, is to check if operation is None.
The next bug is that we don't parse the user input into an integer. The input function returns a string, so we will try to apply mathematical operations to strings, which will not work (+ will just concatenate the two strings).
The final big bug is in the big if-else - the first two if-blocks check the operation, while the third and the fourth, check the raw first number.
If we were to fix the code, it would look something like this:
def parse_operation(raw_operation):
valid_operations = ['+', '-', '*', '/']
if raw_operation not in valid_operations:
return None
return user_operation
raw_operation = input()
operation = parse_operation(raw_operation)
raw_first_number = input()
try:
first_number = int(raw_first_number)
except ValueError as exc:
print(exc)
raw_second_number = input()
try:
second_number = int(raw_second_number)
except ValueError as exc:
print(exc)
if operation is None:
print("Invalid operation")
sys.exit(0)
if operation == '+':
result = first_number + second_number
elif operation == '-':
result = first_number - second_number
elif operation == '*':
result = first_number * second_number
elif operation == '/':
if second_number == 0:
print("Cannot divide by zero")
sys.exit(0)
result = first_number / second_number
print(result)
As you can see, the better the naming, the easier it is to spot the mistakes in the code. As a rule of thumb, names should describe what the variable does. For more tips on naming, you can check out this article here.
Comments and docstrings
Sometimes reading plain English can help with debugging. While comments and docstrings are not technically code, they can also help with debugging. The main idea of comments and docstrings is to document why the code is written like it is - be it the technical decisions, user requirements, or any other limitations imposed on the code. Comments can also be used to explain ideas that are hard to convey by just reading the code.
One of the more important tips and tricks I learned when I was just starting as a junior developer was to leave comments explaining on a high level what a block of code does. Sometimes our code will be long. Sometimes our code will be complex. Sometimes our naming won't be enough to convey the ideas we need it to. In those cases, I prefer to leave out a comment, containing a sentence or two, about what is going on in the code and which part of the code are we looking at. Think of it as anchors that we can refer to later on. For example, if we have a long function where we put the main logic flow of our program, putting a few comments specifying stuff like Here we do this step or This is where the data is gathered can help with figuring out where to look. If we know that our tool broke during a certain part, having comments pinpointing what happens where can save us time.
Logging
Our code will most likely not run on machines that we have direct access to. Or if we do have access, we can't keep the software off until we fix the bug. In both cases, we need some trace of what happened. Exception stack traces are a good indication of what went wrong, but it can be hard to understand what exactly is going on. Also, unhandled exceptions, are not a good idea.
Ideally, our logs should contain all the required information for us to debug an application - what steps were executed, any warnings, errors, debug messages, exceptions caught, and anything else that is deemed relevant to the execution of our application.
By structuring our logs, we can quickly see where in the program flow our bug was hit. We can quickly glance if it's in the beginning or the end, or after which operation. From there we can pinpoint the module or piece of code that causes the issue. We can have different levels of logging in our logs - we can show informational messages, warning messages, error messages or debug information. Each of those can carry different kinds of information - if we are sure that our application threw an error, we can look for "ERROR" messages in the log. If something weird happens, we can also look for "warning" messages. We can even put the state of our objects inside the log, for future reference. For example, I put the command line arguments at the beginning of the log file, as well as logging any state-keeping objects. This allows me to quickly reproduce situations in which bugs occur.
When writing Python applications, I use the built-in logging module. For more information, you can check out here. If you are interested in the topic of logging in Python, drop me a line in the comments!
Single-responsibility principle
As a continuation of the "separating expressions in different variables", it's key that we follow the single-responsibility principle when writing our code.
If you aren't familiar with the single-responsibility principle, here is a quick refresher - "A class should have only one reason to change". Now, this may sound very abstract at first, so allow me to explain it a bit. Imagine we have the following code:
from typing import Optional
def calculate() -> Optional:
try:
num1 = int(input('Please enter first number'))
except ValueError as exc:
print(exc)
return None
try:
num2 = int(input('Please enter second number'))
except ValueError as exc:
print(exc)
return None
operation = input('Please enter operation (+, -, *, /)')
if operation not in ['+', '-', '*', '/']:
print("Operation not supported")
return None
if operation == '+':
result = num1 + num2
elif operation == '-':
result = num1 - num2
elif operation == '*':
result = num1 * num2
elif operation == '/':
result = num1 / num2
return result
While this code is perfectly valid and working, it goes against the single responsibility principle. It has multiple reasons to change - either the input logic changes, the way we do the input handling, or the way we do the actual arithmetical operations.
But what exactly makes this code hard to debug? The main reason is the fact we have a lot of logic that could cause bugs - be it reading the input, parsing it, or the logic for the operations itself. Actually, from a user perspective, there is a big bug that could cause issues. Can you spot it?
Luckily for us, Python will be helpful when we hit the bug - if we try to divide by zero, Python will raise a ZeroDivisionError exception. In other languages, however, we could be facing a weird exception that isn't connected to the division error. If we have to trace the bug in the code, we will have to go through a lot of working code, before we come to the problematic lines. The more reasons for change we have in our function, the higher the complexity of the code, and the higher the complexity goes, the harder it is to debug.
If we separate our code into pieces that do only one thing (or have a single reason to change), we can quickly read through the code we know works and go digging in the place we know the code fails. Let's refactor our code a bit:
from typing import Optional
POSSIBLE_OPERATIONS = ['+', '-', '*', '/']
OPERATION_TO_LAMBDA = {
'+': lambda a, b: a + b,
'-': lambda a, b: a - b,
'*': lambda a, b: a * b,
'/': lambda a, b: a / b
}
def read_user_number(message: str ='') -> Optional[int]:
user_input = input(message)
try:
result = int(user_input)
except ValueError as exc:
# log the exception here
result = None
return result
def is_operation_valid(operation: str) -> bool:
return operation in POSSIBLE_OPERATIONS
def apply_operation(a: int, b: int, operation: str) -> int:
operation_function = OPERATION_TO_LAMBDA[operation]
result = operation_function(a, b)
return result
def calculate() -> Optional[int]:
first_number = read_user_number(message='Please enter first number')
if first_number is None:
return None
second_number = read_user_number(message='Please enter second number')
if second_number is None:
return None
operation = input('Please enter operation (+, -, *, /)')
if is_operation_valid(operation):
print("Operation not supported")
return None
result = apply_operation(first_number, second_number, operation)
return result
Looking at the refactored code, let's examine the "reasons to change" in the code. read_user_number can change only in the way we parse the data, is_operation_valid can change in the way we check if the operation is valid, apply_operation can change if we change the way we apply the operation (as a note - you can see I changed the way the operation is applied. Instead of an if-else, I use a structure that maps the string for the operation to a function) and calculate can change when the error handling is changed or the output handling is changed.
Now, we can quickly look at the functions one-by-one, and determine a possible bug - what happens if the operation throws an exception (the ZeroDivisionError is still not handled)? We can pinpoint the error much quicker than before.
Of course, the single-responsibility principle can help with a lot of things, however, this is outside the scope of this article. (If you want to see an article regarding single-responsibility or other SOLID principles, drop me a line in the comments)
Dependency inversion and dependency injection principles
Two solid principles (pun intended), are the dependency inversion and dependency injections principles. For those of you who haven't heard about those principles, here is another quick refresher:
According to Wikipedia:
Dependency inversion:
High-level modules should not import anything from low-level modules. Both should depend on abstractions (e.g., interfaces).
Abstractions should not depend on details. Details (concrete implementations) should depend on abstractions.
Dependency injection is a programming technique in which an object or function receives other objects or functions that it requires, as opposed to creating them internally.
Now, what does this mean in practice and how can it help our code to be more debugable? Before we dive into the code, let's jump up into the architectural view of software.
Sometimes, we have different levels of modules - a great example is a note-taking app. Let's create a component diagram for such an app. If we are to design such an app, on a component level we would have 3 main components - the notes themselves (everything including creation, editing, and the data they will keep), the storage (how are we going to store the notes themselves - be it a database, or in a simple JSON file) and the user interface (the way we interact with the application/notes). In our case, the note component will be the higher-level component, while the storage and user interface components will be lower-level (this is determined by the level of abstraction. The notes should not care about the specifics of how they are stored).
The dependency-inversion principle states that high-level modules should not import directly low-level modules. So we can't directly import Storage into Note. The way we can solve this is to introduce two new interfaces - IStorage and IUserInterface. Our Note module will import those interfaces, while Storage and User Interface will implement them. This allows us a lot more freedom when changing the implementation of Storage and User Interface - as long as the interfaces stay the same, we can swap out the concrete implementation a lot easier.
Now, let's go back to coding. Continuing our notes app example, let's examine a simple function from the Storage module - load_notes() which will return a list, containing all of our notes (as Note objects). The function accepts the path to the storage file (let's say that we store the notes in a JSON file). The function will open the file, read it, and pass the contents to a NoteSerializer class, which will create the new Note instances.
def read_notes(path: str) -> list[Note]:
serializer = NoteSerializer()
try:
with open(path, 'r') as file_pointer:
content = load(file_pointer)
except (IOError, JSONDecodeError) as exc:
return []
notes = [serializer.from_json(serialized_note) for serialized_note in content]
return notes
This function looks good, but there is a small problem - what if we want to debug it and run only the function? We don't want to start the whole application and read our real notes. We just want to provide a simple structure (be it plain text) in a file, and see if it works. For that, we will need a special serializer, which we cannot have, as we initialize NoteSerializer inside our code. Basically, read_notes can't function without NoteSerializer, which means that we can't be sure if the bug is within read_notes or NoteSerializer. We can't isolate one without the other.
Here comes dependency injection to save the day! The pattern states that our objects or functions must accept any external modules as arguments, instead of initializing them inside the object/function. This can achieved in two ways - either accept an already initialized NoteSerializer object or we can specify a function pointer to the constructor (or __init__ method of the class). If we have followed our dependency inversion pattern, then our read_notes would only import an interface for NoteSerializer, which means we can have our own SimpleNoteSerializer class, which implements the interface for NoteSerializer.
Here is an example using both approaches:
Accepting the serializer as an already-initialized object:
def read_notes(path: str, serializer: NoteSerializer) -> list[Note]:
try:
with open(path, 'r') as file_pointer:
content = load(file_pointer)
except (IOError, JSONDecodeError) as exc:
return []
notes = [serializer.from_json(serialized_note) for serialized_note in content]
return notes
Accepting a function pointer to the constructor:
def read_notes(path: str, serializer_constructor: Callable[None, NoteSerializer]) -> list[Note]:
serializer = serializer_constructor()
try:
with open(path, 'r') as file_pointer:
content = load(file_pointer)
except (IOError, JSONDecodeError) as exc:
return []
notes = [serializer.from_json(serialized_note) for serialized_note in content]
return notes
If we apply those two patterns in our code, we will be able to interact and debug with our module in a stand-alone matter. The main benefit boils down to: "Less code to debug, less time spent debugging"!
Unit tests and TDD
All of the things above are valid and can help with debugging, but nothing beats having tests. Having a good test coverage can massively reduce the amount of bugs that are missed during development.
When writing new code from scratch, after I make my architecture (or at least the initial version of it), I dive into the actual logic. I used to write some logic, then spend almost the same amount of time debugging it, because I missed something. This was very time-consuming, and to be perfectly honest, it was a bit disheartening to see that I wasted a lot of time on something that didn't work. Although I don't expect that my code will work from the first try, it's still a bad feeling when your code doesn't work.
At some point, I learned more about tests and test-driven development (TDD, for short). At first, I was skeptical about it - how can you write tests, if you don't know what the code will be? And even if you do write the tests first, how does that help you? As with everything, the answers to these questions came with time and experience.
Let's take a step back - and think about software development for a bit. In most cases, we write software to do something. We might not know from the get-go how our software will do the thing, but we know that it must do the thing. So if we know what will the software do, we can think about how the software will act in certain situations. If we are making a simple calculator, we know that dividing by zero should throw an error, multiplying something by zero should return zero, or that adding zero to something does not change the original input. This means, that we can create tests way before we have written our calculator because we know what it must do - if you divide by zero, you should receive an error. How does this help development then? It's simple - you will know quicker if your code works or not and if it's working correctly, as expected.
Even if we don't write tests first, we can still benefit from having them as early as possible. When I write new code or refactor existing one, I heavily rely on tests (mainly unit tests). I don't need to rely on myself to check if the logic I wrote is working or not. I can spend some time figuring out how should my module behave and writing the tests. With the tests available, I can quickly check if my logic is correct or not. Or when I'm refactoring the code, I can quickly check if I have broken something or not. I have even taken this to the extreme in my personal projects - first I clear what is expected from the software as a whole (for example, I want to generate UML class diagrams from Python code automatically), then I make the initial version of the architecture (which module will contain which logic and how will the separate pieces will fit together). After that, I write the initial version of the logic itself, module by module. I don't spend too much time thinking "Will this work ? Do I handle all cases ?", which of course could result in me missing some stuff in the code. After I have some version of the code, I focus on the unit tests for the module. From there, I can figure out if the code I wrote works properly or not. If some tests fail, it's a lot easier to fix the code, because I know when and why it doesn't work. From there, it's just a matter of writing more code. No more time spent figuring out "why does this not work"!
Conclusion
There are many ways to improve our code to make it easier to debug. As with most things, it's best to take preventive measures - cleaner code, better architecture, and more tests are always better than spending hours debugging code. However, I hope that these 7 tricks can help you catch that nasty bug a bit quicker next time.
Thank you for reading until the end - as always, feedback is welcome in the comments. Share this article if it is useful, and stay tuned for more!