Yesterday I discovered PySnooper, which describes itself as "a poor man's debugger":
Your story: You're trying to figure out why your Python code isn't doing what you think it should be doing. You'd love to use a full-fledged debugger with breakpoints and watches, but you can't be bothered to set one up right now.
I know that guy! Especially when I'm debugging some Python code in a BitBake class or recipe and attaching a debugger is even more annoying than usual. I've previously written a tiny class to start a rpdb session as needed, but I don't get on with pdb for some reason.
The example makes it look pretty awesome for quick debugging:
Source path:... example.py
Starting var:.. number = 6
11:46:07.482187 call 4 def number_to_bits(number):
11:46:07.482561 line 5 if number:
11:46:07.482655 line 6 bits = []
New var:....... bits = []
11:46:07.482732 line 7 while number:
11:46:07.482830 line 8 number, remainder = divmod(number, 2)
Modified var:.. number = 3
New var:....... remainder = 0
11:46:07.482907 line 9 bits.insert(0, remainder)
Modified var:.. bits = [0]
11:46:07.483028 line 7 while number:
11:46:07.483130 line 8 number, remainder = divmod(number, 2)
Modified var:.. number = 1
Modified var:.. remainder = 1
11:46:07.483208 line 9 bits.insert(0, remainder)
Modified var:.. bits = [1, 0]
11:46:07.483323 line 7 while number:
11:46:07.483419 line 8 number, remainder = divmod(number, 2)
Modified var:.. number = 0
11:46:07.483497 line 9 bits.insert(0, remainder)
Modified var:.. bits = [1, 1, 0]
11:46:07.483593 line 7 while number:
11:46:07.483697 line 10 return bits
11:46:07.483773 return 10 return bits
Return value:.. [1, 1, 0]
Elapsed time: 00:00:00.001749
So here's my thirty second explainer on how to use PySnooper with BitBake. First, we need to install it:
$ pip3 install pysnooper
Then you can just import pysnooper and decorate functions to get them annotated at runtime:
import pysnooper
@pysnooper.snoop()
def some_function():
...
That's the theory, but anyone who has tried throwing print("here") messages into classes or recipes knows this doesn't work. They execute in a child process which doesn't have standard output connected to the console, but luckily the snoop function can instead write the messages to a filename or stream or callable, which lets us glue PySnooper to BitBake's logging:
import pysnooper
@pysnooper.snoop(bb.plain)
def some_function():
...
As a working example, I added the annotation to get_source_date_epoch() in meta/lib/oe/reproducible.py:
import pysnooper
@pysnooper.snoop(bb.plain)
def get_source_date_epoch(d, sourcedir):
return (
get_source_date_epoch_from_git(d, sourcedir) or
get_source_date_epoch_from_youngest_file(d, sourcedir) or
fixed_source_date_epoch(d)
)
And now when we start BitBake, we get to see the output:
Source path:... /home/ross/Yocto/poky/meta/lib/oe/reproducible.py
Starting var:.. d = <bb.data_smart.DataSmart object at 0xffff9e30dcf0>
Starting var:.. sourcedir = '/yocto/ross/build/tmp/work-shared/llvm-project-source-15.0.6-r0/git'
10:56:57.198016 call 156 def get_source_date_epoch(d, sourcedir):
10:56:57.199750 line 158 get_source_date_epoch_from_git(d, sourcedir) or
10:56:57.341387 line 157 return (
10:56:57.341978 return 157 return (
Return value:.. 1669716358
Elapsed time: 00:00:00.144763
Useful!
The default log depth is 1 so you don't see inside functions, but that can be changed when decorating You can also wrap smaller code blocks using with blocks.
The biggest catch is remembering that BitBake classes and recipes are not Python, they just have Python blocks in, so you can't decorate a function inside a class or recipe. In this case you'll need to use with block.
This looks like a very useful tool and I look forward to using it next time I'm tearing my increasingly greying hair out.
NP: Charcoal, Brambles