Invoice Parsing Bugs in CLN

Several invoice parsing bugs were fixed in CLN 23.11, including bugs that caused crashes, undefined behavior, and use of uninitialized memory. These bugs could be reliably triggered by specially crafted invoices, enabling a malicious counterparty to crash the victim’s node upon invoice payment.

The parsing bugs were discovered by a new fuzz test written by Niklas Gögge and enhanced by me.

Bugs fixed in v23.11

The fuzz target

The fuzz target that uncovered these bugs was initially written by Niklas Gögge in December 2022, though it wasn’t made public until October 2023. The target simply provides fuzzer-generated inputs to CLN’s invoice decoding function, similar to fuzz targets written for other implementations [1, 2].

To improve the fuzzer’s efficiency, Niklas also wrote a custom mutator for the target. Invoices are encoded in bech32 which requires a valid checksum at the end of the encoding, making it quite difficult for fuzzers to generate valid bech32 consistently. As a result, bech32-naive fuzzers will generally get stuck at the bech32 decoding stage and have a hard time exploring deeper into the invoice parsing logic. Niklas’ custom mutator teaches the fuzzer how to generate valid bech32 so that it can focus its fuzzing on invoice parsing.

Initial fuzzing in 2022

After writing the fuzz target in December 2022, Niklas privately reported several bugs to CLN including a stack buffer overflow, an assertion failure, and undefined behavior due to a 0-length array. Many of the bugs were fixed in PR 5891 and released in CLN 23.02.

Merging the fuzz target in 2023

In October 2023, Niklas submitted his fuzz target for review in PR 6750. The initial corpus in that PR actually triggered bugs 1 and 2, but Niklas didn’t notice because he had been fuzzing with some UBSan options misconfigured. CLN’s CI didn’t detect the bugs either, since UBSan had previously been accidentally disabled in CI.

Niklas also discovered bug 3 during initial fuzzing, but he initially thought it was a false report and hard-coded an exception for it in the fuzz target.

Enhancements

The initial fuzz target only fuzzed the invoice decoding logic, skipping signature checks. I modified the target to also run the signature-checking logic, which enabled the fuzzer to quickly find bug 4.

While bug 5 should have also been discoverable by the fuzzer after this change, it remained undetected even after many weeks of CPU time. It wasn’t until I added a custom cross-over mutator for the fuzz target that bug 5 was discovered. The cross-over mutator is based on Niklas’ custom mutator and simply combines pieces from multiple bech32-decoded invoices before re-encoding the result in bech32. Within a few CPU hours of fuzzing with this extra mutator, the fuzzer found bug 5.

Impact

The severity of these bugs seems relatively low since they can only be triggered when paying an invoice. If a malicious invoice causes your node to crash, as long as you can restart your node in a timely manner and avoid paying any more invoices from the malicious counterparty, no further harm can be done.

Since bug 2 involves uninitialized memory it could potentially be more serious, as a sophisticated attacker may be able to extract sensitive data from the invoice-decoding process. Such an attack would be quite complex, and it is unclear whether it would even be possible in practice. It’s also unclear exactly what sensitive data could be extracted, since CLN handles private keys in a separate dedicated process (the hsmd daemon).

Takeaways

• Fuzz testing is an essential component of writing robust and secure software. Any API that consumes untrusted inputs should be fuzz tested.
• Custom mutators can be very powerful for fuzzing deeper logic in the codebase.
• Fuzz testing of C or C++ code should use both ASan and UBSan. MSan and valgrind can also be useful.