<div dir="ltr"><a class="gmail_plusreply" id="plusReplyChip-1" href="mailto:ayermolovich@nvidia.com" tabindex="-1">+ayermolovich@nvidia.com</a> as well as having a standardized version of these sounds great and he's up for helping as well. :)<div><br></div><div>And yes, thanks for following up on this Zequan! :)<br><div><br></div><div>-eric</div></div></div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Sat, Oct 31, 2026 at 2:27 AM Suprateeka R Hegde via Dwarf-discuss <<a href="mailto:dwarf-discuss@lists.dwarfstd.org">dwarf-discuss@lists.dwarfstd.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<font face="NVIDIA Sans">NVIDIA has been using this in production
for years now. Thanks for some of these numbers.<br>
<br>
--<br>
Supra<br>
</font><br>
<div>On 31-Oct-2026 04:40 am, Zequan Wu via
Dwarf-discuss wrote:<br>
</div>
<blockquote type="cite">
<table border="1">
<tbody>
<tr>
<td><font face="verdana" size="1"><b>External email: Use
caution opening links or attachments</b>
</font></td>
</tr>
</tbody>
</table>
<br>
<div>
<div dir="ltr">Hi Cary,<br>
<br>
I've prototyped two-level line tables in LLVM, following issue
140906.1 [1] and the wiki page [2], and I'd like to share some
data and ask whether the proposal could be revisited.<br>
<br>
Motivation: today, when the optimizer merges instructions from
different source locations, the merged instruction gets line
0. Sample-based profiling (AutoFDO) then loses the samples for
those source locations. What we want is for one instruction to
map to several source locations, each with its own inline
stack, and to get all of that from the line table alone.
Two-level line tables provide exactly this. At the LLVM Dev
Meeting, Eric Christopher mentioned that some concerns were
raised when this proposal was discussed before. Cary, could
you share what they were? I'd like to know whether they still
apply, and what a revised proposal would need to address.<br>
<br>
To show what the prototype emits, here's a small demo:<br>
```<br>
$ cat -n demo.c<br>
1 void foo(int);<br>
2<br>
3 static inline void work(int c) {<br>
4 if (c)<br>
5 foo(1);<br>
6 else<br>
7 foo(1);<br>
8 }<br>
9<br>
10 void caller(int c) {<br>
11 work(c);<br>
12 }<br>
<br>
$ llvm-dwarfdump --debug-line demo.o<br>
...<br>
file_names[ 0]:<br>
name: "demo.c"<br>
dir_index: 0<br>
md5_checksum: 20924601724a32ef530cc082992f3600<br>
subprograms[ 1]:<br>
name: "caller"<br>
decl_file: 0<br>
decl_line: 10<br>
subprograms[ 2]:<br>
name: "work"<br>
decl_file: 0<br>
decl_line: 3<br>
actuals_table_offset: 0x00000017<br>
<br>
Logicals Table:<br>
Index Line Column File Discriminator Context Subprogram
Flags<br>
------ ------ ------ ------ ------------- -------
------------------ -------------<br>
1 11 3 0 0 1 (caller)<br>
2 5 5 0 0 1 2 (work)
is_stmt prologue_end<br>
3 7 5 0 0 1 2 (work)
is_stmt prologue_end<br>
<br>
Actuals Table:<br>
Address OpIndex Log ISA Flags<br>
------------------ ------- ------ --- -------------<br>
0x0000000000000000 0 2 0<br>
0x0000000000000000 0 3 0<br>
0x000000000000000a 0 3 0 end_sequence<br>
```<br>
Debug info size on clang itself<br>
<br>
I built upstream clang (llvm-project c1d907e151c9, X86 target
only, RelWithDebInfo, -O2, DWARF 5) twice with the prototype
compiler: once with plain DWARF 5 (merged locations get line
0) and once with -gdwarf-two-level-line-tables. Nothing else
differs. After removing the debug sections, the two binaries
are byte-identical. Sizes below are uncompressed.<br>
<br>
With -g: file size 1,686.1 MB -> 2,010.6 MB (+19.2%)
.debug_line 89.0 MB -> 242.4 MB (2.72x) .debug_line_str 0.3
MB -> 160.5 MB all .debug_* 1,511.3 MB -> 1,835.8 MB
(+21.5%)<br>
With -gmlt: file size 500.0 MB -> 670.8 MB (+34.2%)
.debug_line 88.0 MB -> 219.8 MB (2.50x) .debug_line_str 0.3
MB -> 35.4 MB all .debug_* 325.3 MB -> 496.1 MB (+52.5%)<br>
Inside the two-level .debug_line, the logicals tables take 83%
of the bytes with both -g and -gmlt (201.4 MB and 182.3 MB),
the actuals tables 10-11% (24.5 MB in both), and the headers,
i.e. file tables plus subprogram lists, 6-7% (16.5 MB and 13.0
MB).<br>
<br>
Function names are a large part of the growth. To rebuild the
inline stack from .debug_line/.debug_line_str alone, the
subprogram list stores each function's name and declaration
line. Those names repeat what is already in .symtab and
.debug_info. Counting the strings in .debug_line_str plus the
subprogram-list entries in the .debug_line headers, names
account for about 54% of the total growth with -g (175 MB of
324 MB; about 929K unique mangled linkage names, 170 bytes on
average). With -gmlt they account for about 26% (45 MB of 171
MB). At -gmlt, clang omits linkage names unless
-fdebug-info-for-profiling is used (as AutoFDO builds do), so
in this build the list stores short names.<br>
<br>
On the other hand, with -gmlt the line table alone now carries
everything a profiler or symbolizer needs: all source
locations of an instruction and their inline stacks. A build
could therefore keep only .debug_line and .debug_line_str. In
this experiment those two sections total 255.2 MB, less than
the 325.3 MB of all debug sections in today's -gmlt build. In
that setup the names in .debug_line_str are no longer
duplicates of .debug_info. (Consumers would have to build
their own address index, e.g. by parsing all of .debug_line
and recording the address range of each sequence. Today they
reach the line table for an address through the compile unit
in .debug_info or .debug_aranges. My prototype doesn't do this
yet.)<br>
<br>
The other DWARF sections also grow slightly (.debug_info
+0.7%, .debug_loclists +3%, .debug_rnglists +7%). This is
because the two-level build has 157,710 more
DW_TAG_inlined_subroutine entries (8,745,925 -> 8,903,635,
+1.8%). With line 0, a merged instruction moves to the common
ancestor scope, so an inlined call whose instructions were all
merged disappears from .debug_info. With two-level tables the
instruction keeps its original scope, so those inlined calls
are recorded.<br>
<br>
With zstd-compressed debug sections (-gz=zstd), the -g file
grows from 653.3 MB to 758.3 MB (+16.1%): .debug_line 27.0
-> 106.1 MB and .debug_line_str 0.05 -> 20.9 MB.<br>
<br>
Since the original proposal is from 2014, I think the encoding
could also be made more compact. In a sample of 39 compile
units from the clang build above, 40% of logical rows are
call-site rows, which exist only to be referenced as a Context
and which no address refers to. DW_LNS_inlined_call, which
sets Context and Subprogram, precedes 73% of logical rows and
accounts for 37% of the logicals table's bytes. My prototype
writes the call-site rows first and then the other rows in the
order of their first use, which mostly follows addresses. Code
from different inline frames is interleaved in the machine
code, so the (Context, Subprogram) pair changes on most rows.<br>
<br>
The proposal doesn't require any particular order of logical
rows, because actuals rows and Context refer to them by
number. So, as an experiment, I grouped the logical rows of
the same inline frame together, with no format change. This
shrinks .debug_line by about 12%: <br>
<br>
-g: .debug_line 242.4 MB -> 213.1 MB (-12.1%) logicals
201.4 -> 152.9 MB, actuals 24.5 -> 43.8 MB -gmlt:
.debug_line 219.8 MB -> 194.4 MB (-11.6%)<br>
<br>
The actuals tables grow because the row numbers they refer to
no longer mostly increase by one. Compared with plain DWARF 5,
.debug_line is then 2.40x with -g (file +17.5%) and 2.21x with
-gmlt (file +29.1%).<br>
<br>
I'm happy to share the prototype, the measurements, and my
scripts, and to help with a revised proposal.<br>
<br>
Thanks,<br>
Zequan<br>
<br>
[1] <a href="https://dwarfstd.org/issues/140906.1.html" target="_blank">https://dwarfstd.org/issues/140906.1.html</a><br>
[2] <a href="https://wiki.dwarfstd.org/TwoLevelLineTables.md" target="_blank">https://wiki.dwarfstd.org/TwoLevelLineTables.md</a></div>
</div>
<br>
<fieldset></fieldset>
</blockquote>
<br>
</div>
-- <br>
Dwarf-discuss mailing list<br>
<a href="mailto:Dwarf-discuss@lists.dwarfstd.org" target="_blank">Dwarf-discuss@lists.dwarfstd.org</a><br>
<a href="https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss" rel="noreferrer" target="_blank">https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss</a><br>
</blockquote></div>