[Dwarf-Discuss] DWARF and source text embedding

[scott@scottlinder.com]

Hi Paul,

My intention was to support an empty source string; I want to be 
explicit about the presence of embedded source for each file.

When reading the spec I did notice places where an empty string can 
indicate the absence of the attribute (e.g. DW_AT_name), but I would 
prefer to be explicit here.

Scott

On 2018-02-01 11:19, paul.robinson at sony.com wrote:
>> -----Original Message-----
>> From: Dwarf-Discuss [mailto:dwarf-discuss-bounces at lists.dwarfstd.org] 
>> On
>> Behalf Of scott at scottlinder.com
>> Sent: Wednesday, January 31, 2018 2:05 PM
>> To: dwarf-discuss at lists.dwarfstd.org
>> Subject: [Dwarf-Discuss] DWARF and source text embedding
>> 
>> Hello all,
>> 
>> I am a compiler engineer at AMD, working on tools for debugging
>> online-compiled
>> programs. The problem I am attempting to solve was brought up 
>> previously
>> in the
>> DWARF Standard issue 161018.1 titled "DWARF-embedded source for
>> online-compiled
>> programs", and is the result of runtimes like OpenCL doing online
>> compilation
>> in an environment where it is not desireable (or even feasible) to 
>> write
>> sources to disk. In these cases, it would be useful to support 
>> embedding
>> the
>> source directly in the resulting DWARF. I would like to propose a
>> similar
>> solution to the one outlined in the above issue, but without 
>> structural
>> changes
>> to the specification.
>> 
>> ====
>> 
>> Add two new optional fields to the file_names prologue of the line
>> table.
>> 
>> Section 6.2.4.1:
>> Add two bullets after "5. DW_LNCT_MD5"
>> 6. DW_LNCT_has_source
>>      DW_LNCT_has_source indicates that the value is a boolean which
>> affects the
>>      interpretation of an accompanying DW_LNCT_source value. When 
>> present
>> there
>>      must be an accompanying DW_LNCT_source value. When true, 
>> consumers
>> may use
>>      the embedded source instead of attempting to discover the source 
>> on
>> disk.
>>      When false, consumers will ignore the DW_LNCT_source value. This
>> code point
>>      is always paired with a flag form (e.g. DW_FORM_flag or
>>      DW_FORM_flag_present).
>> 7. DW_LNCT_source
>>      DW_LNCT_source indicates that the value is a null-terminated 
>> string
>> which
>>      is the original source text of the file. When present there must 
>> be
>> an
>>      accompanying DW_LNCT_has_source value. The string will contain 
>> the
>> UTF-8
>>      encoded source text with '\n' line endings. When the accompanying
>>      DW_LNCT_has_source value is false, the value of DW_LNCT_source 
>> will
>> be the
>>      empty string. This code point is always paired with a string form
>> (e.g.
>>      DW_FORM_string, DW_FORM_line_strp, DW_FORM_strp).
> 
> Would a zero-length string indicate something other than 
> "has_source=false"?
> If not, then a separate has_source flag seems redundant.
> --paulr
> 
>> 
>> New type codes can be allocated for them in a backwards-compatible 
>> way,
>> or
>> codes for these new content types can be added in the range of
>> [DW_LNCT_lo_user, DW_LNCT_hi_user] to avoid changing the spec itself.
>> 
>> Table 7.27:
>> Add DW_LNCT_has_source  0x6
>> Add DW_LNCT_source      0x7
>> 
>> Any DWARFv5 consumer which is unaware of this extension would continue
>> to
>> operate as before, ignoring the new fields. Any consumer which is 
>> aware
>> of the
>> extension would know to check DW_LNCT_has_source for each file_name
>> entry in
>> order to determine whether the embedded source field (DW_LNCT_source)
>> contains
>> the source text of the corresponding file.
>> 
>> ====
>> 
>> My team and I believe this simplifies the design by removing the need
>> for
>> changes to the compile unit sections, and by avoiding the addition of
>> multiple
>> file_name_entry_formats in a single program, all without sacrificing 
>> any
>> information. We have a preliminary implementation in LLVM/Clang, which
>> supports
>> embedding source (clang -gdwarf-5 -gembed-source) and inspecting it 
>> via
>> llvm-dwarfdump and llvm-objdump (with the -source flag). The patches 
>> are
>> available at https://reviews.llvm.org/D42765 (LLVM) and
>> https://reviews.llvm.org/D42766 (Clang).
>> 
>> I would like any and all feedback on the design, and want to see about
>> the
>> possibility of adding the new content type codes outside of the "user"
>> range
>> (i.e. adding new entries for them in Table 7.27) in the next version 
>> of
>> the
>> specification.
>> 
>> Regards,
>> Scott Linder
>> 
>> _______________________________________________
>> Dwarf-Discuss mailing list
>> Dwarf-Discuss at lists.dwarfstd.org
>> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org