[Dwarf-Discuss] DWARF and source text embedding

[paul.robinson@sony.com]

> -----Original Message-----
> From: Dwarf-Discuss [mailto:dwarf-discuss-bounces at lists.dwarfstd.org] On
> Behalf Of scott at scottlinder.com
> Sent: Wednesday, January 31, 2018 2:05 PM
> To: dwarf-discuss at lists.dwarfstd.org
> Subject: [Dwarf-Discuss] DWARF and source text embedding
> 
> Hello all,
> 
> I am a compiler engineer at AMD, working on tools for debugging
> online-compiled
> programs. The problem I am attempting to solve was brought up previously
> in the
> DWARF Standard issue 161018.1 titled "DWARF-embedded source for
> online-compiled
> programs", and is the result of runtimes like OpenCL doing online
> compilation
> in an environment where it is not desireable (or even feasible) to write
> sources to disk. In these cases, it would be useful to support embedding
> the
> source directly in the resulting DWARF. I would like to propose a
> similar
> solution to the one outlined in the above issue, but without structural
> changes
> to the specification.
> 
> ====
> 
> Add two new optional fields to the file_names prologue of the line
> table.
> 
> Section 6.2.4.1:
> Add two bullets after "5. DW_LNCT_MD5"
> 6. DW_LNCT_has_source
>      DW_LNCT_has_source indicates that the value is a boolean which
> affects the
>      interpretation of an accompanying DW_LNCT_source value. When present
> there
>      must be an accompanying DW_LNCT_source value. When true, consumers
> may use
>      the embedded source instead of attempting to discover the source on
> disk.
>      When false, consumers will ignore the DW_LNCT_source value. This
> code point
>      is always paired with a flag form (e.g. DW_FORM_flag or
>      DW_FORM_flag_present).
> 7. DW_LNCT_source
>      DW_LNCT_source indicates that the value is a null-terminated string
> which
>      is the original source text of the file. When present there must be
> an
>      accompanying DW_LNCT_has_source value. The string will contain the
> UTF-8
>      encoded source text with '\n' line endings. When the accompanying
>      DW_LNCT_has_source value is false, the value of DW_LNCT_source will
> be the
>      empty string. This code point is always paired with a string form
> (e.g.
>      DW_FORM_string, DW_FORM_line_strp, DW_FORM_strp).

Would a zero-length string indicate something other than "has_source=false"?
If not, then a separate has_source flag seems redundant.
--paulr

> 
> New type codes can be allocated for them in a backwards-compatible way,
> or
> codes for these new content types can be added in the range of
> [DW_LNCT_lo_user, DW_LNCT_hi_user] to avoid changing the spec itself.
> 
> Table 7.27:
> Add DW_LNCT_has_source  0x6
> Add DW_LNCT_source      0x7
> 
> Any DWARFv5 consumer which is unaware of this extension would continue
> to
> operate as before, ignoring the new fields. Any consumer which is aware
> of the
> extension would know to check DW_LNCT_has_source for each file_name
> entry in
> order to determine whether the embedded source field (DW_LNCT_source)
> contains
> the source text of the corresponding file.
> 
> ====
> 
> My team and I believe this simplifies the design by removing the need
> for
> changes to the compile unit sections, and by avoiding the addition of
> multiple
> file_name_entry_formats in a single program, all without sacrificing any
> information. We have a preliminary implementation in LLVM/Clang, which
> supports
> embedding source (clang -gdwarf-5 -gembed-source) and inspecting it via
> llvm-dwarfdump and llvm-objdump (with the -source flag). The patches are
> available at https://reviews.llvm.org/D42765 (LLVM) and
> https://reviews.llvm.org/D42766 (Clang).
> 
> I would like any and all feedback on the design, and want to see about
> the
> possibility of adding the new content type codes outside of the "user"
> range
> (i.e. adding new entries for them in Table 7.27) in the next version of
> the
> specification.
> 
> Regards,
> Scott Linder
> 
> _______________________________________________
> Dwarf-Discuss mailing list
> Dwarf-Discuss at lists.dwarfstd.org
> http://lists.dwarfstd.org/listinfo.cgi/dwarf-discuss-dwarfstd.org