[Dwarf-Discuss] DWP mixed (DWARFv4/pre-standard + DWARFv5) content

[David Blaikie]

On Wed, Feb 26, 2020 at 1:05 AM Pavel Labath <labath at google.com> wrote:

> On Tue, 25 Feb 2020 at 21:36, David Blaikie <dblaikie at gmail.com> wrote:
> >
> >
> >
> > On Tue, Feb 25, 2020 at 11:41 AM Pavel Labath <labath at google.com> wrote:
> >>
> >> Yeah, this sounds tricky, but it is actually good timing, because I
> >> was just about to start working on DWP v5 in lldb. I was hoping that
> >> would be an easy ride, but it looks like things will get complicated.
> >>
> >> On Tue, 25 Feb 2020 at 18:53, David Blaikie <dblaikie at gmail.com> wrote:
> >> >
> >> > (please add anyone who has a vested interest in Split DWARF in
> general and dwp in particular)
> >> >
> >> > tl;dr:
> >> > How should DWARFv4 and DWARFv5 coexist in a DWP file:
> >> > 1) Not at all (invalid/unsupported)
> >> > 2) Single index table where the section indexes are subjective (look
> at the version of the referenced CU/TU to decide what a column number means
> (eg: 8 is DW_SECT_MACRO in v4, DW_SECT_RNGLISTS in v5)
> >> > 3) Emit multiple indexes (cu_index and tu_index each would contain
> two indexes) - a v4 (well, index version 2) and a v5 (index version 5)
> index.
> >>
> >> What about option 4)? :P Have the numbers be interpreted relative to
> >> the debug_cu_index version number?
> >> The idea would be that a DWARF v4 (dwp tool seems to use version
> >> number 2 for some reason)
> >
> >
> > Yeah - the version 2 comes from the pre-standard development iterations
> ( https://gcc.gnu.org/wiki/DebugFissionDWP documents version 2/mentions
> the existence of version 1)
> >
> >>
> >> debug_cu_index would keep using the existing
> >> pre-standard scheme, and it would not be able to refer to v5 units. In
> >> a v5 debug_cu_index (*), we would use the official DWARF v5 scheme,
> >> *but* we would, as an extension, support referring to the pre-standard
> >> sections via custom DW_SECT constants (with some high numbers to avoid
> >> conflicting with future standard versions).
> >
> >
> > Yeah, extending the v5 format might be a good idea. I'm open to that.
> >
> > Pity that there wasn't a standard extension space in the DWP spec to
> make that smoother. The existence of macinfo and macro in the pre-standard
> version perhaps could've been a hint that non-standard extensions might be
> needed (debug_macro being a non-standard extension itself).
>
> Yes, the lack of an official extension space is unfortunate, but I
> don't think this needs to be a blocker -- the spec also doesn't
> include an DW_FORM extension space, but that hasn't stopped people
> from inventing new forms. I would say that the situation is much
> better here actually -- it's quite easy for a consumer to ignore an
> unknown DW_SECT constant, whereas an unknown DW_FORM pretty much means
> you need to throw away the whole unit.
>
> Thinking about this further, the lack of a DW_FORM extension space is
> probably a reason for not introducing a DW_SECT extension space --
> it's hard to imagine how someone could meaningfully add a new section
> without a new form to refer to it.
>

Yeah, not many come to mind, though macro seems like the perfect example.
Had debug_macinfo made through to DWARFv5, GNU might've still wanted the
debug_macro extension & being able to define a new column would've worked
fine without the need for new forms (GNU extension debug_macro support used
an extension attribute (DW_AT_GNU_macros) but an existing form
(DW_FORM_sec_offset) - and the things in macro aren't referred to otherwise
(unlike loclist and rnglist) and even if tehy did could probably use
existing forms but extension attributes, etc).


> The main question on my mind now is, what is the likely future
> direction of the DWARF spec -- if say DWARF v6 adds a new section, how
> will it handle mixed v5+v6 debug_cu_indexes? I don't think it will
> want to make that unsupported (1). I also doubt it will resort to
> "subjective" DW_SECT constants, as it is easy to ensure the official
> DW_SECT constants do not overlap.
>

Fair points.


> The option (3) sounds plausible --  I do appreciate the cleanliness of
> having a v5 index containing v5 units only, and follow it with a v6
> index with v6 units. But I am not sure if that's the direction I would
> take, because I also very much like the idea of having just a single
> index table to access all units.
>

Yep - unless someone has significant objections my plan is currently:

Emit a v5 index with extension/non-standard extra column indexes
(specifically: DW_SECT_LOC and 9 and DW_SECT_MACINFO at 10). I hope those
can at least be reserved (like DW_SECT value 2 (originally DW_SECT_TYPES)
was in DWARFv5) in DWARFv6. & maybe open up an extension space in the
future.

I could use an unused index version for this feature, but that seems
probably unnecessary/excessive? (prestandard experiments used version 1 and
2, standard uses version 5 - so without treading on version 6, I could go
back and use version 3 or 4)

Open to ideas - but this is my plan for gold dwp & from there have
coworkers working on llvm-dwp and gdb support. So this choice will
hopefully go into those projects as well - if you own those & have
objections/want to speak up, I'm all ears :)

- Dave


>
> >
> >>
> >> (*) The v5 cu_index would be used whenever we have at least one v5
> >> dwarf unit. Ignoring the section number mismatches, I think this is a
> >> reasonable approach, and it is the one I would make if I was in the
> >> shoes of the DWARF v6 committee and needed to make the debug_cu_index
> >> cover mixed-version units
> >>
> >>
> >> Speaking of DWARF v6, I think the debug_cu/tu_index sections are in
> >> need of some redesign anyway. For example, it's not clear to me how
> >> these sections would work in presence of DWARF64 -- all section
> >> offsets are hardcoded to 4 bytes, and we don't even have the
> >> initial_length mechanism to select bigger sizes.
> >
> >
> > Yep - and if we're enumerating things to keep in mind for v6 here:
> probably collapse the tu_index and cu_index, just index all the things
> together in one index, probably? They're all indexed by signatures and
> they're all in the same section now.
>
> I've been wondering about that too. The way I've explained this for
> myself is that these are actually two different keys they are indexing
> over -- type units have a "type signature", whereas compile units have
> a "dwo id". It's always clear from the context which one are you
> referring to so it's possible to have a compile unit and a type unit
> with the same "id" coexist -- which would be trickier if they were in
> the same index.
>
> >
> > Good point about DWARF64 there. (I feel like, if possible, all the DWARF
> contribution headers should be "DWARF32/64 length, then version, then
> stuff..." - but I guess that might mean needing to rename some sections
> again... (I think debug_macro missed this too - it has no length, just a
> version).
>
> I can see how debug_[ct]u_index could be considered special as (I
> think) there's only ever supposed to be a single contribution in them,
> and we could invent a different way to signal DWARF64 there, but the
> lack of the initial_length field in debug_macro sounds like a big
> omission. I would very much like to clean this up, though I dread the
> idea of introducing yet another macro section (debug_maclists anyone ?
> :P)
>
> pl
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dwarfstd.org/pipermail/dwarf-discuss-dwarfstd.org/attachments/20200309/cd5dd45e/attachment.html>