[Dwarf-Discuss] Vendor extensions in .debug_macinfo

[Jakub Jelinek]

On Wed, Jul 20, 2011 at 01:17:57PM -0700, David Anderson wrote:
> On 07/20/2011 01:07 PM, David Anderson wrote:
> >
> >On the other hand,  I have not seen the detailed proposal, so
> >perhaps that proposal

The previous proposal (the one adding opcodes to the current
.debug_macinfo instead of creating a new section) was worded roughly in
http://gcc.gnu.org/ml/gcc-patches/2011-07/msg01284.html

In http://gcc.gnu.org/ml/gcc-patches/2011-07/msg01736.html
is a rough implementation of .debug_gnu_macro section and
DW_AT_GNU_macros referencing it.

Here are rough DWARF4 edits for this.
I'm not sure in what way should the old .debug_macinfo be
deprecated in DWARF5, the following edits describe both the
.debug_macro section and .debug_macinfo too, and refer to the
latter as legacy.  Another option I guess would be to document
in DWARF5 only .debug_macro section and just in DW_AT_macro_info
description say that it doesn't refer to .debug_macro section,
but instead to legacy .debug_macinfo as defined in DWARF2 through DWARF4
and other than that don't document .debug_macinfo at all, in appendix
F it would have a rows like:
.debug_macinfo	-	-	-	x
.debug_macro	x	x	x	5
then.  What do you prefer?

Also, regarding the section headers, I'm still not convinced they are
particularly useful, as .debug_macro is only useful with the corresponding
referencing .debug_info which has its own version number.  But if you think
it is useful, should it be in front of all .debug_macro chunks (even the
transparent include chains)?  And, should it include the offset size
or is that unnecessary?  With the section headers everywhere and offset size
everywhere some tools could parse/display .debug_macro content even without
parsing referencing .debug_info, just it couldn't give meaning to start_file
filenames (translate the constant into filename string).  Without them it
can't and would need to handle it e.g. similarly how .debug_loc needs to be
handled in such tools.  As the msg01736.html URL above refers to,
in the test binary from the compiler itself, there are 395 DW_AT_GNU_macros
attributes and 511 unique DW_GNU_MACRO_transparent_include chains, so
the size of 3 byte headers for 395+511 chunks is < 3KB for the ~ 1MB size
of .debug_gnu_macro, i.e. .3%.

2.2
Change "Macro information (#define, #undef)" to "Legacy macro information (#define, #undef)".
Add
"DW_AT_macros		Macro information (#define, #undef)"
row to the figure 2.  In figure 3 replace:
add "legacy " before "macro definition" in "macptr" row.  Add
"macroptr	Refers to a location in the DWARF section that holds macro definition information."
row.

3.1.1
5. Add "legacy " before "macro information".

Add:
12. A DW_AT_macros attribute whose value is a section offset to the
macro information for this compilation unit.

This information is placed in a separate object file section from the
debugging information entries themselves. The value of the macro information
attribute is the offset in the .debug_macro section of the section header
of the macro information for this compilation unit (see Section 6.3).

6.3
Replace ".debug_macinfo" with ".debug_macro".
Replace:
"The macro information for each compilation unit is represented as a series
of ?macinfo? entries. Each macinfo entry consists of a ?type code?
and up to two additional operands."
with:
"The macro information for each compilation unit starts with a section
header followed by a series of ?macinfo? entries. Each macinfo entry
consists of a ?type code? and zero or more operands."
Add at the end:
"The section header starts with a 2-byte version number, followed by
1-byte offset size, which is 4 in 32-bit DWARF and 8 in 64-bit DWARF."

6.3.1
Replace:
"DW_MACINFO_define	A macro definition.
DW_MACINFO_undef	A macro undefinition.
DW_MACINFO_start_file	The start of a new source file inclusion.
DW_MACINFO_end_file	The end of the current source file inclusion.
DW_MACINFO_vendor_ext	Vendor specific macro information directives."
with
"DW_MACRO_define		A macro definition.
DW_MACRO_undef			A macro undefinition.
DW_MACRO_start_file		The start of a new source file inclusion.
DW_MACRO_end_file		The end of the current source file inclusion.
DW_MACRO_define_indirect	A macro definition.
DW_MACRO_undef_indirect		A macro undefinition.
DW_MACRO_transparent_include	Include a sequence of entries from given offset.
DW_MACRO_define_opcode		Define extension opcode and its arguments."

6.3.1.1
Replace all "DW_MACINFO_define" occurrences with "DW_MACRO_define" and
all "DW_MACINFO_undef" with "DW_MACRO_undef".  Append:
a"All DW_MACRO_define_indirect and DW_MACRO_undef_indirect entries have
two operands.  The first operand encodes the line number of the source line
on which the relevant defining or undefining macro directives appeared.
The second operand consists of an offset into a string table contained in
the .debug_str section of the object file.  The size of the operand is
given in the section header offset size.  Apart from the
encoding of the operands these entries are equivalent to DW_MACRO_define
resp. DW_MACRO_undef.

6.3.1.2
Replace all "DW_MACINFO_start_file" occurrences with "DW_MACRO_start_file".

6.3.1.3
Replace all "DW_MACINFO_end_file" occurrences with "DW_MACRO_end_file".

6.3.1.4
Replace whole section with:
"6.3.1.4  Transparent inclusion of a sequence of entries

A DW_MACRO_transparent_include entry has one operand, offset into
another part of the .debug_macro section.  The size of the operand
is given in the section header offset size.  This entry instructs
the consumer to replace this entry with a sequence of macinfo entries found
at the given .debug_macro offset, up to, but excluding, the terminating
entry with type code 0.  This entry type is aimed at sharing duplicate
sequences of macinfo entries between macinfo from different compilation
units.  The producer should ensure that all referers to the sequence have
the same section header and that either DW_MACRO_start_file entries aren't
in those sequences, or only macinfo entries referencing the same
.debug_line section part include the sequence."

Add:
"6.3.1.5  Defining new opcodes and operands

A DW_MACRO_define_opcode entry has 2 operands.  The first operand
is a one byte constant with the type code it defines operand types for,
the second operand is a DW_FORM_block encoded array of operand forms.
The second operand starts with an unsigned LEB128 encoded number of operands
and for each of the operands there is one byte, containing a form encoding
how the corresponding operand is encoded.  This entry allows to define
new vendor extension entry types which consumers will be able to skip over
and ignore.  Each so defined opcode is valid for subsequent entries
until the terminating entry with type code 0, including any sequences
included from those entries using DW_MACRO_transparent_include.
Vendor extension macinfo entry codes should be added in the range
DW_MACRO_lo_user to DW_MACRO_hi_user, all other unassigned opcodes
are reserved for future DWARF standards."

6.3.2
Replace "DW_MACINFO_start_file" with "DW_MACRO_start_file" and
"DW_MACINFO_end_file" with "DW_MACRO_end_file".

6.3.3
Replace "DW_MACINFO_define and DW_MACINFO_undef" with
"DW_MACRO_define, DW_MACRO_define_indirect, DW_MACRO_undef and
DW_MACRO_undef_indirect" and replace
"DW_MACINFO_define or DW_MACINFO_undef" with
"DW_MACRO_define, DW_MACRO_define_indirect, DW_MACRO_undef or
DW_MACRO_undef_indirect".
Replace "DW_MACINFO_start_file" with "DW_MACRO_start_file".

6.3.4
Replace ".debug_macinfo" with ".debug_macro".

Add new section:
"6.3.5  Legacy macro information

The legacy macro information is present in section .debug_macinfo
instead of .debug_macro and doesn't start with any section header, it is
just a series of macinfo entries.  There are 5 legacy macinfo opcode types:

DW_MACINFO_define	A macro definition.
DW_MACINFO_undef	A macro undefinition.
DW_MACINFO_start_file	The start of a new source file inclusion.
DW_MACINFO_end_file	The end of the current source file inclusion.
DW_MACINFO_vendor_ext	Vendor specific macro information directives.

The first 4 are, appart from different name, counterparts of
DW_MACRO_define, DW_MACRO_undef, DW_MACRO_start_file and DW_MACRO_end_file,
with the same encoding of arguments and same semantics.

6.3.5.1  Vendor Extension Entries
A DW_MACINFO_vendor_ext entry has two operands. The first is a constant. The
second is a null-terminated character string. The meaning and/or significance
of these operands is intentionally left undefined by this specification.
A consumer must be able to totally ignore all DW_MACINFO_vendor_ext entries
that it does not understand (see Section 7.1)."

7.5.4
Replace
"loclistptr, macptr or rangelistptr;" with "loclistptr, macptr, macroptr or
rangelistptr;".
Add:
"macroptr
This is an offset into the .debug_macro section (DW_FORM_sec_offset). It
consists of an offset from the beginning of the .debug_macinfo section to the
first byte of the data making up the macro information list for the compilation
unit. It is relocatable in a relocatable object file, and relocated in an
executable or shared object. In the 32-bit DWARF format, this
offset is a 4-byte unsigned value; in the 64-bit DWARF format, it is an
8-byte unsigned value (see Section 7.4)."

Add:
"DW_AT_macros	0xXX	macroptr"
to figure 20.

In figure 21 on "DW_FORM_sec_offset" row change "macptr," to "macptr, macroptr,".

7.22
After:
"The macinfo type is encoded as a single byte. The encodings are given in Figure 39."
add a new table (new figure 39):

"Macinfo type name		Value
DW_MACRO_define			0x01
DW_MACRO_undef			0x02
DW_MACRO_start_file		0x03
DW_MACRO_end_file		0x04
DW_MACRO_define_indirect	0x05
DW_MACRO_undef_indirect		0x06
DW_MACRO_transparent_include	0x07
DW_MACRO_define_opcode		0x08
DW_MACRO_lo_user		0xe0
DW_MACRO_hi_user		0xff

Macinfo type encodings"

followed by:

"The legacy macinfo type is encoded as a single byte. The encodings are
given in Figure 40." and replacing "Macinfo type name" with
"Legacy macinfo type name" in the table header and "Macinfo type encodings"
with "Legacy macinfo type encodings" in the description.

Appendix A
Add "DW_AT_macros" to allowable attributes for "DW_TAG_compile_unit"
and "DW_TAG_partial_unit".

Appendix B
Fix a typo, replace "DW_AT_macinfo" with "DW_AT_macro_info".  Add
( .debug_info/.debug_types ) -> [ DW_AT_macros (j) ] -> ( .debug_macro )
and from ( .debug_macro ) -> [ DW_MACRO_transparent_include (k) ] -> ( .debug_macro ) and
( .debug_macro ) -> [ DW_MACRO_define_indirect, DW_MACRO_undef_indirect (l) ] -> (.debug_str )
to the picture.
In (h) add "legacy " before ".debug_macinfo".
Add:
"(j) .debug_info An attribute value of class macroptr (specifically form
		 DW_FORM_sec_offset) is an offset within the .debug_macro
		 section of the section header of the macro information
		 for the referencing unit.
(k) .debug_macro A macinfo operand of the form DW_FORM_sec_offset is an
		 offset into another part of the .debug_macro section,
		 to the first macinfo entry in the sequence instead of
		 a section header.
(l) .debug_macro A macinfo operand of the form DW_FORM_strp is an offset
		 into the .debug_str section of the corresponding string."

Appendix F
Add
".debug_macro	x	x	x	5"
row to the table.

	Jakub