[Dwarf-discuss] Seeking a test program with a >4GB .debug_info section
David Blaikie
dblaikie@gmail.com
Fri Apr 21 20:36:18 GMT 2023
On Fri, Apr 21, 2023 at 12:44 PM John DelSignore <JDelSignore@perforce.com>
wrote:
> Well, it took a long time to compile 5 CUs that contained your test code,
> and things were looking promising, but the link failed:
>
> rocm2 42 04/21 15:14 /build/jdelsign/fatty % make
> g++ -g -c fatty4.cxx -o fatty4.o
> g++ -g -c fatty5.cxx -o fatty5.o
> g++ -g -o fatty fatty.o fatty2.o fatty3.o fatty4.o fatty5.o
> fatty5.o:(.debug_aranges+0x6): relocation truncated to fit: R_X86_64_32
> against `.debug_info'
> collect2: error: ld returned 1 exit status
> make: *** [Makefile:5: fatty] Error 1
> rocm2 43 04/21 15:39 /build/jdelsign/fatty %
>
> I guess I'm now in favor of the proposal to get rid of .debug_aranges. :-)
>
I guess, backing up - what's your goal/what're you trying to do with DWARF
over 4GB?
You do have to use DWARF64 for a .debug_info section over 4GB - for any
section-relative reference in that section, such as cross-CU references
(sec_offset), or aranges or debug_names I think, etc. Because with DWARF32
the section references are 32bit, so can't exceed 4GB.
(also, with a different example, you'd get string data over 4GB, which
you'd also need DWARF64 for, or in DWARFv5 (with some dispensations from
DWARFv6) you could use DWARF64 for .debug_str_offsets (assuming all string
references were strx forms) without using DWARF64 for everything else)
In Split DWARF if each contribution is <4GB you only need a 64 bit
cu/tu_index and 64 bit str_offsets (& you could even do that selectively,
only using DWARF64 str_offsets for contributions that need 64 bit offsets),
since nothing else references across whole sections - which makes it much
more scalable/easier to solve.
Also, another issue is that even if you have simple/small bits of DWARF32
(in some precompiled library, etc), if your total program exceeds 64 bit,
you may not be able link the program because that DWARF32 might end up
being put at the end of the section, and so the debug_aranges, for
instance, needs to record the offset of the CU in 32 bits but can't. So
there's various discussions about linkers being able to sort the DWARF32
contributions earlier/first before the DWARF64 contributions. Then you
could still link DWARF32 precompiled libraries into huge programs that
exceed the DWARF32 limits. I can go find some links to those
threads/discussions if you need them, I think some happened in the LLVM
open source community.
- Dave
> Cheers, John D.
> On 4/21/23 13:28, John DelSignore wrote:
>
> Thanks David, this is useful. I'll see what I can cobble together.
>
> Cheers, John D.
> On 4/20/23 21:58, David Blaikie wrote:
>
> Oh, and I guess you could always make something even more artificial by
> hand - if you compile some random code with -g to assembly, you could then
> just pad out a .debug_info contribution with lots of zeros (there are some
> assembly directives for that, I think, but don't know assembly that well
> off hand) - would make it arbitrarily large without the need to tax the
> compiler creating novel/real DWARF, etc.
>
> On Thu, Apr 20, 2023 at 6:54 PM David Blaikie <dblaikie@gmail.com> wrote:
>
>> I /believe/ that Chromium (maybe specifically on ARM? not sure) may have
>> hit/had problems with the 4GB limit - probably trivially if you build with
>> clang but pass `-fstandalone-debug` which disables many type
>> reduction/deduplication strategies.
>>
>> If you want something more standalone... this:
>>
>>
>> #define MEMBERS(BASE) \
>> int BASE##0 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##1 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##2 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##3 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##4 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##5 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##6 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##7 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##8 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int); \
>> int BASE##9 (int, int, int, int, int, int, int, int, int, int, int,
>> int, int, int, int, int, int, int, int, int);
>> template<int ... i>
>> struct t1 {
>> MEMBERS(f0)
>> MEMBERS(f1)
>> MEMBERS(f2)
>> MEMBERS(f3)
>> MEMBERS(f4)
>> MEMBERS(f5)
>> MEMBERS(f6)
>> MEMBERS(f7)
>> MEMBERS(f8)
>> MEMBERS(f9)
>> };
>> #define ITER(A, B) \
>> template <int... i> \
>> struct A { \
>> B<i..., 0> v0; \
>> B<i..., 1> v1; \
>> B<i..., 2> v2; \
>> B<i..., 3> v3; \
>> B<i..., 4> v4; \
>> B<i..., 5> v5; \
>> B<i..., 6> v6; \
>> B<i..., 7> v7; \
>> B<i..., 8> v8; \
>> B<i..., 9> v9; \
>> };
>> ITER(t2, t1);
>> ITER(t3, t2);
>> ITER(t4, t3);
>> ITER(t5, t4);
>> ITER(t6, t5);
>> ITER(t7, t6);
>> ITER(top, t7);
>> int main() {
>> t6<> v;
>> }
>>
>> Doesn't quite hit 4GB, it's about 1.2GB in .debug_info (& takes 2.5
>> minutes to compile with clang) - 5 of these (could stamp them out by
>> including this file into a few other source files & just changing the
>> `main` function to some other name in each)
>>
>> This specifically doesn't push the .debug_str section as hard - it's
>> about half the size of the .debug_info in this program.
>>
>>
>>
>> On Thu, Apr 20, 2023 at 7:08 AM John DelSignore via Dwarf-discuss <
>> dwarf-discuss@lists.dwarfstd.org> wrote:
>>
>>> Is anyone aware of an open-source program or test program that when
>>> compiled and built on Linux x86_64, results in a .debug_info section that
>>> is greater than 4GB? I'm looking for a test program (realistic or not) that
>>> contains 32-bit DWARF CUs in a .debug_info section that is about 5GB long,
>>> or longer.
>>>
>>> Thanks, John D.
>>>
>>>
>>>
>>> This e-mail may contain information that is privileged or confidential.
>>> If you are not the intended recipient, please delete the e-mail and any
>>> attachments and notify us immediately.
>>>
>>> --
>>> Dwarf-discuss mailing list
>>> Dwarf-discuss@lists.dwarfstd.org
>>> https://lists.dwarfstd.org/mailman/listinfo/dwarf-discuss
>>>
>>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> click on links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
> This e-mail may contain information that is privileged or confidential. If
> you are not the intended recipient, please delete the e-mail and any
> attachments and notify us immediately.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.dwarfstd.org/pipermail/dwarf-discuss/attachments/20230421/99dd7ef8/attachment.htm>
More information about the Dwarf-discuss
mailing list