diff options
Diffstat (limited to 'Documentation/s390/Debugging390.txt')
| -rw-r--r-- | Documentation/s390/Debugging390.txt | 2536 |
1 files changed, 2536 insertions, 0 deletions
diff --git a/Documentation/s390/Debugging390.txt b/Documentation/s390/Debugging390.txt new file mode 100644 index 00000000000..adbfe620c06 --- /dev/null +++ b/Documentation/s390/Debugging390.txt @@ -0,0 +1,2536 @@ + + Debugging on Linux for s/390 & z/Architecture + by + Denis Joseph Barrow (djbarrow@de.ibm.com,barrow_dj@yahoo.com) + Copyright (C) 2000-2001 IBM Deutschland Entwicklung GmbH, IBM Corporation + Best viewed with fixed width fonts + +Overview of Document: +===================== +This document is intended to give an good overview of how to debug +Linux for s/390 & z/Architecture it isn't intended as a complete reference & not a +tutorial on the fundamentals of C & assembly, it dosen't go into +390 IO in any detail. It is intended to complement the documents in the +reference section below & any other worthwhile references you get. + +It is intended like the Enterprise Systems Architecture/390 Reference Summary +to be printed out & used as a quick cheat sheet self help style reference when +problems occur. + +Contents +======== +Register Set +Address Spaces on Intel Linux +Address Spaces on Linux for s/390 & z/Architecture +The Linux for s/390 & z/Architecture Kernel Task Structure +Register Usage & Stackframes on Linux for s/390 & z/Architecture +A sample program with comments +Compiling programs for debugging on Linux for s/390 & z/Architecture +Figuring out gcc compile errors +Debugging Tools +objdump +strace +Performance Debugging +Debugging under VM +s/390 & z/Architecture IO Overview +Debugging IO on s/390 & z/Architecture under VM +GDB on s/390 & z/Architecture +Stack chaining in gdb by hand +Examining core dumps +ldd +Debugging modules +The proc file system +Starting points for debugging scripting languages etc. +Dumptool & Lcrash +SysRq +References +Special Thanks + +Register Set +============ +The current architectures have the following registers. + +16 General propose registers, 32 bit on s/390 64 bit on z/Architecture, r0-r15 or gpr0-gpr15 used for arithmetic & addressing. + +16 Control registers, 32 bit on s/390 64 bit on z/Architecture, ( cr0-cr15 kernel usage only ) used for memory management, +interrupt control,debugging control etc. + +16 Access registers ( ar0-ar15 ) 32 bit on s/390 & z/Architecture +not used by normal programs but potentially could +be used as temporary storage. Their main purpose is their 1 to 1 +association with general purpose registers and are used in +the kernel for copying data between kernel & user address spaces. +Access register 0 ( & access register 1 on z/Architecture ( needs 64 bit +pointer ) ) is currently used by the pthread library as a pointer to +the current running threads private area. + +16 64 bit floating point registers (fp0-fp15 ) IEEE & HFP floating +point format compliant on G5 upwards & a Floating point control reg (FPC) +4 64 bit registers (fp0,fp2,fp4 & fp6) HFP only on older machines. +Note: +Linux (currently) always uses IEEE & emulates G5 IEEE format on older machines, +( provided the kernel is configured for this ). + + +The PSW is the most important register on the machine it +is 64 bit on s/390 & 128 bit on z/Architecture & serves the roles of +a program counter (pc), condition code register,memory space designator. +In IBM standard notation I am counting bit 0 as the MSB. +It has several advantages over a normal program counter +in that you can change address translation & program counter +in a single instruction. To change address translation, +e.g. switching address translation off requires that you +have a logical=physical mapping for the address you are +currently running at. + + Bit Value +s/390 z/Architecture +0 0 Reserved ( must be 0 ) otherwise specification exception occurs. + +1 1 Program Event Recording 1 PER enabled, + PER is used to facilititate debugging e.g. single stepping. + +2-4 2-4 Reserved ( must be 0 ). + +5 5 Dynamic address translation 1=DAT on. + +6 6 Input/Output interrupt Mask + +7 7 External interrupt Mask used primarily for interprocessor signalling & + clock interrupts. + +8-11 8-11 PSW Key used for complex memory protection mechanism not used under linux + +12 12 1 on s/390 0 on z/Architecture + +13 13 Machine Check Mask 1=enable machine check interrupts + +14 14 Wait State set this to 1 to stop the processor except for interrupts & give + time to other LPARS used in CPU idle in the kernel to increase overall + usage of processor resources. + +15 15 Problem state ( if set to 1 certain instructions are disabled ) + all linux user programs run with this bit 1 + ( useful info for debugging under VM ). + +16-17 16-17 Address Space Control + + 00 Primary Space Mode when DAT on + The linux kernel currently runs in this mode, CR1 is affiliated with + this mode & points to the primary segment table origin etc. + + 01 Access register mode this mode is used in functions to + copy data between kernel & user space. + + 10 Secondary space mode not used in linux however CR7 the + register affiliated with this mode is & this & normally + CR13=CR7 to allow us to copy data between kernel & user space. + We do this as follows: + We set ar2 to 0 to designate its + affiliated gpr ( gpr2 )to point to primary=kernel space. + We set ar4 to 1 to designate its + affiliated gpr ( gpr4 ) to point to secondary=home=user space + & then essentially do a memcopy(gpr2,gpr4,size) to + copy data between the address spaces, the reason we use home space for the + kernel & don't keep secondary space free is that code will not run in + secondary space. + + 11 Home Space Mode all user programs run in this mode. + it is affiliated with CR13. + +18-19 18-19 Condition codes (CC) + +20 20 Fixed point overflow mask if 1=FPU exceptions for this event + occur ( normally 0 ) + +21 21 Decimal overflow mask if 1=FPU exceptions for this event occur + ( normally 0 ) + +22 22 Exponent underflow mask if 1=FPU exceptions for this event occur + ( normally 0 ) + +23 23 Significance Mask if 1=FPU exceptions for this event occur + ( normally 0 ) + +24-31 24-30 Reserved Must be 0. + + 31 Extended Addressing Mode + 32 Basic Addressing Mode + Used to set addressing mode + PSW 31 PSW 32 + 0 0 24 bit + 0 1 31 bit + 1 1 64 bit + +32 1=31 bit addressing mode 0=24 bit addressing mode (for backward + compatibility ), linux always runs with this bit set to 1 + +33-64 Instruction address. + 33-63 Reserved must be 0 + 64-127 Address + In 24 bits mode bits 64-103=0 bits 104-127 Address + In 31 bits mode bits 64-96=0 bits 97-127 Address + Note: unlike 31 bit mode on s/390 bit 96 must be zero + when loading the address with LPSWE otherwise a + specification exception occurs, LPSW is fully backward + compatible. + + +Prefix Page(s) +-------------- +This per cpu memory area is too intimately tied to the processor not to mention. +It exists between the real addresses 0-4096 on s/390 & 0-8192 z/Architecture & is exchanged +with a 1 page on s/390 or 2 pages on z/Architecture in absolute storage by the set +prefix instruction in linux'es startup. +This page is mapped to a different prefix for each processor in an SMP configuration +( assuming the os designer is sane of course :-) ). +Bytes 0-512 ( 200 hex ) on s/390 & 0-512,4096-4544,4604-5119 currently on z/Architecture +are used by the processor itself for holding such information as exception indications & +entry points for exceptions. +Bytes after 0xc00 hex are used by linux for per processor globals on s/390 & z/Architecture +( there is a gap on z/Architecure too currently between 0xc00 & 1000 which linux uses ). +The closest thing to this on traditional architectures is the interrupt +vector table. This is a good thing & does simplify some of the kernel coding +however it means that we now cannot catch stray NULL pointers in the +kernel without hard coded checks. + + + +Address Spaces on Intel Linux +============================= + +The traditional Intel Linux is approximately mapped as follows forgive +the ascii art. +0xFFFFFFFF 4GB Himem ***************** + * * + * Kernel Space * + * * + ***************** **************** +User Space Himem (typically 0xC0000000 3GB )* User Stack * * * + ***************** * * + * Shared Libs * * Next Process * + ***************** * to * + * * <== * Run * <== + * User Program * * * + * Data BSS * * * + * Text * * * + * Sections * * * +0x00000000 ***************** **************** + +Now it is easy to see that on Intel it is quite easy to recognise a kernel address +as being one greater than user space himem ( in this case 0xC0000000). +& addresses of less than this are the ones in the current running program on this +processor ( if an smp box ). +If using the virtual machine ( VM ) as a debugger it is quite difficult to +know which user process is running as the address space you are looking at +could be from any process in the run queue. + +The limitation of Intels addressing technique is that the linux +kernel uses a very simple real address to virtual addressing technique +of Real Address=Virtual Address-User Space Himem. +This means that on Intel the kernel linux can typically only address +Himem=0xFFFFFFFF-0xC0000000=1GB & this is all the RAM these machines +can typically use. +They can lower User Himem to 2GB or lower & thus be +able to use 2GB of RAM however this shrinks the maximum size +of User Space from 3GB to 2GB they have a no win limit of 4GB unless +they go to 64 Bit. + + +On 390 our limitations & strengths make us slightly different. +For backward compatibility we are only allowed use 31 bits (2GB) +of our 32 bit addresses,however, we use entirely separate address +spaces for the user & kernel. + +This means we can support 2GB of non Extended RAM on s/390, & more +with the Extended memory management swap device & +currently 4TB of physical memory currently on z/Architecture. + + +Address Spaces on Linux for s/390 & z/Architecture +================================================== + +Our addressing scheme is as follows + + +Himem 0x7fffffff 2GB on s/390 ***************** **************** +currently 0x3ffffffffff (2^42)-1 * User Stack * * * +on z/Architecture. ***************** * * + * Shared Libs * * * + ***************** * * + * * * Kernel * + * User Program * * * + * Data BSS * * * + * Text * * * + * Sections * * * +0x00000000 ***************** **************** + +This also means that we need to look at the PSW problem state bit +or the addressing mode to decide whether we are looking at +user or kernel space. + +Virtual Addresses on s/390 & z/Architecture +=========================================== + +A virtual address on s/390 is made up of 3 parts +The SX ( segment index, roughly corresponding to the PGD & PMD in linux terminology ) +being bits 1-11. +The PX ( page index, corresponding to the page table entry (pte) in linux terminology ) +being bits 12-19. +The remaining bits BX (the byte index are the offset in the page ) +i.e. bits 20 to 31. + +On z/Architecture in linux we currently make up an address from 4 parts. +The region index bits (RX) 0-32 we currently use bits 22-32 +The segment index (SX) being bits 33-43 +The page index (PX) being bits 44-51 +The byte index (BX) being bits 52-63 + +Notes: +1) s/390 has no PMD so the PMD is really the PGD also. +A lot of this stuff is defined in pgtable.h. + +2) Also seeing as s/390's page indexes are only 1k in size +(bits 12-19 x 4 bytes per pte ) we use 1 ( page 4k ) +to make the best use of memory by updating 4 segment indices +entries each time we mess with a PMD & use offsets +0,1024,2048 & 3072 in this page as for our segment indexes. +On z/Architecture our page indexes are now 2k in size +( bits 12-19 x 8 bytes per pte ) we do a similar trick +but only mess with 2 segment indices each time we mess with +a PMD. + +3) As z/Architecture supports upto a massive 5-level page table lookup we +can only use 3 currently on Linux ( as this is all the generic kernel +currently supports ) however this may change in future +this allows us to access ( according to my sums ) +4TB of virtual storage per process i.e. +4096*512(PTES)*1024(PMDS)*2048(PGD) = 4398046511104 bytes, +enough for another 2 or 3 of years I think :-). +to do this we use a region-third-table designation type in +our address space control registers. + + +The Linux for s/390 & z/Architecture Kernel Task Structure +========================================================== +Each process/thread under Linux for S390 has its own kernel task_struct +defined in linux/include/linux/sched.h +The S390 on initialisation & resuming of a process on a cpu sets +the __LC_KERNEL_STACK variable in the spare prefix area for this cpu +( which we use for per processor globals). + +The kernel stack pointer is intimately tied with the task stucture for +each processor as follows. + + s/390 + ************************ + * 1 page kernel stack * + * ( 4K ) * + ************************ + * 1 page task_struct * + * ( 4K ) * +8K aligned ************************ + + z/Architecture + ************************ + * 2 page kernel stack * + * ( 8K ) * + ************************ + * 2 page task_struct * + * ( 8K ) * +16K aligned ************************ + +What this means is that we don't need to dedicate any register or global variable +to point to the current running process & can retrieve it with the following +very simple construct for s/390 & one very similar for z/Architecture. + +static inline struct task_struct * get_current(void) +{ + struct task_struct *current; + __asm__("lhi %0,-8192\n\t" + "nr %0,15" + : "=r" (current) ); + return current; +} + +i.e. just anding the current kernel stack pointer with the mask -8192. +Thankfully because Linux dosen't have support for nested IO interrupts +& our devices have large buffers can survive interrupts being shut for +short amounts of time we don't need a separate stack for interrupts. + + + + +Register Usage & Stackframes on Linux for s/390 & z/Architecture +================================================================= +Overview: +--------- +This is the code that gcc produces at the top & the bottom of +each function, it usually is fairly consistent & similar from +function to function & if you know its layout you can probalby +make some headway in finding the ultimate cause of a problem +after a crash without a source level debugger. + +Note: To follow stackframes requires a knowledge of C or Pascal & +limited knowledge of one assembly language. + +It should be noted that there are some differences between the +s/390 & z/Architecture stack layouts as the z/Architecture stack layout didn't have +to maintain compatibility with older linkage formats. + +Glossary: +--------- +alloca: +This is a built in compiler function for runtime allocation +of extra space on the callers stack which is obviously freed +up on function exit ( e.g. the caller may choose to allocate nothing +of a buffer of 4k if required for temporary purposes ), it generates +very efficient code ( a few cycles ) when compared to alternatives +like malloc. + +automatics: These are local variables on the stack, +i.e they aren't in registers & they aren't static. + +back-chain: +This is a pointer to the stack pointer before entering a +framed functions ( see frameless function ) prologue got by +deferencing the address of the current stack pointer, + i.e. got by accessing the 32 bit value at the stack pointers +current location. + +base-pointer: +This is a pointer to the back of the literal pool which +is an area just behind each procedure used to store constants +in each function. + +call-clobbered: The caller probably needs to save these registers if there +is something of value in them, on the stack or elsewhere before making a +call to another procedure so that it can restore it later. + +epilogue: +The code generated by the compiler to return to the caller. + +frameless-function +A frameless function in Linux for s390 & z/Architecture is one which doesn't +need more than the register save area ( 96 bytes on s/390, 160 on z/Architecture ) +given to it by the caller. +A frameless function never: +1) Sets up a back chain. +2) Calls alloca. +3) Calls other normal functions +4) Has automatics. + +GOT-pointer: +This is a pointer to the global-offset-table in ELF +( Executable Linkable Format, Linux'es most common executable format ), +all globals & shared library objects are found using this pointer. + +lazy-binding +ELF shared libraries are typically only loaded when routines in the shared +library are actually first called at runtime. This is lazy binding. + +procedure-linkage-table +This is a table found from the GOT which contains pointers to routines +in other shared libraries which can't be called to by easier means. + +prologue: +The code generated by the compiler to set up the stack frame. + +outgoing-args: +This is extra area allocated on the stack of the calling function if the +parameters for the callee's cannot all be put in registers, the same +area can be reused by each function the caller calls. + +routine-descriptor: +A COFF executable format based concept of a procedure reference +actually being 8 bytes or more as opposed to a simple pointer to the routine. +This is typically defined as follows +Routine Descriptor offset 0=Pointer to Function +Routine Descriptor offset 4=Pointer to Table of Contents +The table of contents/TOC is roughly equivalent to a GOT pointer. +& it means that shared libraries etc. can be shared between several +environments each with their own TOC. + + +static-chain: This is used in nested functions a concept adopted from pascal +by gcc not used in ansi C or C++ ( although quite useful ), basically it +is a pointer used to reference local variables of enclosing functions. +You might come across this stuff once or twice in your lifetime. + +e.g. +The function below should return 11 though gcc may get upset & toss warnings +about unused variables. +int FunctionA(int a) +{ + int b; + FunctionC(int c) + { + b=c+1; + } + FunctionC(10); + return(b); +} + + +s/390 & z/Architecture Register usage +===================================== +r0 used by syscalls/assembly call-clobbered +r1 used by syscalls/assembly call-clobbered +r2 argument 0 / return value 0 call-clobbered +r3 argument 1 / return value 1 (if long long) call-clobbered +r4 argument 2 call-clobbered +r5 argument 3 call-clobbered +r6 argument 5 saved +r7 pointer-to arguments 5 to ... saved +r8 this & that saved +r9 this & that saved +r10 static-chain ( if nested function ) saved +r11 frame-pointer ( if function used alloca ) saved +r12 got-pointer saved +r13 base-pointer saved +r14 return-address saved +r15 stack-pointer saved + +f0 argument 0 / return value ( float/double ) call-clobbered +f2 argument 1 call-clobbered +f4 z/Architecture argument 2 saved +f6 z/Architecture argument 3 saved +The remaining floating points +f1,f3,f5 f7-f15 are call-clobbered. + +Notes: +------ +1) The only requirement is that registers which are used +by the callee are saved, e.g. the compiler is perfectly +capible of using r11 for purposes other than a frame a +frame pointer if a frame pointer is not needed. +2) In functions with variable arguments e.g. printf the calling procedure +is identical to one without variable arguments & the same number of +parameters. However, the prologue of this function is somewhat more +hairy owing to it having to move these parameters to the stack to +get va_start, va_arg & va_end to work. +3) Access registers are currently unused by gcc but are used in +the kernel. Possibilities exist to use them at the moment for +temporary storage but it isn't recommended. +4) Only 4 of the floating point registers are used for +parameter passing as older machines such as G3 only have only 4 +& it keeps the stack frame compatible with other compilers. +However with IEEE floating point emulation under linux on the +older machines you are free to use the other 12. +5) A long long or double parameter cannot be have the +first 4 bytes in a register & the second four bytes in the +outgoing args area. It must be purely in the outgoing args +area if crossing this boundary. +6) Floating point parameters are mixed with outgoing args +on the outgoing args area in the order the are passed in as parameters. +7) Floating point arguments 2 & 3 are saved in the outgoing args area for +z/Architecture + + +Stack Frame Layout +------------------ +s/390 z/Architecture +0 0 back chain ( a 0 here signifies end of back chain ) +4 8 eos ( end of stack, not used on Linux for S390 used in other linkage formats ) +8 16 glue used in other s/390 linkage formats for saved routine descriptors etc. +12 24 glue used in other s/390 linkage formats for saved routine descriptors etc. +16 32 scratch area +20 40 scratch area +24 48 saved r6 of caller function +28 56 saved r7 of caller function +32 64 saved r8 of caller function +36 72 saved r9 of caller function +40 80 saved r10 of caller function +44 88 saved r11 of caller function +48 96 saved r12 of caller function +52 104 saved r13 of caller function +56 112 saved r14 of caller function +60 120 saved r15 of caller function +64 128 saved f4 of caller function +72 132 saved f6 of caller function +80 undefined +96 160 outgoing args passed from caller to callee +96+x 160+x possible stack alignment ( 8 bytes desirable ) +96+x+y 160+x+y alloca space of caller ( if used ) +96+x+y+z 160+x+y+z automatics of caller ( if used ) +0 back-chain + +A sample program with comments. +=============================== + +Comments on the function test +----------------------------- +1) It didn't need to set up a pointer to the constant pool gpr13 as it isn't used +( :-( ). +2) This is a frameless function & no stack is bought. +3) The compiler was clever enough to recognise that it could return the +value in r2 as well as use it for the passed in parameter ( :-) ). +4) The basr ( branch relative & save ) trick works as follows the instruction +has a special case with r0,r0 with some instruction operands is understood as +the literal value 0, some risc architectures also do this ). So now +we are branching to the next address & the address new program counter is +in r13,so now we subtract the size of the function prologue we have executed ++ the size of the literal pool to get to the top of the literal pool +0040037c int test(int b) +{ # Function prologue below + 40037c: 90 de f0 34 stm %r13,%r14,52(%r15) # Save registers r13 & r14 + 400380: 0d d0 basr %r13,%r0 # Set up pointer to constant pool using + 400382: a7 da ff fa ahi %r13,-6 # basr trick + return(5+b); + # Huge main program + 400386: a7 2a 00 05 ahi %r2,5 # add 5 to r2 + + # Function epilogue below + 40038a: 98 de f0 34 lm %r13,%r14,52(%r15) # restore registers r13 & 14 + 40038e: 07 fe br %r14 # return +} + +Comments on the function main +----------------------------- +1) The compiler did this function optimally ( 8-) ) + +Literal pool for main. +400390: ff ff ff ec .long 0xffffffec +main(int argc,char *argv[]) +{ # Function prologue below + 400394: 90 bf f0 2c stm %r11,%r15,44(%r15) # Save necessary registers + 400398: 18 0f lr %r0,%r15 # copy stack pointer to r0 + 40039a: a7 fa ff a0 ahi %r15,-96 # Make area for callee saving + 40039e: 0d d0 basr %r13,%r0 # Set up r13 to point to + 4003a0: a7 da ff f0 ahi %r13,-16 # literal pool + 4003a4: 50 00 f0 00 st %r0,0(%r15) # Save backchain + + return(test(5)); # Main Program Below + 4003a8: 58 e0 d0 00 l %r14,0(%r13) # load relative address of test from + # literal pool + 4003ac: a7 28 00 05 lhi %r2,5 # Set first parameter to 5 + 4003b0: 4d ee d0 00 bas %r14,0(%r14,%r13) # jump to test setting r14 as return + # address using branch & save instruction. + + # Function Epilogue below + 4003b4: 98 bf f0 8c lm %r11,%r15,140(%r15)# Restore necessary registers. + 4003b8: 07 fe br %r14 # return to do program exit +} + + +Compiler updates +---------------- + +main(int argc,char *argv[]) +{ + 4004fc: 90 7f f0 1c stm %r7,%r15,28(%r15) + 400500: a7 d5 00 04 bras %r13,400508 <main+0xc> + 400504: 00 40 04 f4 .long 0x004004f4 + # compiler now puts constant pool in code to so it saves an instruction + 400508: 18 0f lr %r0,%r15 + 40050a: a7 fa ff a0 ahi %r15,-96 + 40050e: 50 00 f0 00 st %r0,0(%r15) + return(test(5)); + 400512: 58 10 d0 00 l %r1,0(%r13) + 400516: a7 28 00 05 lhi %r2,5 + 40051a: 0d e1 basr %r14,%r1 + # compiler adds 1 extra instruction to epilogue this is done to + # avoid processor pipeline stalls owing to data dependencies on g5 & + # above as register 14 in the old code was needed directly after being loaded + # by the lm %r11,%r15,140(%r15) for the br %14. + 40051c: 58 40 f0 98 l %r4,152(%r15) + 400520: 98 7f f0 7c lm %r7,%r15,124(%r15) + 400524: 07 f4 br %r4 +} + + +Hartmut ( our compiler developer ) also has been threatening to take out the +stack backchain in optimised code as this also causes pipeline stalls, you +have been warned. + +64 bit z/Architecture code disassembly +-------------------------------------- + +If you understand the stuff above you'll understand the stuff +below too so I'll avoid repeating myself & just say that +some of the instructions have g's on the end of them to indicate +they are 64 bit & the stack offsets are a bigger, +the only other difference you'll find between 32 & 64 bit is that +we now use f4 & f6 for floating point arguments on 64 bit. +00000000800005b0 <test>: +int test(int b) +{ + return(5+b); + 800005b0: a7 2a 00 05 ahi %r2,5 + 800005b4: b9 14 00 22 lgfr %r2,%r2 # downcast to integer + 800005b8: 07 fe br %r14 + 800005ba: 07 07 bcr 0,%r7 + + +} + +00000000800005bc <main>: +main(int argc,char *argv[]) +{ + 800005bc: eb bf f0 58 00 24 stmg %r11,%r15,88(%r15) + 800005c2: b9 04 00 1f lgr %r1,%r15 + 800005c6: a7 fb ff 60 aghi %r15,-160 + 800005ca: e3 10 f0 00 00 24 stg %r1,0(%r15) + return(test(5)); + 800005d0: a7 29 00 05 lghi %r2,5 + # brasl allows jumps > 64k & is overkill here bras would do fune + 800005d4: c0 e5 ff ff ff ee brasl %r14,800005b0 <test> + 800005da: e3 40 f1 10 00 04 lg %r4,272(%r15) + 800005e0: eb bf f0 f8 00 04 lmg %r11,%r15,248(%r15) + 800005e6: 07 f4 br %r4 +} + + + +Compiling programs for debugging on Linux for s/390 & z/Architecture +==================================================================== +-gdwarf-2 now works it should be considered the default debugging +format for s/390 & z/Architecture as it is more reliable for debugging +shared libraries, normal -g debugging works much better now +Thanks to the IBM java compiler developers bug reports. + +This is typically done adding/appending the flags -g or -gdwarf-2 to the +CFLAGS & LDFLAGS variables Makefile of the program concerned. + +If using gdb & you would like accurate displays of registers & + stack traces compile without optimisation i.e make sure +that there is no -O2 or similar on the CFLAGS line of the Makefile & +the emitted gcc commands, obviously this will produce worse code +( not advisable for shipment ) but it is an aid to the debugging process. + +This aids debugging because the compiler will copy parameters passed in +in registers onto the stack so backtracing & looking at passed in +parameters will work, however some larger programs which use inline functions +will not compile without optimisation. + +Debugging with optimisation has since much improved after fixing +some bugs, please make sure you are using gdb-5.0 or later developed +after Nov'2000. + +Figuring out gcc compile errors +=============================== +If you are getting a lot of syntax errors compiling a program & the problem +isn't blatantly obvious from the source. +It often helps to just preprocess the file, this is done with the -E +option in gcc. +What this does is that it runs through the very first phase of compilation +( compilation in gcc is done in several stages & gcc calls many programs to +achieve its end result ) with the -E option gcc just calls the gcc preprocessor (cpp). +The c preprocessor does the following, it joins all the files #included together +recursively ( #include files can #include other files ) & also the c file you wish to compile. +It puts a fully qualified path of the #included files in a comment & it +does macro expansion. +This is useful for debugging because +1) You can double check whether the files you expect to be included are the ones +that are being included ( e.g. double check that you aren't going to the i386 asm directory ). +2) Check that macro definitions aren't clashing with typedefs, +3) Check that definitons aren't being used before they are being included. +4) Helps put the line emitting the error under the microscope if it contains macros. + +For convenience the Linux kernel's makefile will do preprocessing automatically for you +by suffixing the file you want built with .i ( instead of .o ) + +e.g. +from the linux directory type +make arch/s390/kernel/signal.i +this will build + +s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer +-fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -E arch/s390/kernel/signal.c +> arch/s390/kernel/signal.i + +Now look at signal.i you should see something like. + + +# 1 "/home1/barrow/linux/include/asm/types.h" 1 +typedef unsigned short umode_t; +typedef __signed__ char __s8; +typedef unsigned char __u8; +typedef __signed__ short __s16; +typedef unsigned short __u16; + +If instead you are getting errors further down e.g. +unknown instruction:2515 "move.l" or better still unknown instruction:2515 +"Fixme not implemented yet, call Martin" you are probably are attempting to compile some code +meant for another architecture or code that is simply not implemented, with a fixme statement +stuck into the inline assembly code so that the author of the file now knows he has work to do. +To look at the assembly emitted by gcc just before it is about to call gas ( the gnu assembler ) +use the -S option. +Again for your convenience the Linux kernel's Makefile will hold your hand & +do all this donkey work for you also by building the file with the .s suffix. +e.g. +from the Linux directory type +make arch/s390/kernel/signal.s + +s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer +-fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -S arch/s390/kernel/signal.c +-o arch/s390/kernel/signal.s + + +This will output something like, ( please note the constant pool & the useful comments +in the prologue to give you a hand at interpreting it ). + +.LC54: + .string "misaligned (__u16 *) in __xchg\n" +.LC57: + .string "misaligned (__u32 *) in __xchg\n" +.L$PG1: # Pool sys_sigsuspend +.LC192: + .long -262401 +.LC193: + .long -1 +.LC194: + .long schedule-.L$PG1 +.LC195: + .long do_signal-.L$PG1 + .align 4 +.globl sys_sigsuspend + .type sys_sigsuspend,@function +sys_sigsuspend: +# leaf function 0 +# automatics 16 +# outgoing args 0 +# need frame pointer 0 +# call alloca 0 +# has varargs 0 +# incoming args (stack) 0 +# function length 168 + STM 8,15,32(15) + LR 0,15 + AHI 15,-112 + BASR 13,0 +.L$CO1: AHI 13,.L$PG1-.L$CO1 + ST 0,0(15) + LR 8,2 + N 5,.LC192-.L$PG1(13) + +Adding -g to the above output makes the output even more useful +e.g. typing +make CC:="s390-gcc -g" kernel/sched.s + +which compiles. +s390-gcc -g -D__KERNEL__ -I/home/barrow/linux-2.3/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce -S kernel/sched.c -o kernel/sched.s + +also outputs stabs ( debugger ) info, from this info you can find out the +offsets & sizes of various elements in structures. +e.g. the stab for the structure +struct rlimit { + unsigned long rlim_cur; + unsigned long rlim_max; +}; +is +.stabs "rlimit:T(151,2)=s8rlim_cur:(0,5),0,32;rlim_max:(0,5),32,32;;",128,0,0,0 +from this stab you can see that +rlimit_cur starts at bit offset 0 & is 32 bits in size +rlimit_max starts at bit offset 32 & is 32 bits in size. + + +Debugging Tools: +================ + +objdump +======= +This is a tool with many options the most useful being ( if compiled with -g). +objdump --source <victim program or object file> > <victims debug listing > + + +The whole kernel can be compiled like this ( Doing this will make a 17MB kernel +& a 200 MB listing ) however you have to strip it before building the image +using the strip command to make it a more reasonable size to boot it. + +A source/assembly mixed dump of the kernel can be done with the line +objdump --source vmlinux > vmlinux.lst +Also if the file isn't compiled -g this will output as much debugging information +as it can ( e.g. function names ), however, this is very slow as it spends lots +of time searching for debugging info, the following self explanitory line should be used +instead if the code isn't compiled -g. +objdump --disassemble-all --syms vmlinux > vmlinux.lst +as it is much faster + +As hard drive space is valuble most of us use the following approach. +1) Look at the emitted psw on the console to find the crash address in the kernel. +2) Look at the file System.map ( in the linux directory ) produced when building +the kernel to find the closest address less than the current PSW to find the +offending function. +3) use grep or similar to search the source tree looking for the source file + with this function if you don't know where it is. +4) rebuild this object file with -g on, as an example suppose the file was +( /arch/s390/kernel/signal.o ) +5) Assuming the file with the erroneous function is signal.c Move to the base of the +Linux source tree. +6) rm /arch/s390/kernel/signal.o +7) make /arch/s390/kernel/signal.o +8) watch the gcc command line emitted +9) type it in again or alernatively cut & paste it on the console adding the -g option. +10) objdump --source arch/s390/kernel/signal.o > signal.lst +This will output the source & the assembly intermixed, as the snippet below shows +This will unfortunately output addresses which aren't the same +as the kernel ones you should be able to get around the mental arithmetic +by playing with the --adjust-vma parameter to objdump. + + + + +extern inline void spin_lock(spinlock_t *lp) +{ + a0: 18 34 lr %r3,%r4 + a2: a7 3a 03 bc ahi %r3,956 + __asm__ __volatile(" lhi 1,-1\n" + a6: a7 18 ff ff lhi %r1,-1 + aa: 1f 00 slr %r0,%r0 + ac: ba 01 30 00 cs %r0,%r1,0(%r3) + b0: a7 44 ff fd jm aa <sys_sigsuspend+0x2e> + saveset = current->blocked; + b4: d2 07 f0 68 mvc 104(8,%r15),972(%r4) + b8: 43 cc + return (set->sig[0] & mask) != 0; +} + +6) If debugging under VM go down to that section in the document for more info. + + +I now have a tool which takes the pain out of --adjust-vma +& you are able to do something like +make /arch/s390/kernel/traps.lst +& it automatically generates the correctly relocated entries for +the text segment in traps.lst. +This tool is now standard in linux distro's in scripts/makelst + +strace: +------- +Q. What is it ? +A. It is a tool for intercepting calls to the kernel & logging them +to a file & on the screen. + +Q. What use is it ? +A. You can used it to find out what files a particular program opens. + + + +Example 1 +--------- +If you wanted to know does ping work but didn't have the source +strace ping -c 1 127.0.0.1 +& then look at the man pages for each of the syscalls below, +( In fact this is sometimes easier |