Stack Smashing in AMD64
okay, so I can't tell if I am being a complete idiot or not(I am going with the idiot hypothesis), but I am having problems with overwriting the .dtors section of some code in a wargame I am playing. I think the problem is that I am trying to do so on the AMD64 architecture. After some digging (reading the damn manual), I found that the rbp register, the AMD64 'equivalent' to the ebp register, is not..... what's the word.... well the rbp functionality isn't as rigid as the ebp register in the IA32 architecture. As in it is more along the lines of an optional ebp register is how I am looking at it. Does this make smashing the stack on AMD64 completely asinine? I even tried writing a more simple stack smashing exercise and couldn't get positive results. Well, I hope you guys can help!
Are you having problems programming a game or having problems passing a game?
This used to happen to me all the time on the AMD64 architecture, especially when using IDA Pro Disassembler, W32DASM, and pretty much any similar tools. Lots and lots of floating point crashes and overall bad compatibility and stability....Won't touch AMD ever again. You could try to achieve the overwrite in Virtualbox or VMWare to see if their architecture would allow it. Good luck!