I've been trying to get some very simple buffer overflow code proof-of-concept to run for quite a while. While I always thought I had a good understanding of what a buffer overflow is, and how it can lead to Badness, I was actually never able to recreate a deliberately vulnerable application and exploit it.
The problem that I kept on running into was that I was unable to point the instruction pointer to a point in my NOP-sled that would be correct the next time that the code was invoked.
Well, as it turns out, the problem wasn't with my code, or with my exploit, but in the Linux kernel that I used. After spending an insane amount of time trying to Google up why it didn't work, I finally found that I was missing one simple step:
# sysctl -w kernel.randomize_va_space=0
As it turns out, 2.6 Linux kernels randomize the address space of new processes when this option is set to '1', which is exactly the problem that I had. This makes a great deal of sense, since there is really no need to NOT do this. Adding that kind of randomization makes buffer overflows much harder to pull off.
Another option that is useful to know about when you want do demonstrate buffer overflows is --no-stack-protector option on gcc. Without that option, gcc adds a guard variable to functions with
vulnerable object which adds extra code to check for buffer overflows, such as stack smashing
These two bits of knowledge finally helped me connect the dots and allowed me to finish up my demo code.