What is Return Oriented Programming?

NX (never execute) or XD (execute disable) is a popular mitigation to buffer overflow attacks and the return oriented programming has come into existence to by-pass the way such technique works.

Return oriented programming(ROP) is a technique of attack which is a successor to another attack technique called ret2libc(Return to libc). Ret2libc replaces the return address to return to an already loaded library in the memory and providing the specific attack inducing data to it. To the system, it appears as a normal system call.

Whereas ROP, instead of passing the control to another function, reuse the already available instructions in the memory to perform the attack. Basically ROP employs different instructions chained together in such a manner that when all instructions finish executing, an attack is perceived, without a single bit of instruction being injected into the code. This effectively make reuse of those instructions in memory pages which are not marked with NX or XD bit.

Different instructions chained together are known as gadgets. Most popular way of ROP is to use those gadgets which end with ret (return) instructions but jmp (jump) and call instructions could also be used. Each gadget or set of gadgets could perform some meaningful task related to attack.

To make the ROP effective, memory regions of popular common shared libraries could be studied and analyzed for the exact memory addresses to be used in making the chain of gadgets. But this technique would be thwarted by used of ASLR (address space layout randomization), still those libraries could be exploited which are not compatible with ASLR.

Generally ROP is used to break the control flow and by-pass or switch off the DEP technique, and later introduce the shellcode, because chaining of gadgets to perform attack largely depends upon the available instructions in memory which becomes difficult as even a single break in the chain would leave the attack ineffective.

Apart from ASLR, stack smashing protection and third party code randomization could also leave ROP ineffective. Since ROP depends mostly on static memory addresses, any technique which prevents this could be used as a countermeasure to ROP attacks.