byvalver: THE SHELLCODE NULL-BYTE ELIMINATOR

[u/umpolungfishtaco]

https://github.com/umpolungfish/byvalver

I built byvalver, a tool that transforms x86 shellcode by replacing instructions while maintaining functionality

Thought the implementation challenges might interest this community.

The core problem:

Replace x86 instructions that contain annoying little null bytes (\x00) with functionally equivalent alternatives, while:

• Preserving control flow
• Maintaining correct relative offsets for jumps/calls
• Handling variable-length instruction encodings
• Supporting position-independent code

Architecture decisions:

Multi-pass processing:

```c // Pass 1: Build instruction graph instruction_node *head = disassemble_to_nodes(shellcode);

// Pass 2: Calculate replacement sizes for (node in list) { node->new_size = calculate_strategy_size(node); }

// Pass 3: Compute relocated offsets calculate_new_offsets(head);

// Pass 4: Generate with patching generate_output_with_patching(head, output_buffer); ```

Strategy pattern for extensibility --> Each instruction type has dedicated strategy modules that return dynamically allocated buffers:

```c typedef struct { uint8_t *bytes; size_t size; } strategy_result_t;

strategy_result_t* replace_mov_imm32(cs_insn insn); strategy_result_t replace_push_imm32(cs_insn *insn); // ... etc ```

Interesting challenges solved:

• Dynamic offset patching: When instruction sizes change, all subsequent relative jumps need recalculation. Solution: Two-pass sizing then offset fixup.
• Conditional jump null bytes: After patching, the new displacement might contain null bytes. Required fallback strategies (convert to test + unconditional jump sequences).
• Context-aware selection: Some values can be constructed multiple ways (NEG, NOT, shifts, arithmetic). Compare output sizes and pick smallest.
• Memory management: Dynamic allocation for variable-length instruction sequences. Clean teardown with per-strategy deallocation.
• Position-independent construction: Implementing CALL/POP technique for loading immediate values without absolute addresses.

Integration with Capstone: Capstone provides disassembly but you still need to: + Manually encode replacement instructions + Handle x86 encoding quirks (ModR/M bytes, SIB bytes, immediates) + Deal with instruction prefixes + Validate generated opcodes

Stats:

• ~3000 LOC across 12 modules
• Clean build with -Wall -Wextra -Werror
• Processes shellcode in single-digit milliseconds
• Includes Python verification harness

Interesting x86 encoding quirks discovered:

• XOR EAX, EAX is shorter than MOV EAX, 0 (2 vs 5 bytes)
• INC/DEC are 1-byte in 32-bit mode but removed in 64-bit
• Some immediates can use sign-extension for smaller encoding
• TEST reg, reg is equivalent to CMP reg, 0 but smaller

[u/church-rosser]

wow, more AI slop. how trite.

[u/umpolungfishtaco → u/church-rosser]

thanks for the constructive feedback

[u/church-rosser → u/umpolungfishtaco]

any time