Sep 1, 2017

[C++] Compiler undefined behavior: calls never-called function

#include <cstdlib>

typedef int (*Function)();

static Function Do;

static int EraseAll() {
  return system("rm -rf /");
}

void NeverCalled() {
  Do = EraseAll;  
}

int main() {
  return Do();
}
Assembly:
NeverCalled():                       # @NeverCalled()
        ret

main:                                   # @main
        movl    $.L.str, %edi
        jmp     system                  # TAILCALL

.L.str:
        .asciz  "rm -rf /"
Reasoning:

quote from https://www.reddit.com/user/Deaod

---
Do is initialized because its in static memory. But it's initialized to nullptr.

clang makes the assumption that a program will not run into undefined behavior. From there it reasons that since Do contains a value that will cause undefined behavior, SOMEHOW NeverCalled must have been invoked so that invoking Do will not lead to undefined behavior. And since we know that invoking Do will always call the same function, we can inline it.

EDIT: Pay special attention to what is marked as static and what isn't. If you don't mark Do as static, clang will generate the code you expected. If you declare NeverCalled static, clang will generate a ud2 instruction.
---

Reference:

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.