Skip to content

Revise FunctionHook API to require function prototype information (.NET 9+) #39

@alexrp

Description

@alexrp

This is a very long term goal.

The idea is to require the user to somehow communicate the function prototype to the API. The hook gate implementation can then use that information to perform a proper call to the hook function instead of the return address hijacking we do now:

// We now know that we are going to call the hook function. Hijack the current return address so
// that the hook function returns to the code below. This gives us the opportunity to exit through
// the gate. RAX contains the hook function address, so use RCX as scratch here.
asm.lea(rcx, __qword_ptr[exit]);
asm.mov(__qword_ptr[rsp + frameSize], rcx);
asm.jmp(done);

We would also be able to inject extra arguments in the call to the hook function, which might be useful in some scenarios.

There are two approaches we could take:

  • Take a Type parameter in FunctionHook.Create() that must be a function pointer type.
  • Change FunctionHook to have a type parameter TFunction that must be a function pointer type.

The second approach is preferable as it would enable the TargetCode, HookCode, and TrampolineCode properties to be typed properly, cutting down on error-prone casts in user code. This would require both dotnet/runtime#13627 and dotnet/runtime#69273.

The first approach would only require dotnet/runtime#69273 (which seems very likely to arrive in .NET 8), but it also doesn't get us the type safety that the second approach does.

We'll have to wait for .NET 8 and see how things look to determine what we'll do here.

Metadata

Metadata

Assignees

Labels

area: hookingIssues related to the function hooking APIs.area: samplesIssues related to the sample projects.state: blockedIssues that are blocked on some other issue or work.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions