How to inline x64 asm in vs2017 ?

[gigaman]

You normally don't align stack like that.
You know that the caller has (according to the calling convention) taken care of its stack alignment and therefore the RSP on entry ends by 8 (the stack was 16B aligned before and then the return address has been pushed there by the CALL).
So the initial PUSH RBP has aligned the stack to 16B again, SUB RSP, 32 didn't break the alignment - and the AND instruction is useless, RSP is already aligned there.

[chants]

Originally Posted by gigaman

You normally don't align stack like that.
You know that the caller has (according to the calling convention) taken care of its stack alignment.

If your asm function is designed to be called from C only then yes I suppose that is a fair assumption. But in fact if it is called from asm including your own or I suppose unknown callers, then it is a false assumption. And this code is known.

The extra 8 bytes comes from having called your own function within proper convention asm already or the return address in case of Windows ABI invocation. CALL F1 in an improperly aligned routine of course adds 8 bytes to the stack and then adding 8 again would misalign it (hence the code examples leaked out there and above which show 40 byte assuming misalignment by an internal call already from ASM despite not having this in examples). The safest assumption is to assume any caller, and realign the stack with an AND RSP, -16 or even to just do it on just the lower 32-bit ESP.

Linux has the same issue even with 32-bit code for late GCC versions as seen in this discussion: Calling printf in extended inline ASM

https://stackoverflow.com/questions/37502841/calling-printf-in-extended-inline-asm/37503773

Even better here is a book on the issue for more in depth detail though unfortunately not listing the generic solution and instead letting you make assumptions or track things:

https://github.com/simon-whitehead/assembly-fun/tree/master/windows-x64

As with the AMD64/SystemV ABI, the Windows ABI dictates that the stack should be aligned on a 16-byte boundary. What this means is that, at the conclusion of the prologue of a function, the memory address that rsp points to should be aligned on a memory address that is a multiple of 16.
The simple act of calling a function misaligns the stack by placing an 8 byte return address on the stack when entering a function.
...
The 64-bit Windows ABI specifies that every single non-leaf function must allocate 32 bytes of stack space for "register spill". This is commonly referred to as "Shadow Space" and must be adjacent to the return address to the previous function. The ABI states that it is the callers job to allocate this stack space, and not the callee. The stack must also always be 16 byte aligned, which can be confusing because on entry to a function the last entry in the stack is the return address of the preview function - which is already 8 bytes. Therefore, for a function to allocate 32 bytes of "Shadow Space" and keep the stack aligned, it must allocate 40 bytes (40 + 8 = 48, which is a multiple of 16).