Introducing an ABI Lowering Library

This summer I'll be working with LLVM as part of Google Summer of Code 2025! Consider me officially G-shocked.

One of LLVM’s most compelling promises is its abstraction over hardware targets. A language frontend just needs to emit LLVM IR, and LLVM handles everything else - from optimization to target-specific code generation. In theory, this abstraction enables frontends to focus on high-level semantics without worrying about the quirks of low-level machine code.

However this abstraction falls apart when C interop is involved.

What is ABI Lowering?

Application Binary Interface (ABI) lowering is the process of transforming high-level function signatures and data structures into the low-level calling conventions.

Think of it as the translation layer between "I want to call this function with these parameters" and "put this value in register RDI, that value on the stack at offset 16, and expect the return value in RAX."

Take the following for instance:

struct Point {
    double x, y;
};

struct Point add_points(struct Point a, struct Point b) {
    return (struct Point){a.x + b.x, a.y + b.y};
}

Now this function passes 2 structs as arguments and returns one. Conceptually this would look something like the following in ABI:

void add_points_lowered(struct Point* return_slot, // the return value becomes a hidden pointer
                       double a_x, double a_y, // Scalarization of Inputs
                       double a_x, double b_y) {
    return_slot->x = a_x + b_x;
    return_slot->y = a_y + b_y;

How Clang Does It

Clang, via it's clang::CodeGen component performs extensive ABI lowering logic in it's frontend. But if LLVM was supposed to be modular, why do it in the frontend? Because C Interop demands it(more on that later).

The clang/lib/CodeGen contains thousands of lines of target specific ABI logic. Here's an example:

/// X86_64ABIInfo - The X86_64 ABI information.
class X86_64ABIInfo : public ABIInfo {
  enum Class {
    Integer = 0,
    SSE,
    SSEUp,
    X87,
    X87Up,
    ComplexX87,
    NoClass,
    Memory
  };
  void classify(QualType T, uint64_t OffsetBase, Class &Lo, Class &Hi,
              bool isNamedArg, bool IsRegCall = false) const;

  ABIArgInfo classifyArgumentType(QualType Ty, unsigned freeIntRegs,
                                  unsigned &neededInt, unsigned &neededSSE,
                                  bool isNamedArg,
                                  bool IsRegCall = false) const;

Now when Clang encounters a function call, it consults these target-specific ABI rules during IR generation:

  /// EmitCall - Generate a call of the given function, expecting the given
  /// result type, and using the given argument list which specifies both the
  /// LLVM arguments and the types they were derived from.
  RValue EmitCall(const CGFunctionInfo &CallInfo, const CGCallee &Callee,
                  ReturnValueSlot ReturnValue, const CallArgList &Args,
                  llvm::CallBase **CallOrInvoke, bool IsMustTail,
                  SourceLocation Loc,
                  bool IsVirtualFunctionPointerThunk = false);

The result of all this is that clang emits pre-lowered target specific IR:

define dso_local { double, double } @add_points(double %0, double %1, double %2, double %3) local_unnamed_addr #0 {
  %5 = fadd double %0, %2
  %6 = fadd double %1, %3
  %7 = insertvalue { double, double } poison, double %5, 0
  %8 = insertvalue { double, double } %7, double %6, 1
  ret { double, double } %8
}

What is Interop?

Interoperability is the ability for code written in different languages to call each other seamlessly. In systems programming, this almost always means "can your language talk to C?" since C established the de facto standard for system interfaces.

At its core, this interoperability is enabled by Foreign Function Interface (FFI) - the mechanism that allows code written in one programming language to call functions defined in another. FFI handles the translation between different calling conventions, data types, and memory layouts that each language uses internally. It's the bridge that makes it possible for a Python script to call a C library, Rust code to invoke system APIs, or JavaScript to interact with native modules. Without FFI, every language would exist in isolation, unable to leverage the vast ecosystem of existing libraries and system interfaces that power modern software.

When we say "C interop", we're really talking about C ABI compatibility - your language needs to produce function calls that look exactly like what a C compiler would generate for the same signature.

To clearly exemplify:

// C declaration in stdlib.h
void qsort(void *base, size_t nitems, size_t size,
           int (*compar)(const void *, const void *));

Now Zig is a systems programming language with great, clean C interop. We could include stdlib.h using the @cImport directive which internally translates C declarations to their corresponding Zig counterparts.

pub extern fn qsort(__base: ?*anyopaque, __nmemb: usize, __size: usize, __compar: __compar_fn_t) void;

// Implies
extern "c" fn qsort(
    base: ?*anyopaque, 
    nitems: c_ulong, 
    size: c_ulong,
    compar: *const fn (?*const anyopaque, ?*const anyopaque) callconv(.C) c_int
) void;

Zig has its own calling convention and ABI. The extern "C" and callconv(.C) annotations tell Zig to use the C calling convention instead of its default calling convention.

Where the Abstraction Breaks

While the promise is that frontends just have to emit high level IR and delegate low-level concerns to the backend, the current implementation forces frontends to be very involved with target specific ABI implementation details.

The core issue lies in calling convention responsibility. LLVM backends handle register allocation and instruction selection, but the semantic transformation of function signatures, determining how composite types are passed, when to use hidden return parameters, and how to decompose aggregate arguments - remains a frontend concern. This division of labor creates a fundamental tension: frontends must possess deep target-specific knowledge that theoretically should be abstracted away.

Each target architecture multiplies this complexity. ARM's AAPCS has different structure passing rules, different register usage patterns, and different alignment requirements. PowerPC introduces big-endian considerations and different floating-point handling. The WebAssembly target has entirely different concepts of function signatures and memory layout.

The precision required is absolute. A single misclassification by passing a structure by value when the ABI demands indirect passing, or incorrectly handling the alignment of packed structures, can cause silent data corruption, stack misalignment, or segmentation faults that manifest only under specific optimization levels or compiler versions.

This ABI burden represents a significant barrier to language implementation. Every frontend that requires C interoperability must either implement comprehensive ABI logic for each target or sacrifice portability. The alternative, which is, linking against C-specific runtime libraries or using external binding generators, introduces additional complexity and potential points of failure.

So when someone asks "How can I support C FFI" the unfortunate answer is to ask them to reimplement around 10000 lines of Clang code.

A Shared ABI Lowering Library

Here's a cool graphic to help you understand:

The solution to this flaw, as you may have guessed, is pretty straightforward: extract clang's ABI lowering logic into a shared library that all LLVM frontends call into. Rather than expecting every frontend to reimplement, which might even cause subtle differences between them, this library could serve as the single authoritative source of ABI knowledge.

If that wasn't enough, Clang's ABI Implementation represents over a decade of accumulated knowledge and bug fixes. When there is a subtle change in the ABI rules of an architecture, the information is encoded into clang and can benefit the entire ecosystem.

The new LLVM-ABI library would act as a translation layer between high level semantics and LLVM IR. The library would convert frontend types (like clang QualType) to an independant typesystem which captures ABI related information like alignments and packing and send back explicit instructions on how to lower to LLVM IR.

Consider the earlier add_points example. Instead of a Zig frontend having to know that x86-64 SysV ABI passes small structs in registers while larger ones go on the stack, it would simply describe the Point struct to LLVMABI and receive back an ABIArgInfo specifying the exact lowering strategy.

The library's type system would be custom-built for ABI classification. Unlike LLVM IR types, which are optimized for transformation and analysis, these types would preserve exactly the information needed for correct ABI decisions. A __int128 and a _BitInt(128) would remain distinct. Packed structs would retain their packing constraints. Unions would be represented explicitly rather than as opaque byte arrays.

And lastly since even clang will be a consumer of this library, there will always remain a canonical reference implementation which would guarantee that the library stays current with the ABI changes and edge cases that emerge from real-world usage.

Closing Thoughts

The library needs to be fast enough that Clang can adopt it without performance regression, simple enough that small frontends can integrate it easily, and complete enough to handle real-world ABI complexity. These goals often conflict, and it'll be interesting to navigate this in the coming weeks.

Current ABI lowering stops at generating the right LLVM IR function signatures. But there's appetite for going further—computing actual register assignments and stack offsets that could eventually move ABI handling entirely out of LLVM backends. This would enable more flexible custom calling conventions and potentially unlock new optimization opportunities.

I plan to explore these questions through prototyping and feedback from my mentors Nikita Popov and Maksim Levental through the GSoC period and write a follow-up when I have something substantial to say.

If you were curious on what the community has to say about this topic consider exploring the RFC.