interop

The interop module provides necessary facilities for interaction with the server via standard natives and callbacks. Using the provided API, it is possible to achieve the same results like with Pawn, and more.

All tables and functions in this module are linked together by reference, not by name. For example, if you set interop.public to nil, the plugin will still use the original table, even though it may be inaccessible to Lua code. This is useful for sandboxing or frameworks, since one can restrict the access to parts of the YALP Lua API without affecting its functionality.

Introduction

When the interop module is loaded, YALP creates a "fake" in-memory AMX program (unless lua_bind is used) and tries to load it as a filterscript. If it succeeds, the filterscript is used as the link between Lua and the server. All AMX calls to the filterscript are hooked with custom handlers that translate various requests to the Lua machine. Conversely, calling natives uses the actual AMX stack of the virtual program.

Since there is a limit of at most 16 filterscripts, you need to be aware of the number of Lua machines you create. The filterscript is automatically unloaded when the machine is closed, but it is better to use a smaller number of Lua states, since each can run any number of scripts.

If the interop module is loaded after lua_bind is used, it will attach the Lua state to the AMX machine that made the call to lua_bind. In this case, no additional filterscript is loaded and all AMX calls to the machine are handled by YALP. This allows using the module in a gamemode as well.

Calling natives

All natives the server and plugins expose to the virtual filterscript are registered by their name, making it easy to dynamically search for one. To minimize the number of lookups and to simplify the calls, there is a special table interop.native that automatically looks for a native when indexed, and caches the located functions.

interop.native.SendClientMessageToAll(-1, "Hello from Lua!")

There is no need to "declare" a native; simply calling it with specific arguments will automatically convert them to values expected by the server. This means that if you call a native with parameters of incorrect type and the native doesn't check their validity, it could potentially corrupt the server's memory or crash it.

By default, the call to a native function results in an opaque value, since the tag of the result is not returned with the value. To convert the value to a correct type, use the conversion functions in the interop module:

local ping = interop.asinteger(interop.native.GetPlayerPing(playerid))

To decrease the complexity of the expression, you can also specify the conversion function as the first argument:

local ping = interop.native.GetPlayerPing(interop.asinteger, playerid)

This also has the advantage that if the caller doesn't request any return value from the call, the conversion function is not called at all:

interop.native.GetPlayerPing(print, playerid) -- print is not called

In order to pass Lua values to the native function, marshalling must take place. This means that for every possible Lua value, YALP must decide how to transform it to an AMX cell (32-bit integer) before passing it. The following table specifies how various Lua types are marshalled:

Lua type	Marshalling method
number	32-bit integer or float, depending on the numeric type
boolean	1 or 0, depending on the value
light userdata	as is (light userdata are 32-bit)
string	copied to the heap as a packed string, address is passed
buffer	the address is passed directly
const buffer	data copied to the heap, address is passed
table	treated as a sequence, contents marshalled to the heap and copied back after the call
nil	only allowed in trailing position, not passed at all

Other arguments cannot be accurately represented and will produce an error if passed. Note that Lua represents both integers and floats as subtypes of a single number type, which usually works fine most of the time, but if a call to a native function is performed with an integer value stored as a float number, it will be treated as a float number.

Natives with variable number of arguments cannot be provided with simple values, since all of them are passed by reference only; use buffers in this case (more information below). Also note that some natives may fail to correctly process packed strings (printf, for example), but you can use buffers as well to pass unpacked strings.

Although all characters from a string will be copied to the heap, AMX will probably find the string end at the first null character, unless the used native function allows for another parameter that specified the length.

There are several conversion functions defined to help you translate the value returned from the function to different Lua types:

Conversion function	Result
asnone	no value is returned
asnil	only nil is returned
asinteger	signed 32-bit integer
asuinteger	unsigned 32-bit integer
asfloat	32-bit floating point number
asboolean	false for 0, true otherwise
asstring	AMX string reference (in natives usually for plugin API only)
asoffset	an offset into a byte buffer; it is divided by 4 and added to 1
ashandle	converted to nil for 0, a light userdata otherwise

Callbacks

Callbacks are registered in a similar way as in Pawn, using a public table. When the AMX is queried for a public function by its name, YALP looks into the interop.public table if it contains this function, and assigns a unique index to its name. The function can be replaced or removed (by setting it to nil), but the index will stay the same forever.

When the public function is called, YALP treats all arguments as light userdata and propagates them to the Lua function. The arguments can be converted back to normal values using the appropriate conversion functions (like for natives). Since strings can be both input and output, YALP cannot take any guesses as to whether automatically convert them to Lua strings or not (despite the usage of the AMX API can be used as a hint to that).

Passing strings to public functions in AMX is done via the heap. The string is copied to the heap and its address is provided to the function. To convert it to a Lua string, interop.asstring is enough.

function public.OnPlayerText(playerid, text)
  playerid = interop.asinteger(playerid)
  text = interop.asstring(text)
  return true
end

Returned value is marshalled similarly to parameters, with two exceptions. Strings cannot be returned, because callbacks do not support their direct returning (only via output parameters), and nil (or no return) is interpreted as 0. Returning an unsupported value prints a warning, and other return values are ignored.

Buffers

Since Lua, unlike Pawn, has no concept of by-reference parameters, buffers are used to represent them. A buffer is a continuous block of cells in memory (an array of cells basically), which can be used as an argument for native calls where an array or by-ref parameter is expected.

A buffer can be created independent of any AMX machine, which is useful for more permanent buffers, or allocated on the AMX heap, suitable for temporary calls. Buffers are allocated in the memory reserved for the Lua state and are automatically collected if unused.

local buf = interop.newbuffer(1) -- a buffer of 1 cell
interop.native.GetPlayerHealth(playerid, buf) -- used as the output of the native
return interop.asfloat(buf[1]) -- indexed from 1, like tables
-- no need to free the buffer, Lua GC will collect it

local buf = interop.heapalloc(1) -- allocated on the AMX heap
interop.native.GetPlayerHealth(playerid, buf)
local value = interop.asfloat(buf[1])
interop.heapfree(buf) -- must be freed, or the heap will grow over time
return value

A buffer can also be read-only (const buffer). interop.tocbuffer creates a read-only buffer from a value. This is useful in cases where it is necessary to prevent something changing the contents of the buffer. In case of native calls, for example, the constness is upheld by copying all data from the buffer into the heap on a temporarily allocated space.

interop.heap is actually also a buffer, but bound to the heap of the virtual AMX machine. Therefore, is may shrink or grow as space is allocated or released, respectively. Since the heap is also a continuous block of memory, freeing an allocated space also frees all the space that was allocated after it.

Buffers can be segmented further into spans. A span is a range inside the buffer, determined by its offset and length, that can be used to limit the access to the whole buffer, or to make it read-only. It can also be used to prevent unnecessary copying:

function interop.public.OnRconCommand(cmd)
  interop.native.print(interop.span(interop.heap, cmd))
  -- print expects an address of the string, so in this case, simply passing `cmd` is sufficient as well.
end

Buffers are useful when calling native functions with variable number of arguments (varargs). Their arguments must be passed indirectly via references, so storing the value in a buffer is necessary. There are several methods for simplifying this, each with its own advantages and drawbacks.

interop.natives.printf("%d %f %s", interop.tobuffer(5, -1.0, "str"))
-- creates a new buffer for every argument; results in memory fragmentation
interop.natives.printf("%d %f %s", interop.tocbuffer(5, -1.0, "str"))
-- making the buffer const requires the values to be copied to the heap before calling the native; very inefficient
interop.natives.printf("%d %f %s", interop.struct(5, -1.0, "str"))
-- creates a single buffer to hold all the arguments, and returns spans to them; the data is kept together when accessed from the native function
local pin = interop.heapalloc(0) -- creates a single span to represent the beginning of arguments
interop.natives.printf("%d %f %s", interop.heapalloc(5, -1.0, "str"))
interop.heapfree(pin) -- heap must be restored to its former state, or it will continue to grow
-- heapalloc allocates new space for all arguments on the heap and returns their spans
local first = interop.heapargs(5) -- allocates a single cell (5) on the heap and returns its address (not span)
interop.natives.printf("%d %f %s", first, interop.heapargs(-1.0, "str"))
interop.heapfree(first) -- also accepts an address
-- heapargs only returns the addresses without creating any spans, so this creates no additional objects at all

Choosing the best method depends on the number and types of arguments. It is inefficient to use heapargs on buffers, for example, since it always copies their contents. To address this issue, there is interop.vacall doing all the work automatically:

interop.vacall(interop.natives.printf, "%d %f %s")(5, -1.0, "str")

Calling the function itself caches all intermediate arguments, and returns a new function to be called with the variable arguments. This function will allocate space on the heap for all simple values, but keep the buffer arguments intact. When the native function is called, it only receives the addresses of the arguments on the heap. After it returns, the heap is first restored, and then the function call ends (or an error is propagated).

This fuction can be also used in cases where a function returns one or more values by reference. It is very efficient and safe to use it for functions like GetPlayerHealth or GetPlayerPos:

local x, y, z = interop.vacall(interop.native.GetPlayerPos, interop.asnone, playerid)(0.0, 0.0, 0.0)

The call remembers the types and number of the three 0.0 arguments, and when the native function ends, it retrieves the values from the heap before freeing it. The new values are then appended to whatever the original function returned (nothing in this case).

Loopback

Some plugins may require to be called from inside a public function. Normal Lua chunks are not run from any AMX machine, and so calling specific natives could cause issues. For this case, there is a special "loopback" public function which can be called from inside the virtual AMX. It has a special name, identified by interop.loopback (might contain an arbitrary, possibly even random, name). YALP exports a special Pawn native function lua_loopback which looks for this public function, and calls it via standard means.

Usually, it is only needed to call this function temporarily, for which case there is another function present, interop.forward. It can be defined using Lua in this way:

function interop.forward(func, ...)
  local old = interop.public[interop.loopback] -- remember the old handler
  local args = table.pack(...) -- store the arguments
  local rets
  interop.public[interop.loopback] = function()
    interop.public[interop.loopback] = old -- restore the old handler
    rets = table.pack(func(table.unpack(args, 1, args.n))) -- call the function and store the results
  end
  interop.native.lua_loopback() -- jump inside the loopback public function
  if rets then
    return table.unpack(rets, 1, rets.n)
  end
end

This function stores the original handler and registers a new one that will restore the original and call the provided function. interop.forward is implemented in C++ and so does handle the arguments more efficiently (without additional tables, if possible).

Public variables

Public variables are not used much by plugins, but in case they are needed, YALP also supports them. The interop.public table can be used to register public variables as well. However, all public variables must be represented as buffers, since Lua variables cannot be accessed directly via pointers.

Tags

Some plugins may rely on using the tagof operator in Pawn. YALP fully mimics the AMX tag API and allows Lua code to dynamically create any tag needed. interop.tagof can take either a string argument, in which case it finds or registers a new tag with its name (a strong tag if it begins on an uppercase letter), and returns its code (as light userdata). For an argument of other type, it behaves in accordance with Pawn. Float numbers are treated as Float, boolean values as bool, and other marshallable values are tagless (returns 0x80000000).

In AMX programs compiled from Pawn, tag id and tag index usually do not match, because the tag is created for each used tag, while only exported ("public") tags have an index. In the fake AMX created by YALP, tag id and tag index always match (only the tag id has its upper flags set if it's strong).

Plugins should take into account that the tag table may change at runtime and should not cache it, or at least update it when a new tag is added (YALP never removes a tag from the table).

Sleep

Some plugins might use the AMX_ERR_SLEEP error code to pause the execution of the script and resume it on a specific event. Lua actually offers the same thing in the form of coroutines, and YALP allows bridging the two mechanisms in an easy way.

When a native function returns the sleep error code, three outcomes can happen. If the native function is not executed inside any public function, the execution ends with an error. If the function is executed in a public function, the function is stopped, and the error code is propagated to the caller. However, if the code is actually running in a (yieldable) coroutine, two values are yielded from the from it: interop.sleep, and the return value from the native function (which is usually used to specify the continuation event). interop.sleep should be then provided with the return code and a continuation function, which it references in the CIP register of the fake AMX. The error code is then propagated to the caller which should save the registers necessary to continue the execution. When the code is resumed, the continuation is called and could resume the coroutine.

To illustrate how it works, natives from PawnPlus are used:

interop.forward(function()
  local c = coroutine.create(function()
    interop.native.wait_ms(100)
    print("after 100 ms")
  end)
  local ok, sleep, ret = coroutine.resume(c)
  sleep(function(arg)coroutine.resume(c, arg)end, ret)  
end)

wait_ms yields sleep and ret, which is linked with the continuation function when sleep is called. If the caller correctly stores and restores the registers, the continuation is properly called, and the coroutine is resumed.

This kind of pattern can be represented with another function, called async (provided in the base module):

function async(func, ...)
  local c = coroutine.create(func)
  
  local function cont(...)
    local sleep, ret = select(2, assert(coroutine.resume(c, ...)))
    if coroutine.status(c) == "suspended" then
      return sleep(cont, ret)
    end
    return sleep, ret
  end
  
  return cont(...)
end

This function is implemented again in native code, so it handles correctly all returned values or errors and is optimized better.

Combined with interop.forward, it is really easy to make asynchronous code:

interop.forward(async, function()
  interop.native.wait_ms(100)
  print("after 100 ms")
end)

The async function can work with other functions, for example using the timer module:

async(function()
  coroutine.yield(timer.ms, 100)
  print("after 100 ms")
end)

Strings

Strings in Pawn are represented in two ways via cell arrays: packed and unpacked. Packed strings can store 4 characters in a single (32-bit) cell, while unpacked strings take a single cell for every character. In order for any function to determine if a string is packed or not, cells in packed strings are filled with characters from the highest to the lowest byte, i.e. "0123" corresponds to 0x30313233. Unpacked strings can be used for UTF-32 strings, but since SA-MP has no use for them and most native functions work fine with packed strings, YALP marshals all strings by default as packed, to save space.

There are 4 functions in the interop package that work directly with strings. setstring(buf, str, offset=1, len=-1, packed=true) stores a string in a buffer, while getstring(buf, offset=1, len=-1) retrieves a string from a buffer (packed or not). There is also asstring intended mainly for use within public function, which receives a string address in the AMX machine, and converts it to a string.

For creating unpacked strings, there is another function, tocellstring, that does precisely this. It creates a new buffer to hold the string and stores every character in each cell of the buffer. It can also be used for strings which contain null characters, since AMX API functions mark these as the end of the string. To circumvent this check, tocellstring assigns 0xFFFF00 to the cell to store the null character in a cell, but make it non-zero. Standard native functions should correctly treat this cell as a non-terminating null character:

local str = "abc\0def"
print(#str, interop.native.strlen(str), interop.native.strlen(interop.tocellstring(str)))
-- 7, 3, 7