SourcePawn Transitional Syntax

EXPERIMENTAL: This document is a prototype and not yet complete.

We would like to provide our users a more modern language. Pawn is showing its age; manual memory management, buffers, tags, and lack of object-oriented API are very frustrating for developers. We can't solve everything all at once, but we can begin to take steps in the right direction.

SourceMod 1.7 introduces what we call the "Transitional API". It is built on a new "Transitional Syntax" in SourcePawn, which is a set of language tools to make Pawn feel more modern. In particular, it allows developers to use older APIs in an object-oriented manner, without breaking compatibility. Someday, if and when SourcePawn can become a full-fledged modern language, the transitional API will making porting efforts very minimal.

The transitional API, so far, has the following major features and changes:

• New Declarators - A cleaner way of declaring variables.
• Methodmaps - Object-oriented wrappers around existing API.
• Classes - Stricter API that eliminates the Handle system.

Methodmaps and classes are very similar, and to the user will look almost identical. The major difference is that methodmaps are an extension of the existing API and tag system. Classes actually introduce new types that have strict behavior. All new APIs will be introduced as classes, and our goal is to migrate older APIs into methodmaps.

New Declarators

Pawn was designed for embedded CPUs and microcontrollers. It does not have a formal type system, and the way variables are declared reflects this. For example,

new Float:x = 5.0;

In Pawn, this is not a variable x of type "Float". It is a 32-bit "cell" tagged as a float. It is possible to remove the tag or change the tag, which is almost unheard of in programming languages. While tagging is flexible, it is dangerous and confusing. The syntax itself is also problematic. The parser does not have the ability to identify characters in between the tag name and the colon. Internally, the compiler cannot represent values that are not exactly 32-bit.

The takeaway message is: there is no sensible way to represent a concept like "int64" or "X is a type that represents an array of floats". The tagging grammar makes it too awkward, and the compiler itself is incapable of attaching such information to a tag. We can't fix that right away, but we can begin to deprecate the tag system via a more normal declaration syntax:

float x = 5.0;    // Replacement for Float
int y = 4;        // Replacement for _
char name[32];    // Replacement for String
 
void DoStuff(float x, int y, char name[]) {
}

Note the crucial difference in this syntax: it will, in the future, allow us to have method signatures like this:

float[3] GetVector();

There is no sane way to express this signature with tags. This syntax will also free up the "new" keyword, and bring Pawn closer in line with languages like C# and Java.

Grammar

The new and old declaration grammar is below.

return-type ::= return-old | return-new
return-new ::= type-expr new-dims?        // Note, dims not yet supported.
return-old ::= old-dims? label?

argdecl ::= arg-old | arg-new
arg-new ::= "const"? type-expr '&'? symbol old-dims? ('=' arg-init)?
arg-old ::= "const"? tags? '&'? symbol old-dims? ('=' arg-init)?

vardecl ::= var-old | var-new
var-new ::= var-new-prefix type-expr symbol old-dims?
var-new-prefix ::= "static" | "const"
var-old ::= var-old-prefix tag? symbol old-dims?
var-old-prefix ::= "new" | "decl" | "static" | "const"

global ::= global-old | global-new
global-new ::= storage-class* type-expr symbol old-dims?
global-old ::= storage-class* tag? symbol old-dims?

storage-class ::= "public" | "static" | "const" | "stock"

type-expr ::= (builtin-type | symbol) new-dims?
builtin-type ::= "void"
               | "int"
               | "float"
               | "char"
               | "bool"

tags ::= tag-vector | tag
tag-vector ::= '{' symbol (',' symbol)* '}' ':'
tag ::= label

new-dims ::= ('[' ']')*
old-dims ::= ('[' expr? ']')+

label ::= symbol ':'
symbol ::= [A-Za-z_]([A-Za-z0-9_]*)

Note that arrays can either be declared next to the type (like "int[]"), or after the variable name (like "int x[]"). The latter style is deeply embedded in the Pawn community, so it will continue to work. However, associating the dimensions with the type is another important step forward. First, it regularizes the positioning when a name isn't needed. Second, another long-term goal is eliminating the need for fixed-length, pre-allocated arrays in the API, and those are one of the only compelling reasons to use the existing syntax.

However, it is illegal to have dimensions in both positions.

Also note, there is no equivalent of decl in the new declarator syntax. decl is considered to be dangerous and unnecessary.

Methodmaps

Introduction

Methodmaps are simple: they attach methods onto a tag. For example, here is our legacy API for Handles:

native CloneHandle(Handle handle);
native CloseHandle(Handle handle);

This is a good example of our legacy API. Using it generally looks something like:

Handle array = CreateAdtArray();
PushArrayCell(array, 4);
CloseHandle(array);

Gross! A Methodmap can clean it up by attach functions to the Handle tag, like this:

methodmap Handle {
    public Clone() = CloneHandle;
    public Close() = CloseHandle;
};

Now, our earlier array code can start to look object-oriented:

Handle array = CreateAdtArray();
PushArrayCell(array, 4);
array.Close();

Inheritance

The Handle system has a "weak" hierarchy. All handles can be passed to CloseHandle, but only AdtArray handles can be passed to functions like PushArrayCell. This hierarchy is not enforced via tags (unfortunately), but instead by run-time checks. Methodmaps allow us to make individual handle types object-oriented, while also moving type-checks into the compiler.

For example, here is a transitional API for arrays:

native AdtArray CreateAdtArray();
 
methodmap AdtArray < Handle {
    public PushCell() = PushArrayCell;
};

Note that CreateAdtArray now returns AdtArray instead of Handle. Normally that would break older code, but since AdtArray inherits from Handle, there is a special rule in the type system that allows coercing an AdtArray to a Handle (but not vice versa).

Now, the API looks much more object-oriented:

AdtArray array = CreateAdtArray();
array.PushCell(4);
array.Close();

Inline Methods

Methodmaps can declare inline methods and accessors. Inline methods can be either natives or Pawn functions. For example:

methodmap AdtArray {
    public native void PushCell(value);
};

This example requires that an "AdtArray.PushCell" native exists somewhere in SourceMod. It has a magic initial parameter called "this", so the signature will look something like:

native void AdtArray.PushCell(AdtArray this, value);

(Of course, this exact signature will not appear in an include file - it's the signature that the C++ implementation should expect, however.)

It's also possible to define new functions without a native:

methodmap AdtArray {
    public native void PushCell(value);
 
    public void PushCells(list[], count) {
        for (int i = 0; i < count; i++) {
            this.PushCell(i);
        }
    }
};

Lastly, we can also define accessors. Currently only "get" accessors are available. For example,

methodmap AdtArray {
    property int Size {
        public get() = GetArraySize;
    }
    property bool Empty {
        public get() {
            return this.Size == 0;
        }
    }
    property int Capacity {
        public native get();
    }
};

The first accessor simply assigns an existing function as an accessor for "Size". The second accessor is an inline method with an implicit "this" parameter. The third accessor will bind to a native with the following name and signature:

native int AdtArray.Capacity.get(AdtArray this);

Custom Tags

Methodmaps don't have to be used with Handles. It is possible to define custom methodmaps on new or existing tags. For example:

methodmap AdminId {
    public int Rights() {
        return GetAdminFlags(this);
    }
};

Now, for example, it is possible to do:

GetPlayerAdmin(id).Rights()

Constructors and Destructors

Methodmaps can also define constructors and destructors, which is useful if they are intended to behave like actual objects. For example,

methodmap AdtArray {
    public AdtArray(blocksize = 1);
    public ~AdtArray();
    public void PushCell(value);
};

Now AdtArrays can be used in a fully object-oriented style:

AdtArray array = AdtArray();
array.PushCell(10);
array.PushCell(20);
array.PushCell(30);
delete array;

Caveats

There are a few caveats to methodmaps:

1. CloseHandle() is not yet gone. It is required to call delete on any object that previously would have required CloseHandle().
2. There can be only one methodmap for a tag.
3. When using existing natives, the first parameter of the native must coerce to the tag of the methodmap. Tag mismatches of the "this" parameter will result in an error. Not a warning!
4. Methodmaps can only be defined on tags. Pawn has some ways of creating actual types (like via struct or class). Methodmaps cannot be created on those types.
5. Methodmaps do not have strong typing. For example, it is still possible to perform "illegal" casts like Float:CreateAdtArray(). This is necessary for backwards compatibility, so methodmap values can flow into natives like PrintToServer or CreateTimer.
6. It is not possible to inherit from anything other than another previously declared methodmap.
7. Methodmaps can only be defined over scalars - that is, the "this" parameter can never be an array. This means they cannot be used for enum-structs.
8. Destructors can only be native. When we are able to achieve garbage collection, delete will do nothing.
9. The signatures of methodmaps must use the new declaration syntax.

Grammar

The grammar for methodmaps is:

visibility ::= "public"
method-args ::= arg-new* "..."?

methodmap ::= "methodmap" symbol? { methodmap-item* } term
methodmap-item ::=
           visibility "~"? symbol "(" ")" "=" symbol term
         | visibility "native" type-expr "~"? symbol methodmap-symbol "(" method-args ")" term
         | visibility type-expr symbol "(" method-args ")" func-body term
         | "property" type-expr symbol { property-decl } term
property-func ::= "get"
property-decl ::= visibility property-impl
property-impl ::=
           "native" property-func "(" ")" term
         | property-func "(" ")" func-body term
         | property-func "(" ")" "=" symbol

Classes

Classes are almost identical to methodmaps, with one very important distinction: they are actual types, rather than an attachment to a tag. In fact, classes form an entirely new type system in Pawn. For example, the following is possible with methodmaps:

native Handle CreateTimer(float interval, Timer func, any data = 0);
 
methodmap AdtArray {
    public AdtArray(blocksize = 1);
    public ~AdtArray();
};
 
public CheckPlayers(Handle timer, any data)
{
    ....
}
 
AdtArray array = AdtArray();
CreateTimer(2.0, CheckPlayers, array);

If AdtArray was created via class instead of methodmap, this code would not compile. The reason is that methodmap creates a tag that can coerce to any. However, class creates a type that coerces only to related types. Those types are AdtArray, and a special builtin type called Object. The type of a class does not coerce via any: or _:.

Explanation

Why does it work this way? The reason is a technical deficiency in the existing Pawn compiler. When you use "any", Pawn does not tell SourceMod what the exact types of its variables are. SourceMod just sees a bunch of bits. It doesn't know whether it's a float, an integer, an array pointer, a handle, or an object. This prevents SourceMod from implementing garbage collection* - SourceMod can't automatically free memory if it can't tell where all its objects are.

Ultimately, we want garbage collection, but it's a very big and complex task. In the meantime, it's important that the transitional API make garbage collection a future possibility. To this end, "class"-based objects do not have observable values. They cannot be passed into variadic functions or functions that would coerce them into an unrelated or non-object type - including the any type. Eventually, we will introduce a new variant of any that is safe to use with objects.

*Well, decent garbage collection. We could implement "conservative" GC, but it's very unappealing.