SourceMod interop

All other Source.Python topics and issues.
fakuivan
Junior Member
Posts: 12
Joined: Sat Nov 11, 2017 4:57 am

SourceMod interop

Postby fakuivan » Thu Jan 25, 2018 5:14 am

I've been developing SourceMod plugins and extensions for a long time now and I feel quite comfortable digging into the source and reading documentation, but one thing I can't stand is SourcePawn, the mediocre language SourceMod plugins are coded in. Python on the other hand is a wonderful language, one of my favorites, I think that most people feel the same too, given that is one of the most used languages out there.

I feel like some sort of bridge intercom between Source.Python plugins and SourceMod plugins would be highly beneficial to both projects. Given that Python can be slow sometimes (and this is especially sensitive in per-frame tasks), but it can be a joy to write, while in the other hand, SourcePawn is quite fast, but it can be very frustrating to code and maintain projects of a decent size.

Technically, SourceMod provides a "native" interface that allows plugins to call functions declared by extensions or other plugins, that along with a forward api should be enough to make something like this work on the SourceMod side, without any modifications to the mod itself.
Sadly I don't have much knowledge on Source.Python to know how to tackle this problem from that side. I'd appreciate some advice on that.

Thanks for reading!
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Sun Jan 28, 2018 1:47 pm

I just had a look at it and it seems doable. Here is my proof of concept for creating natives. It's using AddNatives to add natives created with SP to the core natives of SM. In a real bridge (not proof of concept) we wouldn't use that function, but create an SM extension instead that exposes a C function that accept a name (name of the native) and a function pointer (the native itself). It then adds the natives to its extension natives instead of adding it to the core natives. We should also create a Python extension module that exposes some parts of the SourcePawn language. In this proof of concept, all of this is done via the memory module, which looks more difficult than it is.

1. Create ../sourcemod/scripting/include/sp.inc. This defines the native we will create with SP:

Code: Select all

native int my_multiply(int x, int y);

2. Create ../sourcemod/scripting/sp.sp. This is our SM plugin to test the native created with SP:

Code: Select all

#include <sp>

public void OnPluginStart() {
    PrintToServer("Result: %i", my_multiply(3, 7));
}

3. Compile the plugin and copy sp.smx file to ../sourcemod/plugins.
4. Create the SP plugin (../source-python/plugins/sm/sm.py) that adds the native to SM:

Syntax: Select all

from paths import GAME_PATH

import memory
from memory import Callback
from memory import Convention
from memory import DataType
from memory import NULL

LOGIC_PATH = GAME_PATH / 'addons/sourcemod/bin/sourcemod.logic.so'
if not LOGIC_PATH.isfile():
raise ValueError(f'{LOGIC_PATH} does not exist. This POC only works on Linux.')

binary = memory.find_binary(LOGIC_PATH, check_extension=False)

# void AddNatives(sp_nativeinfo_t *natives)
AddNatives = binary['_ZL10AddNativesP15sp_nativeinfo_s'].make_function(
Convention.CDECL,
[DataType.POINTER],
DataType.VOID)

# typedef cell_t (*SPVM_NATIVE_FUNC)(SourcePawn::IPluginContext *, const cell_t *);
@Callback(
Convention.CDECL,
[DataType.POINTER, DataType.POINTER],
DataType.INT)
def multiply(args):
print('Native called')
context = args[0]
params = args[1]

print('Param count:', params.get_int())
return params.get_int(4) * params.get_int(8)

# Construct the natives array
"""
typedef struct sp_nativeinfo_s
{
const char *name; /**< Name of the native */
SPVM_NATIVE_FUNC func; /**< Address of native implementation */
} sp_nativeinfo_t;
"""
native_name = 'my_multiply'

natives = memory.alloc(16)

# First native
natives.set_string_pointer(native_name)
natives.set_pointer(multiply, 4)

# Second/null native to tell SM this is the end of the array
natives.set_pointer(NULL, 8)
natives.set_pointer(NULL, 12)

# Add the native(s)
AddNatives(natives)
Now, it's time to test:

Code: Select all

sp plugin load sm
[SP] Loading plugin 'sm'...
[SP] Successfully loaded plugin 'sm'.
sm plugins load sp
Native called
Param count: 2
Result: 21
[SM] Loaded plugin sp.smx successfully.
fakuivan
Junior Member
Posts: 12
Joined: Sat Nov 11, 2017 4:57 am

Re: SourceMod interop

Postby fakuivan » Fri Feb 02, 2018 11:15 pm

Thanks for that amazing response! I am currently working on handling dynamically assigned natives, using libffcall (https://github.com/fakuivan/SMConnect/b ... pp#L38-L50)

I see that it is possible to use ``memory`` module to call exported functions on extensions. Now how would we go about handling sp loading first and then trying to query a sourcemod extension but failing because of it not being loaded?

SP loads when the SM extension is ready to process requests (loads late):
[*] SP's entry point -> Third party SourceMod extension that exposes symbols
SM ext loads when SP is ready to process requests (loads earlier):
[*] SM ext entry point -> ?
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Sun Feb 04, 2018 8:38 pm

fakuivan wrote:I see that it is possible to use ``memory`` module to call exported functions on extensions.

AddNatives is not an exported function. The memory module uses its private symbol to get its address. The address is then called by the memory module. So, the memory module goes deeper than just calling a public/exported function.

fakuivan wrote:Now how would we go about handling sp loading first and then trying to query a sourcemod extension but failing because of it not being loaded?

So, you basically want to know of to check with SP whether an SM extension has been loaded?

Btw. you should always load SP after SM. Otherwise SM might not be able to find all function signatures (e.g. if a function has been hooked with SP). SP does find them, because it also looks for hooked functions. So, you actually don't need an answer for that question (in this case).
fakuivan
Junior Member
Posts: 12
Joined: Sat Nov 11, 2017 4:57 am

Re: SourceMod interop

Postby fakuivan » Mon Feb 05, 2018 2:00 pm

Great news. So SP should always load after SM, are there any mechanisms in place that detect mms/sm and do this automatically or is it up to the user?
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Mon Feb 05, 2018 4:19 pm

It's up to the user by loading the plugins via autoexec.cfg.
User avatar
Zeus
Member
Posts: 52
Joined: Sat Mar 24, 2018 5:25 pm
Location: Chicago
Contact:

Re: SourceMod interop

Postby Zeus » Sun Mar 25, 2018 4:38 am

I'm a bit confused; does this mean we can receive forwards from sourcemod plugins?
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Sun Mar 25, 2018 9:24 am

The example above creates a native with Source.Python and makes it available in Sourcemod, so SM plugins can call an SP function. I'm pretty sure we can also make it work vice-versa (call SM natives from SP) or even receive forwards.
fakuivan
Junior Member
Posts: 12
Joined: Sat Nov 11, 2017 4:57 am

Re: SourceMod interop

Postby fakuivan » Sun Mar 25, 2018 2:50 pm

Zeus wrote:I'm a bit confused; does this mean we can receive forwards from sourcemod plugins?


That is not the point of the example, but it can be done easily with an extension that can already communicate with a source.python script.
User avatar
Zeus
Member
Posts: 52
Joined: Sat Mar 24, 2018 5:25 pm
Location: Chicago
Contact:

Re: SourceMod interop

Postby Zeus » Sun Mar 25, 2018 3:34 pm

Ayuto wrote:The example above creates a native with Source.Python and makes it available in Sourcemod, so SM plugins can call an SP function. I'm pretty sure we can also make it work vice-versa (call SM natives from SP) or even receive forwards.


Oh, i'd be very interested in that; theres a few plugins i'd like to write that would require forwards from SM. I'm not going to pretend that I understand the memory stuff since i literally stumbled on this project yesterday XD
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Sun Mar 25, 2018 4:11 pm

Which forwards exactly do you need? Maybe we already have something similar.
User avatar
Zeus
Member
Posts: 52
Joined: Sat Mar 24, 2018 5:25 pm
Location: Chicago
Contact:

Re: SourceMod interop

Postby Zeus » Sun Mar 25, 2018 4:38 pm

theres a plugin for TF2 that uploads game logs to a website; in it, it registers 2 forwards:

Code: Select all

   
   // Make it possible for other plugins to get notified when a log has been uploaded
   g_hLogUploaded = CreateGlobalForward("LogUploaded", ET_Ignore, Param_Cell, Param_String, Param_String);
   
   // Let other plugins block log lines
   g_hBlockLogLine = CreateGlobalForward("BlockLogLine", ET_Event, Param_String);
   


http://www.teamfortress.tv/13598/medics ... od-plugin/ (logstf plugin)
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Sun Mar 25, 2018 5:21 pm

How does a SourceMod plugin listen to those forwards? (I have almost no knowledge about SM plugins)
User avatar
Zeus
Member
Posts: 52
Joined: Sat Mar 24, 2018 5:25 pm
Location: Chicago
Contact:

Re: SourceMod interop

Postby Zeus » Sun Mar 25, 2018 5:35 pm

In SM, simply implement the function (with forward on the function stub) and it'll be called when the original plugin makes the forwarding call.

https://wiki.alliedmods.net/Introduction_to_sourcepawn
https://wiki.alliedmods.net/Function_Calling_API_(SourceMod_Scripting)#Global_Forwards

In my case; this happens with this function, which is called on round end:

Code: Select all


CallLogUploaded
(bool:success, const String:logid[], const String:url[]) {
    Call_StartForward(g_hLogUploaded);     //https://sm.alliedmods.net/new-api/functions/Call_StartForward

    // Push parameters one at a time
    Call_PushCell(success);
    Call_PushString(logid);
    Call_PushString(url);

    // Finish the call
    Call_Finish();
}


I would just implment this:

Code: Select all


forward void OnLogUploaded
(bool success, const char[] logid, const char[] url);
 


SM must keep some kind of reference to all defined forwards?
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Sun Mar 25, 2018 10:51 pm

Okay, I came to the conclusion that the easiest way is to create an SM plugin that listens to that forward. Then you can use the code above to call a native created with SP.

So, it basically looks like this:
  1. logstf creates a new forward and calls it from time to time.
  2. You create a new SM plugin that listens to the forward. The implementation will call a native (e.g. NotifySP_OnLogUploaded(success, logid, url)).
  3. You create the native NotifySP_OnLogUploaded with SP using the code above.
Since the forward uses strings you need a little bit more than what I have shown in my first post:

Syntax: Select all

# sp::PluginContext::LocalToString(int, char **)
_LocalToString = None

def get_LocalToString(context):
global _LocalToString
if _LocalToString is not None:
return _LocalToString

return context.make_virtual_function(
21,
Convention.THISCALL,
[DataType.POINTER, DataType.INT, DataType.POINTER],
DataType.VOID)


# typedef cell_t (*SPVM_NATIVE_FUNC)(SourcePawn::IPluginContext *, const cell_t *);
@Callback(
Convention.CDECL,
[DataType.POINTER, DataType.POINTER],
DataType.INT)
def NotifySP_OnLogUploaded(args):
context = args[0]
params = args[1]
buffer = memory.alloc(4)
LocalToString = get_LocalToString(context)

success = params.get_int(4)

LocalToString(context, params.get_int(8), buffer)
logid = buffer.get_string_pointer()

LocalToString(context, params.get_int(12), buffer)
url = buffer.get_string_pointer()

print(f'OnLogUploaded: {x}, {y}')

return 0
fakuivan
Junior Member
Posts: 12
Joined: Sat Nov 11, 2017 4:57 am

Re: SourceMod interop

Postby fakuivan » Tue May 15, 2018 2:35 am

I have control over an extension binary. How would I go about exposing symbols on the C++ extension so the SP plugin can find them on windows and linux?
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Re: SourceMod interop

Postby Ayuto » Tue May 15, 2018 5:04 pm

You only need to declare your functions with extern "C". Then you can retrieve them with the memory module using their actual names. This might help you:
viewtopic.php?f=20&t=1206&p=7848#p7848

Return to “General Discussion”

Who is online

Users browsing this forum: No registered users and 33 guests