Time to get rid of the GameEvent objects and move to dictionaries?

Discuss API design here.
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Time to get rid of the GameEvent objects and move to dictionaries?

Postby Mahi » Wed Oct 28, 2015 12:17 am

As far as I know, Source.Python has always had the following syntax (in one way or another) for event callbacks:

Syntax: Select all

def player_hurt(event):
print('Userid: {0}'.format(event.get_int('userid'))
print('Weapon: {0}'.format(event.get_string('weapon'))

To be honest, I've always hated it. Python has insanely powerful dictionaries, which can do much more than the current GameEvent objects. I'm suggesting we finally try and bring dictionaries to the event system:

Syntax: Select all

def player_hurt(eargs):  # eargs as in event arguments
print('Userid: {0}'.format(eargs['userid']))
print('Weapon: {0}'.format(eargs['weapon']))

And of course since it's a dictionary now, you can do all kinds of cool stuff with it:

Syntax: Select all

def player_hurt(eargs):
print('Userid: {userid}\nWeapon: {weapon}'.format(**eargs))

And if we were to use the double star syntax, which I would love to see, you could grab any values you want already in the parameters, and the rest would get packed into the eargs dict:

Syntax: Select all

def player_hurt(userid, victim, **eargs):
print('Attacker: {0}, Victim: {1}, Eargs: {3}'.format(userid, victim, eargs))

There was already a discussion about this long ago: http://forums.sourcepython.com/showthread.php?10-Event-system-discussion but since SP has developed a lot from then, I believe we now have better ways of doing this than manually going through every argument. I tried to get my head around it and come up with some kind of hacky try/except as Ayuto suggested in the old thread, but it really wasn't that good. Luckily, Ayuto had a better solution already: http://pastebin.com/U36XQt8U

Using this KeyValues trick, I was able to replicate the suggested behaviour purely on Python:

Syntax: Select all

from collections import defaultdict
from events import Event
from memory import get_object_pointer
from memory import make_object
from keyvalues import KeyValues

OFFSET = 8

class DictEvent(object):

_callbacks = defaultdict(list)

@staticmethod
def fire(event_name, event_arguments):
for callback in DictEvent._callbacks[event_name]:
callback(**event_arguments)

def __init__(self, *event_names):
self._event_names = event_names

def __call__(self, *callbacks):
for callback in callbacks:
for event_name in self._event_names:
if callback not in DictEvent._callbacks[event_name]:
DictEvent._callbacks[event_name].append(callback)


@Event('player_spawn', ..., 'player_death') # You'd want to automatically register all events here
def on_event(event):
pointer = get_object_pointer(event).get_pointer(OFFSET)
keys = make_object(KeyValues, pointer)
event_arguments = keys.as_dict()
DictEvent.fire(event.get_name(), event_arguments)

Now simply instead of using the @Event decorator, one would use @DictEvent decorator:

Syntax: Select all

@DictEvent('player_spawn')
def on_spawn(userid, **eargs):
print(userid, eargs)
And this could probably be improved on the C++ side. Maybe. Regardless, we now have full access to Python's wonderful dictionaries, and I can't see any drawback here.
What do you guys think?
User avatar
satoon101
Project Leader
Posts: 2294
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Wed Oct 28, 2015 1:08 am

This does seem to return with the proper types, so it is definitely worth exploring. The issues mentioned in the other thread seem to be non-existent in this implementation, if that holds true. I will certainly test out the case in which SuperDave pointed out in the ES forum discussion I linked to in the other thread.

I think, since we would have to use an offset that "could" be different per-engine/game, it would be best to do this work in Python. Actually, doing anything using the memory package would be best served being written in Python, imo. There really wouldn't be much of a speed difference anyway.

*Edit: I forgot that SD's post was about L4D and not L4D2. We don't have support for that game, atm, and not sure if we are going to bother.
Image
User avatar
satoon101
Project Leader
Posts: 2294
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Wed Oct 28, 2015 2:03 am

I started a new branch to test this implementation. I did notice that 'headshot' is an integer instead of a boolean in player_death, but it still works properly unless you are comparing it directly to True/False.

*Edit: forgot to post a link to the new branch: https://github.com/Source-Python-Dev-Team/Source.Python/tree/event_test
Image
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Wed Oct 28, 2015 2:40 am

Python it is! If you end up using my implementation, feel free to modify it however you please. The __call__() method should probably take only one callback function as an argument, instead of argument amount of callback functions. Although, if possible, you're probably better off implementing this to the original Event decorator directly, instead of writing an extra layer of listeners.
satoon101 wrote:I started a new branch to test this implementation. I did notice that 'headshot' is an integer instead of a boolean in player_death, but it still works properly unless you are comparing it directly to True/False.
To be more accurate: unless you are comparing it using "is". True and False are just singleton integers after all:

Syntax: Select all

>>> True == 1
True
>>> False == 0
True
>>> issubclass(bool, int)
True
User avatar
satoon101
Project Leader
Posts: 2294
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Wed Oct 28, 2015 2:41 am

Mahi wrote:Python it is! If you end up using my implementation, feel free to modify it however you please. The __call__() method should probably take only one callback function as an argument, instead of argument amount of callback functions. Although, if possible, you're probably better off implementing this to the original Event decorator directly, instead of writing an extra layer of listeners.


Did you look at the implementation in the new branch? The Event decorator just wraps event_manager.(un)register_for_event (same with PreEvent and pre_event_manager). The conversion just needs to take place prior to looping through the registered callbacks for the given event.

Events
Pre-Events
Image
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Wed Oct 28, 2015 2:54 am

satoon101 wrote:Did you look at the implementation in the new branch? The Event decorator just wraps event_manager.(un)register_for_event (same with PreEvent and pre_event_manager). The conversion just needs to take place prior to looping through the registered callbacks for the given event.

Events
Pre-Events
I'm on mobile and it's soon 5 am here, I might have missed the branch :D
But yeah I figured it'd be easy to implement directly into the event system. I'll study the system a little more when I wake up tomorrow.

As a side note, I've accidentally deleted my comment 3 times now through the mobile website. The hitbox for the delete button seems to be somewhat messed up, it goes far below the visual button.
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Wed Oct 28, 2015 2:58 am

The hitbox for the delete button seems to be messed up, it goes far below the visual button. But I'm going offtopic here, just felt like letting you know, and obviously can't edit the old comment due to the delete bug
User avatar
Ayuto
Project Leader
Posts: 1545
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Postby Ayuto » Wed Oct 28, 2015 4:06 pm

There are a few issues with this implementation that we should consider:
  • If you want to use an event variable as a parameter and not through the kwargs, you are force to use the same name. This is problematic, because you can't use event variables that contain invalid characters (e.g. spaces). This can also break the coding convention (e.g. the "Priority" event variable).
  • The GameEvent object gets lost and thus it's methods.
  • The "event_name" workaround to keep the event name is problematic, because it can happen that an event has the event variable "event_name". In that case it would be overridden. Moreover it's not logical to put it in the dict, because it's obviously not an event variable.

When I talked to Mahi yesterday to show him how to get the event variables dynamically I mentioned that I would prefer to just implement the __getattr__. However, that obviously doesn't work because of the reasons I mentioned above. So, it would be probably better to implement __getitem__.
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Wed Oct 28, 2015 6:24 pm

Ayuto wrote:The "event_name" workaround to keep the event name is problematic, because it can happen that an event has the event variable "event_name". In that case it would be overridden. Moreover it's not logical to put it in the dict, because it's obviously not an event variable.
If someone wants to break their plugin, it's pretty much impossible to prevent them from doing so. Just don't add an event variable called "event_name". Also, it's not really event variable, we're simply passing the event's name as a keyword argument to the callback function, what's wrong with that? ^^ To be honest the implementation I made should be changed to directly pass it as a keyword argument instead of adding it to the dict in between. (and then unpacking the dict)

Ayuto wrote:The GameEvent object gets lost and thus it's methods.
I don't see the issue here, dicts are quite unarguably stronger structure than the GameEvent object. Sure we could do the __getitem__ approach, but is there an actual reason to? Other than someone naming their variable event_name, which is their problem. (in my opinion)

Ayuto wrote:If you want to use an event variable as a parameter and not through the kwargs, you are force to use the same name. This is problematic, because you can't use event variables that contain invalid characters (e.g. spaces). This can also break the coding convention (e.g. the "Priority" event variable).
This isn't problematic at all, this is fantastic! Remember that all of the other options require you to always access the event variables through a dict or an object. We now have the upside of usually getting to access it through the parameters instead! So yeah, if there are spaces in the name, we lose the upside of using parameters, but usually we have the luxury of kwargs.
User avatar
Ayuto
Project Leader
Posts: 1545
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Postby Ayuto » Wed Oct 28, 2015 8:44 pm

Mahi wrote:If someone wants to break their plugin, it's pretty much impossible to prevent them from doing so. Just don't add an event variable called "event_name".
Well, that's an unnecessary restriction that could be prevented easily. Moreover, you can't make sure that other server plugins create an event with an "event_name" variable that you want to listen to with SP.

Mahi wrote:Also, it's not really event variable, we're simply passing the event's name as a keyword argument to the callback function, what's wrong with that? ^^
Exactly, it's not an event variable. And that's why it shouldn't be added to the dict that should only contain event variables.

Mahi wrote:To be honest the implementation I made should be changed to directly pass it as a keyword argument instead of adding it to the dict in between. (and then unpacking the dict)
Since you would always pass the event name, it's a positional argument and not a keywords argument. Moreover, it would introduce another issue. See this example:[PYTHON]def f(event_name, userid, **kwargs):
print(event_name, userid, kwargs)

event_variables = dict(userid=2, event_name='asd')
event_name = 'my_event'
f(event_name, **event_variables)[/PYTHON]Output:

Code: Select all

TypeError: f() got multiple values for argument 'event_name'


Mahi wrote:
Ayuto wrote:The GameEvent object gets lost and thus it's methods.
I don't see the issue here, dicts are quite unarguably stronger structure than the GameEvent object. Sure we could do the __getitem__ approach, but is there an actual reason to? Other than someone naming their variable event_name, which is their problem. (in my opinion)
If you loose the GameEvent object you can't call it's methods and everything that requires a GameEvent object anymore. One of SP's advantages is that we directly expose many C++ classes. This is great, because we can use the memory module to get the C++ pointer of the objects and do awesome stuff with it. GameEvent objects should rather behave like a dictionary. We shouldn't completely convert them into a dict.

Mahi wrote:
Ayuto wrote:If you want to use an event variable as a parameter and not through the kwargs, you are force to use the same name. This is problematic, because you can't use event variables that contain invalid characters (e.g. spaces). This can also break the coding convention (e.g. the "Priority" event variable).
This isn't problematic at all, this is fantastic! Remember that all of the other options require you to always access the event variables through a dict or an object. We now have the upside of usually getting to access it through the parameters instead! So yeah, if there are spaces in the name, we lose the upside of using parameters, but usually we have the luxury of kwargs.
I wouldn't call this fantastic. I would call this weird behaviour and it will confuse beginners.
User avatar
satoon101
Project Leader
Posts: 2294
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Wed Oct 28, 2015 9:46 pm

I am in agreement with Ayuto. This sounds like a much better/safer option to me. We can just implement __getitem__ and __setitem__ on the GameEvent object itself. Since we still have access to the KeyValues object, we can iterate through it on__getitem__ and raise a KeyError if the variable doesn't exist. For __setitem__, we wouldn't really need the KeyValue object, as we can just use the type passed to call the proper set_<type> method.
Image
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Wed Oct 28, 2015 10:12 pm

Yeah I'm starting to lean on Ayuto's solution after the C++ argument (although I still don't agree with everything). Any possibility of adding a method which gets the event variables as a dict? Unless you plan on making the GameEvent object iterable/mapping, in which case I can't see much use for such method.

Edit: This might be a bit over the top and have some huge drawbacks, and I haven't really thought if it's even possible, but what about just subclassing dict and convert the event variables to key-value pairs when the GameEvent object is created?

Edit 2: Nvm, I guess you couldn't be able to use it with all the C++ functions anymore. Wouldn't make much sense anyways.
User avatar
Ayuto
Project Leader
Posts: 1545
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Postby Ayuto » Wed Oct 28, 2015 10:17 pm

We wouldn't even need to use the KeyValues object to check if an event variable exists. Though, that is a bit hacky.

Syntax: Select all

from events import Event
from events import GameEvent

def has_var(self, name):
"""Return True if the given event variable exists."""
# If something else than 'a' was returned, the variable definitely exists.
if self.get_string(name, 'a') != 'a':
return True

# Now, we need to make sure the default value wasn't returned. We can
# easily do that by retrieving the event variable again, but do not
# provide a default value. In that case the default value for the default
# value is an empty string
return self.get_string(name) == 'a'

GameEvent.has_var = has_var

@Event('player_footstep')
def on_player_footstep(event):
print(event.has_var('userid')) #Prints True
print(event.has_var('x')) # Prints False
However, we will still need it if we want to automatically convert the event variables into to the proper type. Do we really want to convert them automatically? I like automatic conversion, but if we do that we should do it consistently. For example we do not provide a Entity.get_property method, but Entity.get_property_<type>, although we know the types in that case as well.
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Wed Oct 28, 2015 10:47 pm

Ayuto wrote:Do we really want to convert them automatically?
That's a yes from me, if it has any weight!

Ayuto wrote:I like automatic conversion, but if we do that we should do it consistently. For example we do not provide a Entity.get_property method, but Entity.get_property_<type>, although we know the types in that case as well.
I've always hated the get_property_<type>. Maybe even implement __getitem__ and __setitem__ here too, Python is giving you insanely strong tools and you're using C++
Not sure if it's as good here as it is with GameEvent though.
User avatar
satoon101
Project Leader
Posts: 2294
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Wed Oct 28, 2015 11:18 pm

If we are going to make any changes, it should be so that they are the correct type. Making it just like ES where everything is a string value and we have to typecast anyway seems pointless to me, which is the main reason this was never done in the first place. Honestly, I am going to test the player_say event when I get the chance to see if saying a number still results in a string or if it is an integer. If it is an integer, then, in my mind, we are back to what I stated in the old thread. If we cannot rely on getting the proper type, I think we should stick with the current system.
Image
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Wed Oct 28, 2015 11:30 pm

satoon101 wrote:If we are going to make any changes, it should be so that they are the correct type. Making it just like ES where everything is a string value and we have to typecast anyway seems pointless to me, which is the main reason this was never done in the first place. Honestly, I am going to test the player_say event when I get the chance to see if saying a number still results in a string or if it is an integer. If it is an integer, then, in my mind, we are back to what I stated in the old thread. If we cannot rely on getting the proper type, I think we should stick with the current system.

Using the first event_test branch:

Syntax: Select all

from events import Event

@Event('player_say')
def on_event(**eargs):
print(eargs)

Output:

Code: Select all

Mahi: 1
{'splitscreenplayer': 0, 'userid': 2, 'text': '1', 'Priority': 1}
Mahi: 3.0
{'splitscreenplayer': 0, 'userid': 2, 'text': '3.0', 'Priority': 1}
Mahi: True
{'splitscreenplayer': 0, 'userid': 2, 'text': 'True', 'Priority': 1}
User avatar
satoon101
Project Leader
Posts: 2294
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Thu Oct 29, 2015 1:23 am

Ok, thank you. I also just tested on all supported servers, and the results are the same. That's good, I really didn't want to scrap this idea. From everything else I have tested, it seems we can rely on the variables to be of the correct type. Of course, there could be a curve ball thrown in at some point, but I have yet to find one.

I would like to move forward with the __getitem__/__setitem__ idea where we make sure to get the correct type. I know that one issue is that other plugin writers can throw off the types by setting using the wrong one. However, I think that would be an issue with that plugin and not our implementation.
Image
User avatar
satoon101
Project Leader
Posts: 2294
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Thu Oct 29, 2015 2:39 pm

Ayuto wrote:We wouldn't even need to use the KeyValues object to check if an event variable exists. Though, that is a bit hacky.

Syntax: Select all

from events import Event
from events import GameEvent

def has_var(self, name):
"""Return True if the given event variable exists."""
# If something else than 'a' was returned, the variable definitely exists.
if self.get_string(name, 'a') != 'a':
return True

# Now, we need to make sure the default value wasn't returned. We can
# easily do that by retrieving the event variable again, but do not
# provide a default value. In that case the default value for the default
# value is an empty string
return self.get_string(name) == 'a'

GameEvent.has_var = has_var

@Event('player_footstep')
def on_player_footstep(event):
print(event.has_var('userid')) #Prints True
print(event.has_var('x')) # Prints False
However, we will still need it if we want to automatically convert the event variables into to the proper type. Do we really want to convert them automatically? I like automatic conversion, but if we do that we should do it consistently. For example we do not provide a Entity.get_property method, but Entity.get_property_<type>, although we know the types in that case as well.


From a consistency standpoint, we do allow for automatic conversion with get_property_<type> through the __getattr__ and __setattr___ magic methods, which is essentially what we would be doing with GameEvent. The only caveat is that, with the Entity implementation, the property needs to be included in the data.
Image
User avatar
Ayuto
Project Leader
Posts: 1545
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Postby Ayuto » Thu Oct 29, 2015 5:48 pm

satoon101 wrote:I know that one issue is that other plugin writers can throw off the types by setting using the wrong one.
We can check the the data type of the event variable by using KeyValues.get_data_type().

satoon101 wrote:From a consistency standpoint, we do allow for automatic conversion with get_property_<type> through the __getattr__ and __setattr___ magic methods, which is essentially what we would be doing with GameEvent. The only caveat is that, with the Entity implementation, the property needs to be included in the data.
Yeah, but we could actually provide a get_property() method.
Mahi
Senior Member
Posts: 210
Joined: Wed Aug 29, 2012 8:39 pm
Location: Finland

Postby Mahi » Sat Oct 31, 2015 7:38 am

I don't mean to be nosy or annoying, but any news on this? Do you guys have any future plans? I would love to code some SP plugins, but I'd like to know the new syntax etc. before I start

Return to “API Design”

Who is online

Users browsing this forum: No registered users and 1 guest