Adding Entity/PlayerEntity/WeaponEntity attributes via data

Discuss API design here.
User avatar
satoon101
Project Leader
Posts: 2697
Joined: Sat Jul 07, 2012 1:59 am

Adding Entity/PlayerEntity/WeaponEntity attributes via data

Postby satoon101 » Wed Nov 04, 2015 2:49 am

The Entity classes already have lots of data built in to be able to do all sorts of wonderful things. However, it would be really awesome to have our members start helping to add to the data to include more functionality, as well as to be prepared to help update the data when engines/games get updated. The following is a quick tutorial on how to find the different types of data and how to add them to the data files.

First things first, you need to know what entity types there are on the server. To find that out, use the dumpentityfactories command in the server's console. For the purposes of this tutorial, we will be using the env_smokestack entity.

Once you have an entity type you wish to find data for, you need to know what ServerClasses and DataMaps that particular entity has access to. You can use something similar to the following to find those:

Syntax: Select all

from entities.entity import Entity
from entities.helpers import index_from_edict
from filters.entities import EntityIter
from filters.players import PlayerIter
from players.bots import bot_manager
from players.entity import PlayerEntity


# ================================================
# >> CONFIGURATION
# ================================================
# Set to the entity classname you wish to know the ServerClasses/DataMaps for
entity_classname = 'env_smokestack'

# ================================================
# >> END CONFIGURATION
# ================================================


if entity_classname == 'worldspawn':
entity = Entity(0)
elif entity_classname == 'player':
for entity in PlayerIter(return_types='player'):
break
else:
entity = PlayerEntity(index_from_edict(
bot_manager.create_bot('test_bot')))
else:
for entity in EntityIter(entity_classname, True, 'entity'):
break
else:
try:
entity = Entity.create(entity_classname)
except (OverflowError, ValueError):
raise ValueError('Could not get entity of type "{0}"'.format(entity_classname))
if entity_classname != entity.classname:
raise ValueError('Could not get entity of type "{0}"'.format(entity_classname))
for server_class in entity.server_classes:
print(server_class.__name__)

This will print out (for env_smokestack):

Code: Select all

CSmokeStack
CBaseParticleEntity
CBaseEntity

CBaseEntity is carried by all entities, so let's not worry too much about that one for now. The other two are not currently included in the entity data for Source.Python.

The next part is a bit tricky. When adding the data, you need to know which directory to put everything into. If a data value is the same across all engines/games, you just put it into the base ../addons/source-python/data/source-python/entities/ directory. If it is engine specific, you use the engine's directory within the base directory. If it is game specific, you use the game's directory within the engine's directory. Lucky for you, all you need to do is utilize a virtual offset finder I created (with the base offset finder code coming from L'In20Cible) and a plugin.

Unfortunately, you will need a server for each server/game we currently support (you only need the game if there is no server for it). Without that, the plugin will not show data in the correct directories. Currently we support:

  • Black Mesa Source
  • Counter-Strike: Global Offensive
  • Counter-Strike: Source
  • Day of Defeat: Source
  • Half-Life 2
  • Half-Life 2: DeathMatch
  • Half-Life 2: Episode 1
  • Half-Life 2: Episode 2
  • Left4Dead 2
  • Portal (just 1 not 2 currently)
  • Team Fortress 2

You also need the Linux binaries for all the servers. I have my local Linux (and Windows) update a directory in my Dropbox account when the server updates. You can find them here:
https://www.dropbox.com/sh/d3ef67x7cg3bpit/AAAZ_uMWBx5hucD7RDUoWwKOa?dl=0

Just download the server.so and server_svr.so files. The others are unnecessary for the purpose of this tutorial. Place them all in a directory somewhere on your system. Then, Download and unzip the following into your Python3 installation's site-packages directory:

virtuals.zip
(3.12 KiB) Downloaded 1044 times



Next, you will want to edit the virtuals.ini file to your liking. Once you have done that, open your Command Prompt and run (make sure your Python3 installation is in your Path environment variable):

Code: Select all

python -m virtuals


This will get all the virtual offsets from each of the Linux binaries and store them by DataMap/ServerClass class name. Note that if you receive an ImportError for "No module named 'path'", you need to first run the following in the Command Prompt:

Code: Select all

pip install path.py


The same goes for "No module named 'configobj'":

Code: Select all

pip install configobj



After all of the binaries have been parsed, load each game/server with Source.Python installed and load the following script (it's a little rough around the edges right now, but gets the job done):
http://pastebin.com/9r0jxFnA

Make sure before you load it that you change the value of VIRTUAL_PATH in it to be the same value as SAVE_DIR in your virtuals.ini you edited earlier. All you need to do is load the plugin, nothing else, and wait till it is done, then you can move to the next game/server.

Once you have loaded the plugin on the last game/server, run the following commands in order:

Code: Select all

compare games
compare engines
dump_attributes


This will create a directory within the plugin's directory called results.

Open that directory and find all instances of the DataMap/ServerClass class name files you are wanting to find. Remember, for this example, those are:

Code: Select all

CSmokeStack
CBaseParticleEntity


In the base directory's CBaseParticleEntity, there are no real values we can use. In the base directory's CSmokeStack, however, there are plenty of values (skipping the virtual_function values):

Code: Select all

input
    Toggle
    TurnOff
    TurnOn
keyvalue
    BaseSpread
    EndSize
    InitialState
    JetLength
    Rate
    Roll
    Speed
    SpreadSpeed
    StartSize
    Twist
    WindAngle
    WindSpeed
property
    m_AmbientLight.m_flIntensity
    m_AmbientLight.m_vColor
    m_AmbientLight.m_vPos
    m_DirLight.m_flIntensity
    m_DirLight.m_vColor
    m_DirLight.m_vPos
    m_EndSize
    m_JetLength
    m_Rate
    m_Speed
    m_SpreadSpeed
    m_StartSize
    m_bEmit
    m_flBaseSpread
    m_flRollSpeed
    m_flTwist
    m_iMaterialModel
    m_strMaterialModel
    m_vWind


Since no file exists, create a file at ../addons/source-python/data/source-python/entities/CSmokeStack.ini.

Inputs are easy to add. Simply add the ones above as <method_name> = <input_name>:

Syntax: Select all

[input]

toggle = Toggle
turn_off = TurnOff
turn_on = TurnOn


When it comes to keyvalues and properties, if a value exists in both, you typically want to use the keyvalue. So, for instance, you will want to use the JetLength keyvalue for jet_length and not the m_JetLength property. The rest would look something like this:

Syntax: Select all

[keyvalue]

base_spread = BaseSpread
end_size = EndSize
initial_state = InitialState
jet_length = JetLength
rate = Rate
roll = Roll
speed = Speed
spread_speed = SpreadSpeed
start_size = StartSize
twist = Twist
wind_angle = WindAngle
wind_speed = WindSpeed

[property]

ambient_intensity = m_AmbientLight.m_flIntensity
ambient_color = m_AmbientLight.m_vColor
ambient_position = m_AmbientLight.m_vPos
direct_intensity = m_DirLight.m_flIntensity
direct_color = m_DirLight.m_vColor
direct_position = m_DirLight.m_vPos
emit = m_bEmit
roll_speed = m_flRollSpeed


Some of the more ambiguous ones might need a bit of testing to see what they do before adding them to the data. And some, like roll/roll_speed might be doing the same thing, so test those as well and only add 1.

Next, you want to walk the directories in the results directory to find if there are any engine/game specific values. Engines bms, csgo, and l4d2 as well as games cstrike, dods, hl2mp, and tf all have the following:

Code: Select all

property
    m_InitialState
    m_WindAngle
    m_WindSpeed


Since we already have keyvalues for each of those, there is no reason to add them as properties. So, for this entity, it seems there really isn't anything useful from an engine/game specific standpoint. If there was, you would just add it using the same directory structure as the results directory they are listed in.

If you have any other questions on this, please feel free to ask. I have also symlinked my results directory to Dropbox, so you can look there instead of doing all that work yourself. It might not always be up-to-date, though, which is why I wanted to cover that portion in the tutorial.
https://www.dropbox.com/sh/29hlaeh5fb90fsp/AABkNHTHcY0Q5f6XGne__zvHa?dl=0


I will show how to add more data types (dynamic/virtual functions and instance attributes) in the next post, once I get the opportunity to do so.
Image
Predz
Senior Member
Posts: 158
Joined: Wed Aug 08, 2012 9:05 pm
Location: Bristol, United Kingdom

Postby Predz » Wed Nov 04, 2015 9:39 am

WOW! Didn't realize all this was possible, will have to get to work then.
Thanks :)

EDIT: For people using Python3.4 ensure you replace "from path import Path" with "from pathlib import Path" inside the virtuals/__init__.py
User avatar
satoon101
Project Leader
Posts: 2697
Joined: Sat Jul 07, 2012 1:59 am

Postby satoon101 » Wed Nov 04, 2015 12:11 pm

Or install the path.py site-package as we have in Source.Python with:

Code: Select all

pip install path.py

Good point, though, I will add that to the tutorial later.
Image
User avatar
Doldol
Senior Member
Posts: 200
Joined: Sat Jul 07, 2012 7:09 pm
Location: Belgium

Postby Doldol » Sat Nov 14, 2015 5:01 am

Adding CTeam::GetNumPlayers(void)

Syntax: Select all

[property]

team = m_iTeamNum
team_name = m_szTeamname


[virtual_function]

# _ZN5CTeam13GetNumPlayersEv
[[get_num_players]]
offset_linux = 201
offset_windows = 200


Would be correct like this?
(There were no CS specific classes in the table so I'd think this should go into the generic .ini)
User avatar
Ayuto
Project Leader
Posts: 2193
Joined: Sat Jul 07, 2012 8:17 am
Location: Germany

Postby Ayuto » Sat Nov 14, 2015 11:04 am

You are missing the "return_type = INT" entry. Are those offsets the same on all games? I doubt, so we will probably need different offsets for the other games.

However, I'm not sure if we really want to add that to the data files. It's another offset we need to maintain for a functionality that already exist and doesn't work with all teams (if it's true what you said in the other thread).

Return to “API Design”

Who is online

Users browsing this forum: No registered users and 15 guests