SRCDS integration with the webserver

[iPlayer]

Hey there.
Currently I'm implementing some of the features requested by cr4nkz and thought of his class-based buymenu idea.

We can't alter game's buymenu, but we can do radio popups or ESC menus. But what I decided to do is put buymenu in MOTD.

I once had MOTD used for filling ban reasons: http://i.imgur.com/lRJwwhi.png

Sequence of events is

• Admin bans a player, game server puts the ban (with empty reason) in MySQL database. This database is shared between web- and game-server.
• Admin types !reason, server sends them MOTD with the following url:
  

  Code: Select all

  http://example.org/banlist/<admin steamid>/<admin auth string>/
  
  <admin steamid> - admin Steam Community id
  <admin auth string> - sha512(<constant secret salt> + <admin steamid>)
• Website takes admin steamid that is received, calculates sha512(<same secret salt> + <admin steamid) on its own and compares result to the received auth string. If they're equal, this is a valid admin coming from our server. If not, somebody's being too smart.
• Website returns the page (you can see the screen above) with all registered bans for this admin, each ban has a field to edit the ban reason
• Admin fill the reason and clicks "Save". Form sends request to the same exact URL + additional POST data which contains the ban reason
• Website processes the URL once again, but this time saves received ban reason to the MySQL database.
• Website returns the page (something like "Success"), admin closes MOTD.

Security of this approach can be improved by storing the secret salt in the database and changing it after every request.
MySQL database acts as a coordinator between website (MOTD) and gameserver. For filling out ban reasons post-factum this is more than enough.

But if we're talking about buymenu, any action (e.g. players clicks 'buy weapon' in MOTD) needs a response from the game server.

Approach #1. (dirty)
Still use MySQL as a coordinator. Website puts the data like 'player X bought weapon Y' to the database.
Game server either:
1. Checks the database every 5 seconds to see if there're any purchases made.
2. Checks the database as soon as the player closes their MOTD - we can detect that if player starts moving. If they start moving, check the database action queue and equip player with the weapons.

Disadvantages:
Database is not meant to be used this way, this approach is too hacky. Also requires database to be hosted somewhere.

Approach #2.
With the power of mighty Python open another TCP port on SRCDS and serve it for web. Then we can send MOTD with the following URL

Code: Select all

http://<your server ip>:27080/buymenu/<player steamid>/<auth string>/

Or, alternatively, open TCP port on SRCDS but hide it from web. Instead, serve the web with a web-server. The web-server then connects to SRCDS on every HTTP request to pass data to it.

Disadvantages:
SRCDS is not meant to be used this way. Then, any self-respectful game hosting will tell you what they think about you and your buymenu. So it still requires the server to be hosted somewhere serious, say, VPS.

Approach #3.
Create your own coordinator and connect SRCDS to it on a constant basis.

I see 3 ways of doing that:
1. Standalone buymenu server. SRCDS is connected to it (as a client) constantly through sockets. Webserver serves MOTD. Webserver connects to buymenu server on every HTTP request and exchanges data with buymenu server, then buymenu server exchanges data with SRCDS.
2. Standalone buymenu server. SRCDS is connected to it constantly through sockets. Buymenu server is open to the web and serves MOTD by itself. On every HTTP request it exchanges data with SRCDS.
3. Web-sockets web-server. SRCDS is connected to it constantly through web-sockets as one of the HTTP clients.

In all three cases the game server acts as a client, no need to open extra ports.

First one is more protected. Buymenu server is hidden from the web, all three servers (buymenu, web, game) can be located on the same machine. Webserver serves MOTD and basically does the job it's written for. Then you've got CloudFlare, all this stuff to protect you.

Second one is more flexible. For example, since we're basically writing our own hybrid web-server, we can do support for web-sockets thus allowing data to pass from the game server into the MOTD, too. Very important thing to understand that this buymenu server is meant to serve 64 players at best. Which are not a trouble. Until somebody decides to DDoS you.

Third one may sound as a good idea, but basically this is still about writing your own web-application from scratch. Both Apache and nginx are designed to work on per-request basis. CloudFlare is designed to work on per-request basis. And here instead of a script which runs on every HTTP request, you have script that's running indefinitely in a loop. I thought of nginx + uWSGI, but still, I see no benefits to using this way over the second one, because any web-designed DDoS-protection is not suitable in the case.

Disadvantages:
Requires either to host buymenu server (VPS) or finding a good web-hosting that would support web-sockets.


Thank you so much for reading. What do you guys think of these ways? I'm choosing between 3.1 and 3.2.

[necavi]

Personally I'd be interested to see an attempt to run a webserver from srcds directly, at least for certain testing environments.

[iPlayer]

So that's approach #2.1.

There's SourceWeb. But it seems abandoned. Not that I would use it (I need the web-server running in the same namespace with my plugin, in our case - on SP), but it may be abandoned for a reason.
A long time ago on some Russian forums I've read that there was a SRCDS-based web-server (maybe it was SourceWeb) but because of some fatal design flaws it was easy to DoS. Not to DDoS, but to simply harass and crash the whole thing. That's why I wrote "SRCDS is not meant to be used this way".

Edit: There's also sourcemod shttpd extension

In terms of SP, it should be as easy as importing http.server

[iPlayer]

I went with #2.2 - open port on SRCDS but serve MOTD with a web-server which then connects to SRCDS and exchanges data with it.

Current sequence of events varies. But any initial MOTD leads to the following

1. Server sends MOTD with the url like
   

   Code: Select all

   http://<server ip>/motdplayer/<page id>/<player steamid>/<auth method>/<auth token>/<session id>/
   
   Where:
   <page id> is something like a name of the script SRCDS wants to execute on webserver - basically this tells webserver WHAT page it should serve
   <player steamid> is player's community id (SteamID64) - this tells webserver WHO it serves the page for
   <auth method> defines method of authentification: MOTDs that were just sent by SRCDS will use one auth method, and pages that are opened consequently in the same MOTD will use another.
   <session id> is nothing more than a number of MOTDs sent to user. The thing is, when you send 2 MOTDs in a row without delay, the player will receive the first one; but if you send 2 MOTDs with a little delay, the player will end up seeing the second one. Session ID is unique for each sent MOTD. When webserver sends it back to SRCDS, SRCDS will know which MOTD a player worked with.
   <auth token> is

   Code: Select all

   sha512(<personal player salt> + <player steamid> + <page id> + <session id> + <constant salt for both srcds and webserver>)
   
   This hash excludes the possibility of faking any of the components: page id, session id or steamid.
   Constantly changing personal player salt makes it impossible to access this URL twice.
2. Webserver calculates the same hash, compares it to the received auth token and considers player valid
3. Webserver establishes connection to SRCDS and sends new personal salt for this user - then waits for the SRCDS to confirm that the new salt was accepted
4. Webserver calls proper function to serve request based on Page ID (the following and all other examples are made in Flask):
   

   Syntax: Select all

   @srcds_request(app, 'test_page')
   def route_test_page(steamid, web_auth_token, session_id, data_exchanger, error):

5. This function can use passed data_exchanger object to send and immediately receive data from SRCDS:
   

   Syntax: Select all

   custom_data = {
           "we say": "Hello",
       }
       response = data_exchanger.exchange(custom_data)
       print(response)    # {'server says': 'World!'}

   
   
   Any time webserver uses data_exchanger, SRCDS consequently runs callback and sends back data returned by that callback:
   

   Syntax: Select all

   def callback(data, error):
       print("Received data: {}".format(data))     # {'we say': 'Hello'}
       return {"server says": "World!"}

6. As soon as route_test_page returns, webserver closes connection to SRCDS and passes result to Flask - the page is rendered to MOTD

[iPlayer]

Here we go!

[Ayuto]

That's quite cool! I know someone who has planned something similar. I have sent him a link to this thread. I think if you both work together, this can become something really great!

[iPlayer]

Thank you, Ayuto

[satoon101]

That is really cool! I can't wait to see where this could all lead. I was hoping your video would show what happens when you cannot afford a weapon. I can definitely see me utilizing this for GunGame in a few different ways

[iPlayer]

Thanks, Satoon.

My fault. I had $16000 in cash in the beginning of the video. Every time I bought something (I made requests via AJAX), server sent back a list of weapons that I can afford for the remaining cash. But since I had so much money, the page was rebuilding with all the weapons again, because I could afford all of them. I'm gonna make a new video.

About GunGame: this approach requires a webserver (particularly Flask webserver) to be running on the same machine with SRCDS. Plus you'll need to have an open port on it.
This can be made more appealing if I also provide PHP backend. Also you can probably run webserver on a separate machine, because I've implemented a whitelist for the hosts that can utilize SRCDS communication port. You just whitelist IP of your webserver and in theory you escape from internet hazards.
But still, yeah - webserver plus additional port to open. Definitely not a regular game hosting.

[iPlayer]

Specially for you, Satoon

I can't wait to see where this could all lead.

The biggest thing here that I'm scared of is Valve. They're unpredictable with their updates. It'd be too sad if they just annihilate my work.

[Predz]

Hey, loving the idea of this!

Do you have much of an idea if this is possible for CSGO? I have tried to display websites in MOTDs in CSGO before but to no avail myself. Really would like to try implement this into Warcraft GO if it is at all possible.

I have multiple servers that I can use so hosting the website on the same system is not an issue.

[iPlayer]

Yep, I heard of CS:GO problems with MOTD. I have not tested yet, but here is the way I will go http://forums.sourcepython.com/showthread.php?902-Csgo-motd&p=6350&viewfull=1#post6350

Honestly, I'm more worried about CS:S because even broken CS:GO MOTD is better than working CS:S one. Broken CS:GO MOTD will be fixed, but working CS:S MOTD they will break and it'll never be fixed.

[iPlayer]

Good news:

1. I've managed to open MOTD popup (not to confuse with the main MOTD window which is unavailable in CS:GO) with that JS trick.
2. Package API remained the same, all redirection stuff is done automatically whenever the package detects CS:GO. You'll only need to put extra template in your templates folder and do a bunch of URL configuration.

Bad news:

1. Weird stuff happens with CS:GO. Whenever I launch Fraps with it, buymenu reports invalid auth.
   I checked personal salts on both webserver and SRCDS and they are synchronized.
   But Flask console is filled with errors 10053 when trying to send data back to CS:GO client. Even for some static files. I've yet to find any pattern, it continues until I restart the game. As said, I suspect Fraps and its framelimiter to do something with it, because without Fraps everything goes smootly. Or I just did too little testing.
2. It's not main MOTD window - that one is fixed size, can't be moved around and has OK button on it. It's a popup - you can open another one without closing the previous one, but that's useless, because my auth system implies that all other possible windows are invalidated once you open a new one. Also you can't access ESC menu without closing the popup first.

Since I've failed to record CS:GO with Fraps, here's a screenie
http://images.akamai.steamusercontent.com/ugc/307737558703013111/4D283B66D71A0E93935C82050C08465434722C79/

Oh, I also encountered some odd stuff when sending motd fails at all - server reported something about SendNetMsg overflow or something. I couldn't even receive description for 'buy' command when I tried to type that in game console - usually it prints a whole list of available weapons.

[Predz]

Yeh that SendNetMsg error seems to be popping up everywhere when I try to send too many messages in WCGO. I am guessing that sending too many large UserMsg's, causes some sort of problem. The main problem about that error is that the player cannot receive any type of UserMsg after the error occurs. Possibly something to do with the UserMsg being limited on the data it can carry to a client?

Thanks for testing on CSGO as well, greatly appreciated, hehe

[L'In20Cible]

Really cool!

[iPlayer]

L'In20Cible, thanks too. I appreciate your guys support.

Regarding to the problem. It has nothing to do with Fraps and SentNetMsg apparently. I reached a consistency. You open buymenu, buy something, close it and then open it again - invalid auth error.
I figured out where that 10053 comes from.

When CS:GO opens first MOTD, its browser:

1. Loads redirection page
2. Loads target page twice in a row (why?)
3. Loads all static files including .js file
4. .js file sends JSON through AJAX
5. Receives a list of weapons as a AJAX result

then any purchases work well

Upon opening MOTD for the second time, its browser:

1. Loads redirection page
2. Loads target page
3. .js file sends JSON through AJAX - where did it get this .js file from?
4. Closes connection as soon as Flask starts sending response (Flask fails but does it after already changing the token, even status is 200 OK, it only fails when trying to write response to closed socket)
5. Loads target page again with the same URL - but the token in this URL is expired
6. Decides to load all static files this time, including .js file
7. .js file sends JSON through AJAX
8. Flask refuses to give the list of weapons because it has received old token

If I restart the web-server, CS:GO doesn't drop connection and then reload all static files. Instead, it loads all static files first (like if it was first MOTD) and sends AJAX-request without dropping connection. But when I open MOTD for the second time, it all repeats.
I guess it has to do something with caching on CS:GO part, I'll need to check what timestamps Flask sends to CS:GO.


EDIT: Good news! I forced CS:GO to reload javascript files every time by adding auth token to the path of the script:

Code: Select all

<script type="application/javascript" src="/static/js/{{ script }}.js?{{ auth_token }}"></script>

See, auth token goes directly into nowhere, all it does is making .js URL random. That worked.
Also, I'm sure once I set all this up on a production server, say, nginx, I can also solve this problem by setting Cache-Control: no-cache header when serving static files. Currently Flask development server sends Cache-Control header with the instruction to actually cache the file on a client.
Bad news! I tried to record it with Fraps and guess what. Invalid auth and after a few attempts - SendNetMsg overflow. However, everything works if I turn off its frame limiter.

So I guess the problem is solved and I officially pronounce this working on CS:GO.

[iPlayer]

So this it what I recorded.