Directory structure

[satoon101]

So, we have discussed this a little internally, but I would like to get other opinions. We are considering changing the folder structure inside the ../addons/source-python/ directory. Currently we have this setup:

• _core - holds the core.dll or core.so file
• _engines - holds the Python3 built-in packages
• _libs - holds Source.Python's Python-side API

One change that is definitely going to be made is _libs will be made _packages, as we more use packages with modules instead of all-in-one libraries. We are also considering adding a special packages directory to be used for helper packages created by the community. I'm not sure what to call this, but _custom_packages is one option. That stems from this thread:
http://www.sourcepython.com/showthread.php?202-k2tools-General-purpose-libary-package-WIP-SVN&p=1966&viewfull=1#post1966

The biggest change, though, that we are considering, is adding a folder for the scripts to go into. We can call that folder any of the following:

• addons
• scripts
• plugins

If you prefer one of these, or none of them, please let us know. If you have a better suggestion as to what to call it, please post that here, as well. If we end up creating this folder change, all of the leading underscores will be removed from the other directories.

Let us know your thoughts on all of this. Thanks,
Satoon

[Omega_K2]

I'd settle with plugins. I think it describes the nature of source-python plugins the best.

Some thoughts in general:

What about renaming some of the other directories and moving some contents? Eventually removing the underscores too?

ie- _core to _bin, and also place the loader in there, so it contains all source-python binaries (and future ones as well, if any are going to be added)
_engines to _python or better _python_core - I think it's clearer to see what is for in that case

Another idea instead of having

Code: Select all

_packages

and

Code: Select all

_user_packages

to have

Code: Select all

packages/custom/ (or packages/user/)
packages/site-packages/  [moved from the original location)
packages/source-python/


And finally, I think it might be good to move the data out of the package/_lib directory into a single one, like addons/source-python/data/
and mirror the directory stucture within, so that scripts that come with their own data (i.e. if they use custom signatures and so forth), can place their own data files in there, so it's all in one place.
So you'd end up with (included the suggestion above)

Code: Select all

addons/source-python/data/packages/user/
addons/source-python/data/packages/source-python/
addons/source-python/data/addons/

And to summarize, how it could look:

Code: Select all

bin/core.so
bin/source-python.so
data/packages/user/
data/packages/source-python/
data/addons/
packages/user/
packages/site-packages/
packages/source-python/
plugins/
python-core/

[satoon101]

Wow, very detailed, and I like a lot of the points you make. I was thinking about the possibility of moving the _data, but wasn't fully sure where I wanted to move it. bin sounds like a good change, though I am not sure if we want to move the source-python binary there as well. I do prefer "custom" to "user". What about packages/python/ for the Py3 packages? And, just throwing ideas out, but what about moving site-packages outside the python directory as well?

Overall, we really need to decide on a directory name for the addons/scripts/plugins if we are going to make that change. I'm still 100% up in the air on that one. I first thought of naming it "addons", but then, a good point was brought up that it would be ../addons/source-python/addons/. And if an addon chose to use its own structure to load its own addons, it could look like (using GunGame as an example, cause I am currently developing it and am at that point ) ../addons/source-python/addons/gungame/addons/

Other opinions are certainly welcome, so keep them coming.

*Edit: it seems as if you are still editing, so I will wait to make further comments till later.

Satoon

[Omega_K2]

yeah, I'm finished editing :P


I'm definetly for moving site-packages out of the python directory.

But I'm not so sure about /packages/python because after all python isn't exactly a package, but the core distribution of python including several binaries for windows and linux. Though I'd feel better about it if it was packages/python-core as I think it makes it more clear that it is the core distribution and not just some random python directory.

As for plugins vs addons vs scripts vs whatever.
scripts -> worst choice imho, because on python installations there are also script folders and I'd rather expect a script to be an excutable script that doesn't require SP (i.e. a standalone script that you'll call with python <script>.py)
addons -> yeah redundancy, plus I think the word is lighter on the "load & unload" nature, it's more like "add on top"-nature for my recognition of the word in the english langauge actually
plugins -> not redundant, plus it fits the "load & unload" nature better I think

And yeah, for my RPG I've settled with plugins for plugins (and skills for skills..duh :P)

[mister-man]

Hay,

I really like the idea of Omega_K2 with a little change.
I thing "site-packages" and "user-packages are none sp build in packages. So these packages can place in same directory.

Code: Select all

bin/core.so
bin/source-python.so
data/packages/user/   <--- Maybe package folder
data/packages/source-python/  <--- When the others are in own directories: data/
data/addons/  <--- Maybe plugin folder
packages/user/
packages/source-python/
plugins/
python-core/


And a last note, don't split the plugins in more than one folder. For users that want delete a plugin it was easy to delete only one folder (the plugin <name> folder). If you split plugins in more than one folder (like sourcemod) [folder for: plugin, language...], it is hard to clean up the server from an unneeded or outdatet plugins/user-packages. All plugin things should place in the plugin folder.

[La Muerte]

_engines does look a lot better then _python-core IMO (although I agree on your point, I prefer _engines simply because of its looks compared to the other directories)

indeed, it'd be data/plugins/

and plugins makes a lot more sense yeah

custom makes more sense then user to me

Code: Select all

bin/
data/packages/custom/
data/packages/source-python/
data/plugins/
engines/
packages/custom/
packages/site-packages/
packages/source-python/
plugins/


or with what Mister-Man suggests in mind

Code: Select all

bin/
engines/
packages/site-packages/
packages/source-python/data
plugins/example/
plugins/example/packages
plugins/example/data


but I personally think that having unused packages or data sitting somewhere is not a major issue, and should not be considered as one when deciding the folder structure. Although it would be easier for us server managers and therefore could be considered.

Either way: bin, data, engines, packages and plugins are by far my favorite words as they compliment each other nicely
Where do the languages go again?

[Ayuto]

Some great ideas were brought up here! And here is another one!

Code: Select all

bin/core.dll
bin/source-python.dll

// This won't be changed very often and only contains pure Python stuff.
engines/site-packages/

// If a custom package provides data, then place it somewhere in the
// custom package. e.g. packages/custom/my_package/data
packages/custom/
packages/source-python/
packages/source-python/data

// I'm fine with scripts, addons or plugins
plugins/

[mister-man]

or with what Mister-Man suggests in mind

Code: Select all

bin/
engines/
packages/site-packages/
packages/source-python/data
plugins/example/
plugins/example/packages
plugins/example/data


Thats my favorite... a little easier then my post. Next time first thinking then writing :-)

[Omega_K2]

I don't think it's good to place the data in the plugin directory, the point of having centralized directories is that you don't have to search everywhere for files. Plus, if you use FTP/access priviledges for other people, you can easily give access to certain files but not the entire script, in practice, this means:

DENY addons/source-python/
ALLOW addons/source-python/data

is much easier then
DENY addons/source-python
ALLOW addons/source-python/packages/source-python/data
ALLOW addons/source-python/packages/user/k2tools/data
ALLOW addons/source-python/plugins/myplugin/data
ALLOW addons/source-python/plugins/myplugin2/data

Where do the languages go again?

resource/source-python/translations/

Current structure: http://www.sourcepython.com/showwiki.php?title=Wiki:Administration+and+General+Usage

[mister-man]

Omega, I know what youre mean.
But its only easy if you want to give access for all data. if you give a user access only to data for a single package, you must do the same as in your secound example.
But in the secound example you can easy clean up your server as administrator from uneeded.

One central folder is only a benefit if more then one package/addon use the same data as another.

(postet form mobile phone sry for spelling)

[Omega_K2]

The issue of removing addons should be none if you have centralized folders you'll know where anything addon related is. Plus, for approved addon I'm sure I've read there is an es_install and es_uninsall equivalent planned, which can take care of removing/adding files for you.

[La Muerte]

well, if translations are already in a completely different place, then I don't see the benefits of putting the rest together either, so then those also would need to be added within plugins, and then so for each and every one. Which makes even more folders for multi language support for every plugin that has it, which becomes just ugly. I wouldn't see the point of putting data etc within the plugins folder when you can't add the languages in there.

/languages/EN
/languages/FR
/...

better to have this type of setup centralized rather then having these sub folders multiple times for each plugin

Therefore in my opinion we should just use centralized folders as it becomes cleaner if you look at the entire picture.

[mister-man]

My last years with CSS server shows me, that the most addons doesn't use es install. So I must do it by myself.

But what will you do to identifie the data from addon x? Place subfolders with addonname? I mean this is a better way, then mixing all in one folder

Edit1:
Okay, is these a better way?

Code: Select all

bin/
data/example/
engines/
languages/example/
packages/user/example/
packages/source-python/
plugins/example


[La Muerte]

data/rpg_players.sql
data/rpg_props.sql
data/scrabble.sql

?

a redundant amount of folders is also not good IMO

your reasoning is to put all files related to a plugin within the plugin folder, and it only makes sense if you do it with ALL the files related to the plugin.
but doing it with the language files just creates a seriously redundant amount of folders IMO. And if you can't do it with the language files, then it makes no sense doing it with the rest.


I wonder, should it be something like:

Code: Select all

bin/core.dll
bin/source-python.dll
data/myplugin.sql
data/packages/
engines/site-packages/
packages/mypackage
packages/source-python/
plugins/

or

Code: Select all

bin/core.dll
bin/source-python.dll
data/plugins/myplugin.sql
data/packages/
engines/site-packages/
packages/plugins/mypackage
packages/source-python/
plugins/

?

Isn't it exactly the same but the first is quicker to navigate trough? :P

[mister-man]

okay.... Your right. One data folder is enough

[La Muerte]

Edit1:
Okay, is these a better way?

Code: Select all

bin/
data/example/
engines/
languages/example/
packages/user/example/
packages/source-python/
plugins/example


I think there was decided earlier on to put languages within the Resource folder, rather then inside the addons folder.
as omega_k2 linked a few posts back: http://www.sourcepython.com/showwiki.php?title=Wiki:Administration+and+General+Usage
It sounds as if it makes no sense, but it kinda does again make perfect sense when you look a the total structure :P

[satoon101]

We already have set locations for cfg files, translation files, custom event files, and log files, all of which are outside of the ../addons/source-python/ directory. They are exactly where "should" be, imo. Having everything inside the plugin's specific folder makes absolutely no sense. Sure, it would make it easier to remove a plugin, but using that as the only purpose is not a good idea. Everything has its place. Putting everything in one folder just clutters it up. I couldn't have said this better myself:

Therefore in my opinion we should just use centralized folders as it becomes cleaner if you look at the entire picture.

So far, I am in agreement with the following structure:

• addons/source-python/
  • bin/
    • core binaries
  • data/
    • custom/ - data for custom packages
    • plugins/ - data location for plugins
    • source-python/ - data for SP's Python API
  • packages/
    • custom/ - custom packages created by SP scripters
    • source-python/ - SP's Python API
  • plugins/
    • user made addons to be loaded by "sp load"

I still think that the source-python binary itself should be left in the ../addons/ directory along with the .vdf file.

The above structure still leaves out the Python core as well as site-packages. I'm still not 100% sure where to put them and what to call them at this point.

Also, I really think that custom packages should have their own section in our download manager (once we have one of those). Those packages should not be included inside any plugin, but any plugin that requires it should mark it as such.

Satoon

[Omega_K2]

@Muerte: As I said it makes sense to mirror the directory structure within, so it should not be data/myfile, especially not if you have more then one file in which the addon uses. To maintain consistency.
Also the current implemenation only uses a single language file (like es, with en=,de=,whaterver= tags), no en/de/whatever subdirectories. As such, you'll have a single language file per plugin, or in rare cases multiple if the plugin uses more then one (internally!).

Reason for the location of language files (and event files) is that the game itself places it's translation files in the folder, like resource/csgo_english.txt

[satoon101]

As such, you'll have a single language file per plugin

That is not entirely accurate. When a plugin loads a language file, a second, server-specific, file is then created. Any translations added to the server-specific file will be used in place of the plugin's translations. This is mostly supposed to be used to add "new" languages to the support of the given plugin for the server. Most servers will probably not even bother with that file. And it would be more helpful for the server operators to report new translations to the plugin author to add in.

Satoon

[Omega_K2]

The above structure still leaves out the Python core as well as site-packages. I'm still not 100% sure where to put them and what to call them at this point.

Also, I really think that custom packages should have their own section in our download manager (once we have one of those). Those packages should not be included inside any plugin, but any plugin that requires it should mark it as such.

Satoon

I still feel like site-packages should earn it's place in packages/ as for the python folder, I don't know, I still think the best name for it is somthing like python_core or pycore and it should be sperate.

Agree. I think that could work very well if an addon system has dependencies, so you can just mark addons or packages as dependencys and with an installation feature (i.e. sp install) they could just be installed along if they are approved. (but I'd still like to see loadable sub-addons =P!)

That is not entirely accurate. When a plugin loads a language file, a second, server-specific, file is then created. Any translations added to the server-specific file will be used in place of the plugin's translations. This is mostly supposed to be used to add "new" languages to the support of the given plugin for the server. Most servers will probably not even bother with that file. And it would be more helpful for the server operators to report new translations to the plugin author to add in.

Satoon

Yeah, I actually meant the file that is shipped with the plugin.