Birds fly. Sun shines. And brother? Brutes shoot people.

Best Way to Log Data From Mutator/Game

For questions and discussion about UnrealEd, UnrealScript, and other aspects of Unreal Engine design.

Moderators: ividyon, Semfry, zYnthetic

User avatar jaypeezy
Skaarj Berserker Skaarj Berserker
Posts: 310
Joined: 25 Sep 2010, 04:32

Subject: Best Way to Log Data From Mutator/Game

Post Posted: 28 Aug 2016, 15:48

EDIT: This post regards solutions that could work in Unreal or UT99 easily, mainly UT99 as that's what I'm going to work with first. But always open to any suggestions/solutions.

I have a project in mind where I would like to begin pulling data from this dinosaur of a game engine, and insert it into a database.

Because I'm more interested in the database side of things I want to focus on getting the data out as quickly and easily as possible - which is, the name of a player, and the type of monster that they fragged. Furthermore I'd only log human player frags (bIsPlayer or check if it isn't a bot with some other function) and get the fragged monster name through a similar function.

Now, my question is: is there an ideal way to handle the logging? So far, I only know about the log() function and I tend to pepper that in code where required (with a bDebug variable elsewhere set through .ini to switch it on/off). This has the effect of logging events for the mutator in question along with other events that are getting logged from different things. But is there a way to output all the logged events in a mutator/gametype only when the round ends, so that they're in one place rather than interspersed throughout? (Even better, is there a way to write a separate log file altogether for a specific mutator/gametype?)
Last edited by jaypeezy on 28 Aug 2016, 17:20, edited 1 time in total.

User avatar sn260591
Skaarj Scout Skaarj Scout
Posts: 18
Joined: 02 Jun 2013, 15:40
Location: Russia

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 28 Aug 2016, 16:25

If Unreal227 is meant, then you can try SQLite3 wrapper http://www.oldunreal.com/cgi-bin/yabb2/ ... 1395035189
Sorry for my bad English

User avatar jaypeezy
Skaarj Berserker Skaarj Berserker
Posts: 310
Joined: 25 Sep 2010, 04:32

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 28 Aug 2016, 17:21

Wow, thanks for the suggestions! That's pretty cool, maybe Casey was even going to pop in/suggest that at some point. But it seems to rely on functionality specific to that version of Unreal, unless I'm wrong.

EDIT: Here's a related question. I know PostBeginPlay() and similar functions exist to perform certain tasks before a round starts, is there an equivalent handle in UnrealScript for doing stuff when a round ends? (EndPlay, EndRound, etc. something to that effect)

User avatar zYnthetic
Trustee Member Trustee Member
Posts: 507
Joined: 12 Nov 2007, 00:10

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 29 Aug 2016, 07:57

There's a possibility that the functionality may be present. I imagine it would have to be for ngstats.

Actually reading a bit more into it, ngstats was a 3rd party util that parsed logs from UT and probably not part of the uscript. NetGames was later purchased by MS and it seems to not be included in GOTY or later versions. Even so, there seems to be a replacement, UTStats, that functions similarly but uses SQL. I'm not sure if the source is available. If not, it at least serves as a proof of concept that it can be done.
Image

User avatar jaypeezy
Skaarj Berserker Skaarj Berserker
Posts: 310
Joined: 25 Sep 2010, 04:32

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 29 Aug 2016, 15:31

EDIT: Just found out about this thing: http://www.madrixis.de/undox/Source_engine/statlogfile.html Can I extend this class to create a function that logs custom events to a separate file?

Ngstats! I remember that, back when it was widely used in UT. I might dig around there some more. Mainly I would be interested in seeing if it logs data to a separate file (other than the main server.log, UnrealTournament.log, etc. as I'll explain below).

I did some tinkering around yesterday after my post and learned a few things:
    1. Logging killer/killed data was easier than expected
    2. There are indeed handles in UnrealScript that can defer code execution to the end/restart of a round, HandleEndGame() and HandleRestartGame(), respectively.
    3. It's even possible to do my logging to an .ini file.

I'm basically using the following statement, with the appropriate conditional checks (was killer a human or bot?) to log each frag:

Code: Select all

if (!Killer.IsA('Bot') && !Killer.IsA('Bots')) {
   log(MFL_EntryStart $ "('" $ Killer.GetHumanName() $ "'" $ MFL_EntrySeparator $ "'" $ Killer.Weapon.ItemName $ "'" $ MFL_EntrySeparator $ "'" $ Other.Class $ "'" $ MFL_EntrySeparator $ "'" $ Other.MenuName $ "')" $ MFL_EntryEnd);
}


Of course if I was logging to HandleEndGame as well then I would be storing each logged event somewhere, rather than putting it to the log file immediately. That's the one thing I still don't know how to do, because I don't think there's anything in UnrealScript that could store so much data (other than a really big string array, and even then you better believe I'm going to be cautious and enforce a limit of 256, 512 etc. logged entries).

In which case, I imagine something like this would suffice but I have yet to test it in a real game/code. I have a global array that's holding each logged event, I just have to push it to its respective location in an array of config string vars and then save the config:

Code: Select all

function bool HandleEndGame() {
   // if I've surpassed the log array limit, return what's appropriate for HandleEndGame, otherwise...
   if (ci > limit) {
      log("cut it out!");
      return false;
   }
   
   // ...save each logged event, iterating with appropriate array index checks + whatever:
   MyLogArray[ci] = GlobalSavedLoggedEvents[ci];
   
   // static save config
}


Saving to .ini, even with a hard limit, makes sense for two reasons:
    1. Again, I don't know of a good construct available in UnrealScript all this data for it to be handle at a later time, which would also allow for persistence elsewhere.
    2. It turns out I won't have programmatic access to the server logs to my server, at least not while the game is running. So I'd have to use some web console I have to manually get the data, which is fine, but I would prefer to automate this as much as possible. (cron job to get the .ini periodically seems doable)

bob
Skaarj Lord Skaarj Lord
Posts: 204
Joined: 23 Apr 2011, 02:24
Location: USA
Contact:

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 29 Aug 2016, 17:14

1)You can ask Casey to recompile the sql libary for ut99?

2) you can use UdpLink and some basic code to have unreal/ut spit data out tho udp to a outsde servce-like a java/python app that translates that data into a sql database.

3) UBrowserHTTPClient > post data> php > mysql

User avatar jaypeezy
Skaarj Berserker Skaarj Berserker
Posts: 310
Joined: 25 Sep 2010, 04:32

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 29 Aug 2016, 19:08

Hey Bob, maybe something lower level would be best here. I am however limited by the fact that I don't have complete access to the server, it's another company that's hosting and all I have is FTP.

I wouldn't want to bother people by asking them to recompile their work just to fit into UT99. :B And, of course, there are some differences between what be done in U227 vs UT99 code, I think. Nonetheless, such a module would be nice to have.

User avatar jaypeezy
Skaarj Berserker Skaarj Berserker
Posts: 310
Joined: 25 Sep 2010, 04:32

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 07 Sep 2016, 04:25

zYnthetic wrote: ...Even so, there seems to be a replacement, UTStats, that functions similarly but uses SQL. I'm not sure if the source is available. If not, it at least serves as a proof of concept that it can be done.


Okay, this pretty much took me some reading the UTStats source, googling "extends StatLogFile" and a little coding to figure out but my stupid idea of logging to a config is stupid and unnecessary. zYnthetic, huge thanks for pointing me in the right direction with UTStats/ngStats, that StatLogFile was definitely the thing to use in this case! \o/

Turns out writing to a custom log file with its own name, timestamp etc is very easy. The base StatLogFile class even includes handy methods for logging kills but, since I'm doing something different, I'm writing a lightweight mutator that calls LogEventString (my custom log file class) from ScoreKill (method available to check whenever a ScriptedPawn is fragged). It works perfectly for my needs.

Which leaves the task of inputting to a database, but (thankfully) I'm handling that with some bash/PHP script cronjobs.

Quick and dirty code:
► Show Spoiler

User avatar Buff Skeleton
>:E >:E
Posts: 4069
Joined: 15 Dec 2007, 00:46

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 07 Sep 2016, 05:46

I am really eager to see how this turns out for probably obvious reasons!
Image

bob
Skaarj Lord Skaarj Lord
Posts: 204
Joined: 23 Apr 2011, 02:24
Location: USA
Contact:

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 08 Sep 2016, 06:28

if i take your code as is in UG227, and try to use it to log simple data from spawnnotify in a mutator this is all it will log. this data has nothing to do with what i am logging lol

30.00 player Ping 0 0
60.02 player Ping 0 0
90.01 player Ping 0 0
120.00 player Ping 0 0
150.01 player Ping 0 0
180.01 player Ping 0 0
210.00 player Ping 0 0
240.03 player Ping 0 0
270.01 player Ping 0 0
300.01 player Ping 0 0
330.00 player Ping 0 0
359.99 player Ping 0 0
389.98 player Ping 0 0
419.96 player Ping 0 0
449.96 player Ping 0 0
479.95 player Ping 0 0
509.95 player Ping 0 0
539.95 player Ping 0 0
569.96 player Ping 0 0
599.99 player Ping 0 0
629.99 player Ping 0 0

gopostal
Skaarj Assassin Skaarj Assassin
Posts: 132
Joined: 01 Aug 2008, 06:35

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 09 Sep 2016, 17:06

Cumulative stats like kills, deaths, etc can be logged at HandleEndGame() and should be saved in an array until then. On-the-fly things like "player A killed player B" need to be logged as you go. Be careful with this though as writing to log files is inefficient and can lead to a noticeable impact on your server's performance if you overdo it.

Casey is a rock star with code but it's been my experience that send/receive from a UT server to a remote database can be sketchy to get right. We had a lot of trouble getting even simple player checks on a remote server to be solid, stable, and reliable. It often crashed the server. Overall it's a much better idea to log your statistics to your server then scrape them up and import that into your DB.

UTStats does this right though. It's just a lot of code to get your head around for something much simpler like what you want to do.

User avatar jaypeezy
Skaarj Berserker Skaarj Berserker
Posts: 310
Joined: 25 Sep 2010, 04:32

Subject: Re: Best Way to Log Data From Mutator/Game

Post Posted: 12 Sep 2016, 16:05

bob: I am mainly developing this for UT99 servers running ScriptedPawn mutators. Not sure if, or to what degree, it would work with Unreal of any version as its extending from that base class StatFile, that's part of the ngStats feature.

gopostal: Yeah, that is the approach I'm sticking with here, getting the data logged in a format I can work with, then automating tasks that download the scripts to a server, then parses them and uploads that data to a database.

I finally managed to come up with something basic. I'll post some update this week on what progress I've made.


Who is online

Users browsing this forum: No registered users and 3 guests