Uploads

[Isaura]

Is there something wrong with the uploading?
The data from yesterday and the day before yesterday aren't processed.
I update on the site, not through uniuploader or something
So I was wondering if something is wrong?

[DM.]

Yup looks to be stopped,

http://www.warcraftrealms.com/dbstats.php

Last processed file was uploaded on 2007-10-31 10:34:27.
Current server time is 2007-11-02 08:23:45
_________________

Click my sig

[Hybuir]

/RollieSignal

[Balgair]

Yep noticed that myself, it's moving but very very slowly and stops for a few hours at a time. Plus even when files are processed my total updates count isn't going up - it's stayed the same for about 3 days now, and I have had a few files processed in that time
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --

[Rollie]

Digging
_________________
phpbb:phpinfo()

[Rollie]

Well, looks like it's mainly just that the update process is slowing again. Moving to more powerful hardware helped out for nearly a year, but I knew it wouldn't be a permanent fix.

At first glance, I am faced with the same decision I had to make back in January.

1) I can either start pruning/archiving characters (cheap and not too hard)
2) I can move to even more powerful hardware (expensive and only a stop gap until we reach the next level)
3) I can move to server clusters that could constantly expand to meet demand (very expensive and basically out of the question)

At this point, I'm leaning towards #1. What I could potentially do is move all characters not seen in XX months into an archive table that is not looked at for updates which means smaller number of characters to check against when doing updates. I could also add maybe a once per month process that can check all latest added characters against the archive and if a match is found, it can 'un-archive' those characters. This adds some complexity rather than just archiving the data for good, but probably the best bet to be honest.
_________________
phpbb:phpinfo()

[Rollie]

Upon further investigation it appears that it is particular uploads from the same individual that is prematurely killing the process. Digging further.

One thing of note, if I looked to start doing archives, here's a few stats:

5M Chars not seen in 24 months
15M Chars not seen in 18 months
26M Chars not seen in 12 months

I think 12 months would be too soon, and 24 months wouldn't give enough benefit. Maybe 18 months is not a bad amount of time for a character to be dormant before being archived?
_________________
phpbb:phpinfo()

[Rollie]

Okay, it was WyriHaximus! Let's get him!!! Wyri, please PURGE or at least PRUNE your file. Also, can you check your UU settings as I am getting an upload from you every 30 minutes and surely the file shouldn't be changing that often?
_________________
phpbb:phpinfo()

[Balgair]

Phew, was wondering if it was me at first 18 months sounds a fair compromise, especially if you can run a check now and again to make sure they haven't become active again (maybe not even as often as monthly, every 2-3 months should be enough).
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --

[DM.]

LOL Wyri!!


And yea 18 months does seem reasonable.
_________________

Click my sig

[Hybuir]

[DM.]

Wyri's been getting a string of 0's all the time he submits.....
_________________

Click my sig

[Rollie]

Because his files have been unparsable.
_________________
phpbb:phpinfo()

[WyriHaximus]

[xpolockx]

Heh Wyri, that's pretty funny Don't forget to turn on aspect of the cheetah while you run...

I think 18 months is a pretty good marker - most people that quit for longer than a year don't end up playing that same character when they come back (not everyone, obviously, but most of the people I've talked to who come back after being gone for a year or more roll new characters to get back into the game.) Of course, having a monthly check to make sure that archived characters haven't been seen recently is also a good idea. Anything to help out your poor servers
_________________
US-Whisperwind:
Kayni, Resto Shaman
Scenario, MW Monk

[Balgair]

Also a point on archiving characters, perhaps have an even longer threshold on archiving ones on servers which consistently have very low submissions - it's perfectly possible for a server with average 1-5 submissions per month to just never see a character that only logs on for a few hours a week
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --

[WyriHaximus]

[Skyfire]

Much as I hate to say it (I would hate to lose my sig ;( ), 18 months sounds good. Of course, that leaves me set until this June, so who knows.
_________________
Admin on WoWWiki
Moderator, Blogger on Wowhead

[Rollie]

Well, appears the process has caught up already so after Wyr's files that were choking the system were cleaned out, we're back where we are supposed to be =)
_________________
phpbb:phpinfo()

[Balgair]

Good to know - I noticed it seemed to be catching up fast

Btw Rollie, could you take a look at this server and see if there's anything wrong with the display there, or if it genuinely is just zero entries on both sides? I'd kinda hoped it'd say 0:0 on the entries space if it was, the blank spaces leave me unsure if it's really 0:0 or just having issues
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --