WarcraftReamls.com
  FAQFAQ    SearchSearch    MemberlistMemberlist    UsergroupsUsergroups   RegisterRegister 
  ProfileProfile    Log in to check your private messagesLog in to check your private messages    Log inLog in 
Uploads
Goto page 1, 2  Next
 
Post new topic   Reply to topic    WarcraftRealms.com Forum Index -> WarcraftRealms.com Bugs
View previous topic :: View next topic  
Author Message
Isaura
Census Taker


Joined: 08 Oct 2006
Posts: 40
Location: Netherlands
WR Updates: 1,334,593
Isaura WR Profile

PostPosted: Fri Nov 02, 2007 6:15 am    Post subject: Uploads Reply with quote

Is there something wrong with the uploading?
The data from yesterday and the day before yesterday aren't processed.
I update on the site, not through uniuploader or something
So I was wondering if something is wrong?
Back to top
View user's profile Send private message
DM.
Census Taker


Joined: 03 Oct 2005
Posts: 1155
Location: Toronto, Canada
WR Updates: 841,833
DM. WR Profile

PostPosted: Fri Nov 02, 2007 7:24 am    Post subject: Reply with quote

Yup looks to be stopped,

http://www.warcraftrealms.com/dbstats.php

Last processed file was uploaded on 2007-10-31 10:34:27.
Current server time is 2007-11-02 08:23:45
_________________

Click my sig
Back to top
View user's profile Send private message
Hybuir
Gear Dependent Squirrel
Gear Dependent Squirrel


Joined: 06 Sep 2005
Posts: 1547
Location: Austin, TX
WR Updates: 2,883,877
Hybuir WR Profile

PostPosted: Fri Nov 02, 2007 9:08 am    Post subject: Reply with quote

/RollieSignal
Back to top
View user's profile Send private message Visit poster's website AIM Address
Balgair
Araiceil


Joined: 30 Sep 2005
Posts: 1697
Location: UK
WR Updates: 12,800,493
Balgair WR Profile

PostPosted: Fri Nov 02, 2007 10:31 am    Post subject: Reply with quote

Yep noticed that myself, it's moving but very very slowly and stops for a few hours at a time. Plus even when files are processed my total updates count isn't going up - it's stayed the same for about 3 days now, and I have had a few files processed in that time Wink
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --
Back to top
View user's profile Send private message
Rollie
Site Admin


Joined: 28 Nov 2004
Posts: 5376
Location: Austin, TX
WR Updates: 480,131
Rollie WR Profile

PostPosted: Fri Nov 02, 2007 10:36 am    Post subject: Reply with quote

Digging
_________________
phpbb:phpinfo()
Back to top
View user's profile Send private message Visit poster's website
Rollie
Site Admin


Joined: 28 Nov 2004
Posts: 5376
Location: Austin, TX
WR Updates: 480,131
Rollie WR Profile

PostPosted: Fri Nov 02, 2007 10:52 am    Post subject: Reply with quote

Well, looks like it's mainly just that the update process is slowing again. Moving to more powerful hardware helped out for nearly a year, but I knew it wouldn't be a permanent fix.

At first glance, I am faced with the same decision I had to make back in January.

1) I can either start pruning/archiving characters (cheap and not too hard)
2) I can move to even more powerful hardware (expensive and only a stop gap until we reach the next level)
3) I can move to server clusters that could constantly expand to meet demand (very expensive and basically out of the question)

At this point, I'm leaning towards #1. What I could potentially do is move all characters not seen in XX months into an archive table that is not looked at for updates which means smaller number of characters to check against when doing updates. I could also add maybe a once per month process that can check all latest added characters against the archive and if a match is found, it can 'un-archive' those characters. This adds some complexity rather than just archiving the data for good, but probably the best bet to be honest.
_________________
phpbb:phpinfo()
Back to top
View user's profile Send private message Visit poster's website
Rollie
Site Admin


Joined: 28 Nov 2004
Posts: 5376
Location: Austin, TX
WR Updates: 480,131
Rollie WR Profile

PostPosted: Fri Nov 02, 2007 11:03 am    Post subject: Reply with quote

Upon further investigation it appears that it is particular uploads from the same individual that is prematurely killing the process. Digging further.

One thing of note, if I looked to start doing archives, here's a few stats:

5M Chars not seen in 24 months
15M Chars not seen in 18 months
26M Chars not seen in 12 months

I think 12 months would be too soon, and 24 months wouldn't give enough benefit. Maybe 18 months is not a bad amount of time for a character to be dormant before being archived?
_________________
phpbb:phpinfo()
Back to top
View user's profile Send private message Visit poster's website
Rollie
Site Admin


Joined: 28 Nov 2004
Posts: 5376
Location: Austin, TX
WR Updates: 480,131
Rollie WR Profile

PostPosted: Fri Nov 02, 2007 11:08 am    Post subject: Reply with quote

Okay, it was WyriHaximus! Let's get him!!! Wyri, please PURGE or at least PRUNE your file. Also, can you check your UU settings as I am getting an upload from you every 30 minutes and surely the file shouldn't be changing that often?
_________________
phpbb:phpinfo()
Back to top
View user's profile Send private message Visit poster's website
Balgair
Araiceil


Joined: 30 Sep 2005
Posts: 1697
Location: UK
WR Updates: 12,800,493
Balgair WR Profile

PostPosted: Fri Nov 02, 2007 11:32 am    Post subject: Reply with quote

Phew, was wondering if it was me at first Wink 18 months sounds a fair compromise, especially if you can run a check now and again to make sure they haven't become active again (maybe not even as often as monthly, every 2-3 months should be enough).
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --
Back to top
View user's profile Send private message
DM.
Census Taker


Joined: 03 Oct 2005
Posts: 1155
Location: Toronto, Canada
WR Updates: 841,833
DM. WR Profile

PostPosted: Fri Nov 02, 2007 12:04 pm    Post subject: Reply with quote

LOL Wyri!!


And yea 18 months does seem reasonable.
_________________

Click my sig
Back to top
View user's profile Send private message
Hybuir
Gear Dependent Squirrel
Gear Dependent Squirrel


Joined: 06 Sep 2005
Posts: 1547
Location: Austin, TX
WR Updates: 2,883,877
Hybuir WR Profile

PostPosted: Fri Nov 02, 2007 1:17 pm    Post subject: Reply with quote

Rollie wrote:
Okay, it was WyriHaximus! Let's get him!!! Wyri, please PURGE or at least PRUNE your file. Also, can you check your UU settings as I am getting an upload from you every 30 minutes and surely the file shouldn't be changing that often?
Wyri's secrets come out! *scrambles for UU*

Why don't we just do both prune & archive
Back to top
View user's profile Send private message Visit poster's website AIM Address
DM.
Census Taker


Joined: 03 Oct 2005
Posts: 1155
Location: Toronto, Canada
WR Updates: 841,833
DM. WR Profile

PostPosted: Fri Nov 02, 2007 1:55 pm    Post subject: Reply with quote

Wyri's been getting a string of 0's all the time he submits.....
_________________

Click my sig
Back to top
View user's profile Send private message
Rollie
Site Admin


Joined: 28 Nov 2004
Posts: 5376
Location: Austin, TX
WR Updates: 480,131
Rollie WR Profile

PostPosted: Fri Nov 02, 2007 4:18 pm    Post subject: Reply with quote

Because his files have been unparsable.
_________________
phpbb:phpinfo()
Back to top
View user's profile Send private message Visit poster's website
WyriHaximus
Census Taker


Joined: 18 Oct 2005
Posts: 244
Location: Koedijk, Alkmaar, Noord-Holland, The Netherlands
WR Updates: 1,520,754
WyriHaximus WR Profile

PostPosted: Sat Nov 03, 2007 4:13 pm    Post subject: Reply with quote

Rollie wrote:
Okay, it was WyriHaximus! Let's get him!!! Wyri, please PURGE or at least PRUNE your file. Also, can you check your UU settings as I am getting an upload from you every 30 minutes and surely the file shouldn't be changing that often?
/me starts running for his life!

Anyway this is what happend:
Quote:
Yeah it was a realy stupid error. Using a script to check and if changed upload the lua save file. After rollie poked me I trew away the lua file on my server and the check file with the md5 hash of the lua file. The system I'm using is kind simple. Got my WS and my server. My WS copies the lua files each minute to my server. My server checks every 15 minutes if the lua file is update by using a md5 hash. If changed the script updates the md5 check file and uploads the lua file to WCR. Somewhere along the way something went wrong (we still don't know what exactly) but by trewing away both files the problem got fixed Smile.

_________________

My World of Warcraft Screenshots / Map
Back to top
View user's profile Send private message Visit poster's website
xpolockx
Superior Census Taker


Joined: 10 Aug 2005
Posts: 788
Location: Lynchburg, VA
WR Updates: 4,478,809
xpolockx WR Profile

PostPosted: Sat Nov 03, 2007 7:21 pm    Post subject: Reply with quote

Heh Wyri, that's pretty funny Razz Don't forget to turn on aspect of the cheetah while you run...

I think 18 months is a pretty good marker - most people that quit for longer than a year don't end up playing that same character when they come back (not everyone, obviously, but most of the people I've talked to who come back after being gone for a year or more roll new characters to get back into the game.) Of course, having a monthly check to make sure that archived characters haven't been seen recently is also a good idea. Anything to help out your poor servers Wink
_________________
US-Whisperwind:
Kayni, Resto Shaman
Scenario, MW Monk
Back to top
View user's profile Send private message Visit poster's website AIM Address
Balgair
Araiceil


Joined: 30 Sep 2005
Posts: 1697
Location: UK
WR Updates: 12,800,493
Balgair WR Profile

PostPosted: Sat Nov 03, 2007 10:43 pm    Post subject: Reply with quote

Also a point on archiving characters, perhaps have an even longer threshold on archiving ones on servers which consistently have very low submissions - it's perfectly possible for a server with average 1-5 submissions per month to just never see a character that only logs on for a few hours a week Wink
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --
Back to top
View user's profile Send private message
WyriHaximus
Census Taker


Joined: 18 Oct 2005
Posts: 244
Location: Koedijk, Alkmaar, Noord-Holland, The Netherlands
WR Updates: 1,520,754
WyriHaximus WR Profile

PostPosted: Sun Nov 04, 2007 9:46 am    Post subject: Reply with quote

xpolockx wrote:
Heh Wyri, that's pretty funny Razz Don't forget to turn on aspect of the cheetah while you run...
Sprint + Preparation + Sprint + Vanish ftw Very Happy!
_________________

My World of Warcraft Screenshots / Map
Back to top
View user's profile Send private message Visit poster's website
Skyfire
Trolling Enforcement


Joined: 18 Aug 2005
Posts: 746
Location: New Jersey
WR Updates: 44,279
Skyfire WR Profile

PostPosted: Sun Nov 04, 2007 9:48 pm    Post subject: Reply with quote

Much as I hate to say it (I would hate to lose my sig ;( ), 18 months sounds good. Of course, that leaves me set until this June, so who knows.
_________________
Admin on WoWWiki
Moderator, Blogger on Wowhead
Back to top
View user's profile Send private message
Rollie
Site Admin


Joined: 28 Nov 2004
Posts: 5376
Location: Austin, TX
WR Updates: 480,131
Rollie WR Profile

PostPosted: Mon Nov 05, 2007 2:28 pm    Post subject: Reply with quote

Well, appears the process has caught up already so after Wyr's files that were choking the system were cleaned out, we're back where we are supposed to be =)
_________________
phpbb:phpinfo()
Back to top
View user's profile Send private message Visit poster's website
Balgair
Araiceil


Joined: 30 Sep 2005
Posts: 1697
Location: UK
WR Updates: 12,800,493
Balgair WR Profile

PostPosted: Mon Nov 05, 2007 2:32 pm    Post subject: Reply with quote

Good to know - I noticed it seemed to be catching up fast Smile

Btw Rollie, could you take a look at this server and see if there's anything wrong with the display there, or if it genuinely is just zero entries on both sides? I'd kinda hoped it'd say 0:0 on the entries space if it was, the blank spaces leave me unsure if it's really 0:0 or just having issues Wink
_________________
EU-Draenor:
-- Sagart - Tairbh - Balgair - Buinne - Eilnich - Ruire - Dubh - Laidir - Naomh - Falaich --
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    WarcraftRealms.com Forum Index -> WarcraftRealms.com Bugs All times are GMT - 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
WarcraftRealms.com  


Powered by phpBB © 2001, 2005 phpBB Group