Translating...

Per communication from Rollie about two hours ago.

He has been working on the updates needed to get this site compatible with the Mists of Pandaria release. However his real life put the food on the table work has been extra heavy this past week and more.

Currently he is modifying his Development environment with the changes needed and tested.. to which he says "It's close". After he his sure he has all the changes in the database schema and the PHP linkage programming is finished he will be ready to deploy to the live environment.

Since the database needs to change to support the new class and split faction race there will be a [b:7f92f9014a]total shutdown of processing ahead of us[/b:7f92f9014a]. Rollie expects that the rebuild of the live database "will likely take a LONG time to process (as in 24 hours or so)."

Since this will take a long time... and with the expectation that new processing will place a heavy burden on the database machine.. he is looking at doing a swap out of the computer that supports the database with faster hardware. This extra might increase or decrease the time needed to get the site ready, but will certainly improve the processing speed for handling the backlog of data which is building up on your machines.

The final question that everyone has is "yes, but when will it be released?", the only answer we can give is the same as Blizzards.. "when it is ready."

A final comment from myself is that I see from the database stats page that people are still submitting using the old pre MoP version of the addon or (shudder) editing the version number in the current version savedvariables file [Very Bad Idea.. don't!!] [b:7f92f9014a]Please stop uploading now[/b:7f92f9014a] the database is currently at +8 Hours in backlog.. which will need to be finished processing before Rollie can make his changes on the live database. Rollie's time to support this site is very limited.. and we don't want to lose any window of opportunity for his updates because he has to wait for the backlog to clear.

Considering the amount backlog, maybe the best option would be to pull the site offline ASAP so it can catchup. There will be a couple of hours of EU maintenance shutdown of WoW tomorrow morning, which means it would be better, if WCR went into "hardware update mode" at earliest possible point in time.

Despite the loss of my own computer, my stockpile folders are starting to stack... Thank you to all for updates... I'll be watching... and stockpiling even more stuff, I guess. Good luck to all!

I totally think the site should be shut down. Lots of people don't read these forums, and you're never going to stop them all from uploading.

Rollie will do so.. when he gets to that point... I'm just trying to cut the backlog a little.
It is true not every CensusPlus user keeps up to date by visiting the forums and reading.. but then there are times even when people read the forums they don't understand or follow through with requests :roll:

@ bringoutyourdead

If/when you "talk" to Rollie next, could you ask him to reduce the time between uploads to 1 minute, or even better remove it completely, for a period of time, so we can upload all our stockpiled files?

With the current 15min time limit, it will take me a minimum of 9 hours to upload all my files, if I press upload at the correct time.. Most likely MUCH longer..

I'd be inclined to do the opposite and [i:30e1733b1e]increase[/i:30e1733b1e] the time between uploads, since everyone hitting the database with their entire backlog at once will likely overload the server and it'll take weeks to process :P

Last time, it was decreased to 5 minutes sucessfully...

Ahh no wonder it complained about me trying to upload 6.0.0 data and the latest was 5.0.2 and that I should get the latest version.

I agree with above, turn uploads off. Pointless if it doesn't work.

Hi guys,

Nice to hear that the MOP version ist on it's way.

Keep up the good work!

I honestly don't see how stockpiling is really gonna be all that useful anyway.

I set my data to auto-prune to 7 days. This keeps my uploads to the server (which are automatic every time I play WoW thanks to Uniuploader) fairly small, even when I get on an alt frenzy and play on like 15 different servers in the course of a week.

The website auto-prunes at 30 days. The only reason a history longer than that matters is to keep track of the speed people reached 90 or their guild history and frankly I don't really use that, myself.

I'm more interested in the present than in the past. Why stockpile?

Because LOTS of people are interested in the past. It's mitigated somewhat by all the faction/server/name changes people are able to easily do versus 2-3 years ago, but for long-term server residents, guild history is something they look at when taking applications.

I think your setting of auto-prune at 7 days likely puts you in the minority. My guess is 30 is much more common. I'm not sure what you mean by "the website auto-prunes at 30 days."

@Pennet -

I use the website for data like this:

http://www.warcraftrealms.com/census.php?serverid=-2&factionid=-1&minlevel=85&maxlevel=85&servertypeid=-1

Or this:

http://www.warcraftrealms.com/quickstats.php

Both of those only include stats from the last 30 days.

Hi There,

It appears that all the versions of the Census addons are not working.

The download version from the website causes continuous in game errors.

The new version from Curse does not have the right version to upload.

We seem to be completely broken for Census uploads.

Anyone have any workarounds or other ideas?

Love the functionality of this site.

Metalbeast

Unfortunately, we depend on Rollie alone to finish update on the site itself. Bringoutyourdead and Balgair already did a huge amount of work for version 6 of addon... now we just need a site that take all the info in...

I have an expanding pile of stockpiled files... and I guess Fuxie is slowly going insane with his pile, which is probably 5 to 20 times as big as mine... if he hasn't burned out already... *shudders*

I can only advice more patience and hope that Rollie is able to sacrifice the required time really soon... this forced upload break is getting rather long and there have been literally zero new posts in the past couple of days... *deep sigh*

Even my stockpile is growing a lot and I'm only changing file once a day, Fuxie's must be crazy size! The silence is worrying, I very much hope Rollie can make the time soon :?

Cata took 23 days to fix the site, so we have another week.

To be honest, I am starting to worry. Long breaks are not only bad for the data, they also cause longer term problems in the form of user loss... :( And my own computer still hasn't returned from repairs, which compounds my personal problems.

I also worry about Fuxie's mental health... if he has been keeping up the pace he said... he's now at 200+ files and counting... *shudders* Due to the loss of my own machine, I am at only about 40 files... but most of them are at 7+ MB size class. I fully expect to need several days for my uploads once the site is updated.

*deep sigh*

Epic Censi wishes everyone good luck!

[quote:9a90214645="Balgair"]Fuxie's must be crazy size![/quote:9a90214645]56and counting :P

Hi There,

I have been a programmer/developer for over 30 years. I also own several web design, web hosting, and Internet connectivity companies.

If the existing programming department is unable to complete, then I would be very interested in stepping in to keep this project going.

The censusplus.lua data file seems very straightforward and is working good at collecting server information.

I would also be willing to invest in some improved hardware to make the site more responsive.

The Best,

MetalBeast

I've been semi lurking around the site and the data for a few years now. Just yesterday I decided to get the addon and contribute.

It took me a while to figure out why I wasn't able up upload etc. It may be beneficial to make this information more obvious on the site, as potential new users may get confused/discouraged and give up (I almost did).

Update please !!!!

Bit hard to update with no information - I know as much as you guys do, and I'm sure if bringoutyourdead had heard anything he'd have told us. Just be patient, is all anyone can do right now.

I see that other week just passed so this update is taking longer than when Cata came out.

For me, it's now 24th... that means a month has gone by and in about 32 hours the last remaining shreds of Cataclysm data will be gone, leaving a few measly snippets of very partial MoP data provided by a handful of people who did not upgrade their addon.

I don't quite know what to say... some people have already given up, others have simply fallen silent because there is nothing more for them to say. Excluding myself, there haven't even been any forum messages within couple of days.

This is the Epic Censi, signing off for tonight...

Unfortunately Rollie's family suffered another loss.
He was called away for the funeral and all, but is now back in town.

He does want to get the site finished... he said he just had a "last couple of items" to take care of first on the site updates.

[quote:ef7ff054be="1974ER"]others have simply fallen silent because there is nothing more for them to say.[/quote:ef7ff054be]

This is it. I currently have 91 census files to upload, 92 by the end of raid tonight. That's 24.5 hours of uploading, if I do one every 16 minutes. And then it'll probably take a month or more before those, and everyone else's backlog is even processed. I'm seriously starting to wonder if it's worth my time.

I doubt there will be much backlog... as I said, lots of people have already given up. You, me, Balgair, Fuxie, Bringoutyourdead, maybe half a dozen others and that's about it, I think.

I just hope the fairly long 403 phase earlier today means we are finally close to actually getting back on track...

I doubt there will be much backlog... as I said, lots of people have already given up. You, me, Balgair, Fuxie, Bringoutyourdead, maybe half a dozen others and that's about it, I think.

I just hope the fairly long 403 phase earlier today means we are finally close to actually getting back on track...

At what point do we officially declare the site as dead?

It's official when Rollie says it, and not one minute before.

Sigh.

That is all.

Considering the reason which bringoutyourdead has given us for Rollie's most recent absence I'm more inclined to "sigh" at some of the impatience shown around here. Maybe you guys would put a website you haven't done much with for several years above your family, but I know I wouldn't... some things are actually more important. Back when Diablo 3 was released, on the release day I got a phonecall... my mother had been rushed to hospital. Well, that was that for D3 for a few days, family comes first :P

Last I heard was 2 nights ago btw, Rollie was close to having it finished but wanted some more data (which I gave him), hopefully it's nearly done now.

Hi, im new to the site and just joined. Hope all is well with Rollie and his family and wait for when i can begin to help with your data flow.

Welcome :) Haven't heard any more from Rollie which is a little frustrating, but then I don't know which part of the US he's in, he may well have been affected by power cuts the last few days, to be fair! Just hope he'll resurface sooner or later and finish things off, he sounded optimistic that it didn't need much more work last I heard.

If he's on the East Coast, he's probably had more to worry about than updating the site. In the NYC area there are still folks without power and there was a big fire that wiped out 100 homes the night of the storm.

Welcome aboard Madisaymad.

I don't know Rollie's location either, but judging by the server time difference, he isn't very far west from NYC. He has obviously had a hard year. I hope things improve and he can finish the updating process. I am close to 100 files and will probably break half gigabyte limit in a about 2 days...

Rollie is from Austin, Texas. --> http://www.warcraftrealms.com/forum/profile.php?mode=viewprofile&u=2

Anyone know why http://www.warcraftrealms.com/weeklyactivity.php is showing player activity if we haven't been able to upload?

I mean even if people are modifying their addon version to upload 'bad' data, there shouldn't be _that_ much bad data uploaded?

The data is "bad" in terms of not having anything concerning the new class (Monk) and races (Pandarean). Everything else is updated and the people showing as top contributors for this month are the only people who have been bothering to upload anything.

It also doesn't include anyone over lvl 85 so... no Monks, no Pandaren and no 85+ means it's missing pretty much everything that's actually been active for the past month. Which is why most of us are not uploading, since there's little point, if you care about accurate data.

Balgair is correct about almost all realms... and the very few realms that don't fit that pattern... Well, I at least prefer to retain the ability to update monks, pandaren and 85+s once Rollie is able to update the site.

Which... I assume will still take more time as there apparently aren't any new news. In the meanwhile, my stockpiles will be breaking the half gigabyte barrier soonish. Probably not today, as the maintenance by Blizzard is still going on and because I may need to devote excessive time on my own computer... which finally got back from it's almost SIX week journey to repairs and back... only for me to find out that after the repair, the Windows language has been pre-selected by the repair crew and unlike with brand new system, I have been disallowed choise.

Which means I still need to consult the customer service and tech support, because I don't want to mess the system up by attempting something that might not be allowed (in this case, try to system restore to English from the backup disks I made after purchase).

Update: I got a call from the tech support while writing this and am now slowly getting back on track with English Windows... Now I just need the site updated so I can start uploading data out of father's computer, which I need to use for some more time, as I don't have an external hard drive available to copy my WoW installation. Off to installing Firefox, anti-virus and then... WoW. *a very small happy dance, finally*

[quote:fd95a7ce9b="1974ER"]In the meanwhile, my stockpiles will be breaking the half gigabyte barrier soonish.[/quote:fd95a7ce9b]WOOT??? I have 88 files, of a total of 66,5MB waiting..

But then again, I guess, I do more playing and less census'ing than you ;)

You guys make me feel like I'm slacking, only got about 30MB to upload :P I've been entirely playing and no censusing though, or only censusing my home realm :)

I am sure you are not slacking... MoP is simply very time consuming... and at the launch rush, the counts were really high. Many factions resulted in over 4000 seen in one run... which often took over an hour to finish. Now things are settling back down. Of the realms I follow, only Outland locks on occasion... and even there, the queues are down to about 30 to 100... instead of 1900+ I saw on a few occasions shortly after launch.

Heh yeah my home realm had 3800+ on my faction at launch time, I've not really censused anything else (in any case it's kinda hard to tell where to start when ALL realms are on no/very low submissions) but at least EU-Draenor should be well covered when we can finally upload ;) MoP sure is a busy expansion for stuff to do, have never found time to go off my own realm really, too much to do while playing! :)

Kinda new here but is there any real point to gathering census data when that data is inaccurate and well wrong? I mean the census doesnt count monks pandas or any lvl over 85 so all the data anyone has at this time is kinda wasted correct ?

Madisaymad, let me fix that misconception quickly. The version 6 on www.curse.com can get data on monks, pandaren and 86+. The problem is that Rollie, the site admin has been swamped with work and family tragedies. Bringoutyourdead and Balgair, who are both long time actives, made a huge effort to update the addon itself, but unfortunately the site has to be updated by Rollie personally.

Until Rollie can find the time and energy to fix the site, those of us who still persist, keep on gathering what we can and stockpile it to wait for that day. I feel sorry that I can't help him. Last year, my mother died after battling two uncurable illnesses for over 9 months and in the past about 5 months, I have lost several members of my extended family, even personally carried two of the caskets to their final resting places, so I know how devastating loss can be. And unlike me, Rollie has his own family, which must come first.

I now feel sad again, so I'll sign off for now. Strength to Rollie and his family and happy censusing to all who still soldier on...

Epic Censi

OK thats great i did not know there was an updated census beyond the 5.0.2 version i got from here (as i said new here) i have downloaded the 1 from curse but do have some questions that I could use some answers on about the mod and so fourth. More than a few in fact.

Well, since we have more waiting to do... ask away. Sharing knowledge is a useful passtime. :)

Well is there any special I need to do with the mod length of time frequency that sort of thing?
can do multiple servers

Some english problems but I'll assume your asking what server need more help and how long and frequently you should do a census.

Someone more experienced will be able to give you a better answer but I would say that it's very hard to know what servers need more help until people can start uploading again and we see what server's are "red".

As for the length of time/frequency it depends on how much help the server needs. If your the only uploader of a server it would be good if you could make full census very frequently.

Otherwise a more experienced user can say how frequently they find it necessary to make a census on each server.

Nashinoshi already covered the main points. I'll try to expand a bit:

Where most help is needed will indeed be more evident after Rollie gets the site updated. For the time being, concentrating on the faction(s) you play on is entirely viable option. One important factor is that neither I nor anyone else can tell how many of us will be left once the site is back up to date.

So far, very few factions have been covered regularly by more than one person and many have gotten only the most superficial treatment, having been visited by someone roughly once a month or even less. :(

Ideally, each faction should receive at least 4 census runs per day, but that heavy coverage was true of perhaps two handfuls among almost 1200 existing factions before... How it will be... nobody knows.

In closing, anything and everything, no matter how little, helps.

bringoutyourdead, have you discussed with Rollie any ideas about giving you (or someone else he trusts) permissions to work on this site? I understand that this is his intellectual property and it might be difficult for someone other than him to continue exactly the way that he would. He could always make adjustments when he had time and give that person (or people) dirrection. It would reduce or elliminate the workload from him and give us a working model of the website at very least. I'm sure he would rather except some temporary relief in lue of seeing his creation lose so much viabilaty. Surely he would not be difficult for hime to find someone he trusts with the site that has the skills to update it given the community that he has created.

[quote:bfa6d42e13="Eightykiller"]bringoutyourdead, have you discussed with Rollie any ideas about giving you (or someone else he trusts) permissions to work on this site?[/quote:bfa6d42e13]AFAIK, this have been discussed/attempted..

The conclusion was, that the structure of the site, is so complex, that it would take longer time to get familar with it, than wait for Rollie to make the changes himself..

Correct me, if I'm wrong..

So what you're saying is, assuming the person knows how to read code, it would still take them longer 7 weeks (and counting) since the release of the new xpac to figure out how to update it?

I seriously doubt that. I only have a rudimentary understanding of coding, but I bet I could figure it out in seven weeks if I put my mind to it.

I'm basically done. I've been religiously copying over my .lua files since release, in the beginning 4-5 times a day, then 3, then 2, and then just once a day for the past couple weeks. But I honestly, sitting here right now, can't remember when I last copied my file over to my archive? 3 days ago? 4? It's not even something I think about at this point, because I feel like it's wasted effort.

I just checked, and it was 3 days ago. I have just over a hundred files now; at the current 15 minute update timer, that'll take me 27 hours just to upload. If the server came up in the next hour, I'm not certain I'd care enough to bother.

I have spoken with Rollie on a number of occasions. The last one was going on three weeks now. Up until then, I thought we were getting close to a new update and a possible partnership for using his data for a new WoW Population web site.

I definitely can accomplish what needs to be done to recreate this.

This is 100% Rollies deal though so if he is not responsive I don't think anyone else can do anything. We are all stymied.

To be honest 6 weeks of downtime between expansions is losing huge amount of followers, users, and more. It is really sad.

I own several web design and hosting companies and have been programming for over 30 years. I enjoy complex problem solving. I have deployed hundreds of systems around databases.

I plan to being a new system for WoW Population tracking in the next several weeks. I have reached out one last email to Rollie to see if he is interested.

[quote:64d1b210d9="FuxieDK"][quote:64d1b210d9="Balgair"]Fuxie's must be crazy size![/quote:64d1b210d9]56and counting :P[/quote:64d1b210d9]Almost doubled in number now.. 106 fileswaiting for upload... :(

34 here, I'm not bothering to change every day any more though, just every 2-3 days. I mailed Rollie last night asking for an update since it's over 3 weeks since I last heard anything, I'm not too hopeful but we'll see.

If I haven't seen something substantive by the 25th, which will be two full months since the xpac released, I'm done. I might keep running the addon for my own uses (which are few), but I'm done checking in every few days to see if anything's happened here. If it's not a priority for him, I don't see why it should be a priority for me.

DB updates are in progress. Hopefully everything will be fully operational within 12 hours or so!

Thank goodness for that, here's hoping it'll all work fine when it's up! :)

DB backup ran late into the night so I went to bed. Backup complete, table rebuilds in process.

Great to see you back!

So, when it completes and comes back up, do we immediately hammer the site as hard as we can with uploads, or should we be gentle with it at first?

Nah, just unload on it =) I'll relax all the timing constraints so that older data will get processed/updated

WOOT... :D

I can't wait to get home, and upload my 108 files.." :mrgreen:

Very thankful now I am keep uploading now

Due to something going wrong, I kept getting sent back to the "tempdown" page. Apparently the site has actually already been up a rather long while... *deep sigh, another of relief*

I will be starting to dump files from my father's computer shortly... 700 MB exceeded and almost 140 files in total to go from two different computers... *another deep sigh* This will take a while... :D

Thank you for getting the site back on its feet Rollie! Three cheers! :D

There might be an issue with manual uploads... and file size... as another message noted that at 6MB it failed.. and that file was later uploaded without issue using Uniuploader.

Confirming the problem. Manual upload fails at 6+ MB. Which in turn means that I will not be submitting more data until this problem is corrected as I will NOT be manually altering 135+ files. Fix needed ASAP!

[quote:e1f9a3e0e0="1974ER"]Confirming the problem. Manual upload fails at 6+ MB. Which in turn means that I will not be submitting more data until this problem is corrected as I will NOT be manually altering 135+ files. Fix needed ASAP![/quote:e1f9a3e0e0]I understand you won't manually edit 100+ files, and a fix is required..

However, you could use Uniupload to upload files, even the stock piles ones.. You just have to change what file it uploads for each upload..

It's not a fix, but a work-around.. ;-)

Unfortunately, that option is not available for me due to the current circumstances. I need the fix, before I can resume uploads.

Not sure how often Rollie can check the forum so I've mailed him to let him know about the filesize issue and also that the processing seems to be way behind - hopefully he'll take a look soon.

Rollie says:

[quote:e45001c586]Max file upload size should be 10M now. Anything larger and I'm afraid it'll bugger up the process.[/quote:e45001c586]

Hopefully that'll fix your issues ER :)

He also says regarding the processing issues:

[quote:e45001c586]Looks like there are some UniUploaded files that are choking up the process. Each time the file that is choking it up is a UniUpload file from the same IPAddr, but not someone who is submitting username and password so I am unable to contact them. One thing to note is that each of those files is over 10M so it could be the general filesize.

I have removed the other files they have submitted so we'll see if that helps. [/quote:e45001c586]

If anyone here recognises themselves (giant files via UU, while not logged in), try and tone down your filesizes a bit please ;)

[quote:31907da9ab="Balgair"]Rollie says:

[quote:31907da9ab]Max file upload size should be 10M now. Anything larger and I'm afraid it'll bugger up the process.[/quote:31907da9ab]

Hopefully that'll fix your issues ER :)

He also says regarding the processing issues:

[quote:31907da9ab]Looks like there are some UniUploaded files that are choking up the process. Each time the file that is choking it up is a UniUpload file from the same IPAddr, but not someone who is submitting username and password so I am unable to contact them. One thing to note is that each of those files is over 10M so it could be the general filesize.

I have removed the other files they have submitted so we'll see if that helps. [/quote:31907da9ab]

If anyone here recognises themselves (giant files via UU, while not logged in), try and tone down your filesizes a bit please ;)[/quote:31907da9ab]

This is why I put auto-prune in my census+ addon.

I would often play on several servers and have like 6-7 realms of data to upload. Then when I stuck to just one server, every upload was huge every time I logged out of the game because of all that past data.

I now auto-prune at 7 days. I just set it up to run the prune function every time it displays the results of a scan. I personally never use the data in-game, I view it on the website, so theoretically I could even set it to prune at 1-2 days and I'd be fine.

But this was based on my never pruning it at all (I'm a "set it and forget it" type of person, that's why I use uni uploader) and I actually got contacted by Rollie saying that I was making the database clog up. So, now I autoprune.

I'm thinking an autoprune of at least 30 days should be a default part of the addon anyway. If you haven't seen someone online in 30 days, do you really care about that person anymore? Does data that's over 30 days old really need to be uploaded?

What's going on now is a special case because the site has been down, but when the site is up and you're using uni uploader to upload every time you log out of WoW, do you really need to upload over 30 days of data every time you log out of WoW? (Especially if you log out several times a day, like I do when developing addons of my own, or take breaks and do something else and come back.)

I like the idea of autoprune, maybe something Rollie/Bringoutyourdead can consider adding for future addon updates, at least as an option. I usually remember to prune mine regularly, but occasionally it does stack up a bit, so an automated one for times when I forget would be very useful indeed.

I'll look on father's computer when I have a chance to, but I am sure Rollie fixed the problem so I can resume uploads later tonight.

As for me and prunes and purges. My current .lua is less than 30 HOURS old and has already exceeded 3 MB. Also, to be honest, I did NOT expect to be unable to upload for two months nor to lose two computers within less than three weeks... nor the latter spending a total of almost 6 weeks in repair shop and in transit back and forth.

Also, in the grand scheme, as far as I can tell, a significant number of users census so little they basically never need to purge and prune happens maybe every 4 or 6 months. Long ago, when I was censusing a slightly larger number of factions, I partially in joke noted that I would require a /censusprunehours X type of command. Now things are no longer quite that extreme, but data still stacks up pretty fast. :D

In any case, thank yous and good luck to all!

Busy Epic Censi, signing off for a little bit. :D

[quote:4f95cfa61f="Balgair"]I like the idea of autoprune, maybe something Rollie/Bringoutyourdead can consider adding for future addon updates, at least as an option. I usually remember to prune mine regularly, but occasionally it does stack up a bit, so an automated one for times when I forget would be very useful indeed.[/quote:4f95cfa61f]Auto-prune is not always good..

Some people use CensusPlus to keep track of data on their own server(s), and never upload.
Prune the data, and you kill their leisure..

[quote:c67cf0c3da="FuxieDK"][quote:c67cf0c3da="Balgair"]I like the idea of autoprune, maybe something Rollie/Bringoutyourdead can consider adding for future addon updates, at least as an option. I usually remember to prune mine regularly, but occasionally it does stack up a bit, so an automated one for times when I forget would be very useful indeed.[/quote:c67cf0c3da]Auto-prune is not always good..

Some people use CensusPlus to keep track of data on their own server(s), and never upload.
Prune the data, and you kill their leisure..[/quote:c67cf0c3da]

I did say "as an option" - this way, those who want it, can enable it, those who don't, can keep their full set of data :)

Well, three first of my "post-hard drive destruction" .luas have been processed successfully. So far, so good. Three down, over 140 to go, from which about 50 still wait for me to get around to uploading them. Since there is a massive backlog, I am going to give the queue some time to catch up... already uploaded about 80 files this morning, leaving all that is on my own computer as still to do... :D

After those three files, I am now at about 10% of Balgair's figures on the monthly top 10 lists. It will be really interesting to see how things turn out by the time the month ends. :D

I think you'll be top by a very very long way :P I only censused my own server from MoP launch until the day the website was fixed, surprised I was top with that tbh, you'll certainly pass me easily if you've been censusing multiple servers all the while (which I guess you have from the number of large files ;))

We'll find out fairly soon if nothing major goes wrong. :D One also has to keep in mind that if things had gone normally, I would have made roughly 240 to 360 submissions within two months. Now the figure will be well below 200 for a periods that actually slightly exceeds that.

Additional unrelated update note... have we lost the ability to edit already sent forum posts? Or is Epic Censi suffering from less epic blindness? :(

You're in the news and announcements folder, which works differently to other folders, and often does weird stuff :P Should be fine to edit posts in other sections of the forum, at least I'm not aware of any changes, just know this one's always been weird compared with the others.

The search I use on the site is this:

http://www.warcraftrealms.com/census.php?serverid=-2&factionid=-1&minlevel=90&maxlevel=90&servertypeid=-1

Somehow I think there's much more level 90's than that.

I don't think the site's actually working, yet.

250k sounds fair tbh, there's also 255k lvl 90s in Europe as well, and given there's only been 300k characters of any level seen in the last 7 days, it seems pretty reasonable from the available data.

Yes, I'm sure there's a lot more lvl 90s than that out there, but look how many servers have no data since before MoP, or only have 1-2 censuses run on them in that time; there'll be many thousands upon thousands of lvl 90s that haven't been seen. It's the lack of data causing those numbers, not the site, I'd say.

Did you actually click on the link or did you fail to look?

I'm pretty sure there are more than 21 level 90 Humans, for instance. And more than 30 level 90 Monks.

Also:

- Clicking on a class immediately removes all of your other search strings (level, faction, realm, race)

- Unable to filter by race at all (and clicking on it removes your other search strings)

I did click on the link and read the fairly obvious 249k characters, which seemed perfectly reasonable so didn't look further - might've helped if you'd made it clear what you were looking at so I didn't just focus on the obvious ;)

Seems there's something up with the other figures, yes, I'll pass it onto Rollie.

Edit to add: filter by race works alright for me, except for Pandaren, which indeed don't function. Mailing Rollie with all this :)

While you're at it, this page doesn't have pandas or monks:

http://www.warcraftrealms.com/quickstats.php

Lol, sent the mail 5 mins ago - will mention that in my next mail to him then if he doesn't check here first ;)

Although, looking at that link, some of it hasn't been updated since vanilla lol (only 8 races and 9 classes outside the race/class stats pane) - methinks it's not very high up on the website priorities :P

Speaking of the quickstats page, I've always wished I could filter those results similar to the first link. It gives the race-class breakdown I want, but not the filters: I'm only interested in max-level characters of US realms.

Actually, now that I look at quickstats, it doesn't have any races or classes other than classic WoW. Not even BC stuff. o.o

Actually, as I noted in another thread, that page needs a major overhaul. Some parts are more up to date than others, but none are ok.

Quick race stats has only 8 races.

Quick class stats has 9 classes.

Quick race class stats has 14! races (including Unknown :D ) and only 9 classes.

And now, I have been awake far too long, so I am off to bed and will look around again in the morning. :D

Hmmm... queue spent a rather long while "stuck" and my files:

Nov 27, 12 15:56:52 0 0 E
Nov 27, 12 15:19:36 0 0 E

misprocessed. :( I wonder if they caused the long delay. :(

My apologies to all, seems that my files are causing the queue indigestion...

Nov 28, 12 00:26:00 0 0 E

and

Nov 28, 12 00:29:14 0 0 E

also indicate failures. :(

May be worth mailing a couple of them to Rollie to see if he can see the problem, assuming it's not just size - rollie at warcraftrealms dot com. You're not the only one anyway, it was hiccupping before you started uploads so not to worry, it happens :) Eventually your huge collection will process (or error) and we'll get back to normal :P

Äh, well... there is just the small matter of me still having an almost equally large pile on my own computer... those files that are currently in the queue were only all that was on father's. Also, unfortunately, due to the large number of files and a small error I made, I have no way of telling which files are causing problems. :(

*deep sigh*

Nov 28, 12 00:47:27 0 0 E

and

Nov 28, 12 00:51:02 0 0 E

also failed with one success between them. I have no idea why some files fail and others work fine. There were no differences in the manual upload processes, file size is obviously variable, etc.

I re-apologize for the delays I am causing... and to make matters "worse", I will be dumping even more data into the queue somewhat later today...

Going to try tagging these in some way so if it chokes on any of them, you can identify which file it was? Might help with figuring out what the issue is.

This process is out of my hands...

Nov 28, 12 00:52:54 0 0 E

also failed... :( I am pretty sure Rollie can't babysit the queue, so I'll just have to live with whatever manages to make it through successfully. *deep sigh* All files went out successfully as far as I could tell at the time, so I have zero idea on why some of them are failing.

I'll just start feeding in the rest in a little bit.

Strange... now that one registers as success and the next one as faulty... *scratches head* Strange beyond my limited understanding... Oh well, just to let everyone know that I have now uploaded everything I had ready. Which currently translates into about five and half pages of not yet processed files.

A few hours of sleep later...

Nov 28, 12 00:58:13 0 0 E
Nov 28, 12 00:56:32 1936 9676 Y
Nov 28, 12 00:54:30 0 0 E
Nov 28, 12 00:52:54 2067 9919 Y
Nov 28, 12 00:51:02 0 0 E
Nov 28, 12 00:49:43 884 5027 Y
Nov 28, 12 00:47:27 0 0 E
Nov 28, 12 00:45:49 1398 6696 Y

Like a Swiss cheese and no idea why... *deep sigh*

hi,
any chance of linking to the latest addon, to run censuses with,at the top of this thread? I know Ive seen it somewhere,but am having trouble finding it (assuming 5.02 is not the current one, since dated 2010?!)
thanks :D

Good idea, will do that! http://www.curse.com/addons/wow/census-plus is the latest version :)

Many hours later... a few successes, then more Swiss cheese...

Nov 28, 12 01:26:03 0 0 E
Nov 28, 12 01:24:43 1011 5725 Y
Nov 28, 12 01:23:20 0 0 E
Nov 28, 12 01:21:52 1357 6832 Y
Nov 28, 12 01:20:28 0 0 E

*sigh*

Brillig, thanks :D
[quote:3d1ee9418f="Balgair"]Good idea, will do that! http://www.curse.com/addons/wow/census-plus is the latest version :)[/quote:3d1ee9418f]

Even more hours later... the queue has slown down to a permanent crawl and the Swiss cheese just keeps piling up...

Nov 28, 12 01:32:00 0 0 E
Nov 28, 12 01:30:36 874 4834 Y
Nov 28, 12 01:28:55 0 0 E
Nov 28, 12 01:27:40 845 4661 Y

Just curious, how many more from that batch of uploads are left?

1974ER's files to be processed.... 27 uploaded on the 28th then 2 days later
52 files, 2 files on dec 1st and 1 on the 2nd. Give it about 2 1/2 to 3 weeks
to clear up and things will be back to 'normal'. Sucks that he has so many
errors not only in server lag time but the loss of census credit to his
account.

I most seriously hope it won't take that long... Two months of enforced break was apparently too much for the queue engine, too. :/ Strange thing about the errors is that they seem to mostly alternate with successes. It's unfortunate that Rollie is so busy, it would be nice to have some sort of input from him. It has been a very hard year for him, so I hope things start going better as Christmas is approaching fast.

Also, I would be happy with much smaller figures if it meant all files process correctly. All the failures that slip through mean some data is permanently lost. And judging by the numbers in the successes, in some cases the data loss is rather significant.

Seems the queue is struggling again...

Nov 28, 12 01:59:57 0 0 E

*sigh*

The first 43 files, taking in failure rate and time per failure, has taken about
6 days and you have over 80 queued so, if that continues, 11 or more days,
2 weeks tops plus our files ;) , is not out of range. I hope for a smaller
failure rate not only for time but your count. As for Rollie, absolutely family
comes first and I wish him well and better days to come. I'm glad he could
keep such a creation as this site running as a hobby/interest considering life
as it happens.

I've mailed Rollie to let him know about your files ER, maybe if he has a spare minute he can take a look and see what's wrong with them. Can't imagine what's wrong myself, since I'm sure you're doing nothing different to usual!

Padanfain, as far as I am aware, due to certain technical quirks most other people who stockpiled had already cleared their piles by the time I began... So what remains of November... is mostly me.

Thank you, Balgair. As I noted earlier, I have no idea why the files go awry... and the success, fail, success, fail, success, fail pattern that is evident multiple times suggests to me that the problem might actually be in the queue, not in the files themselves. It's also possible that I simply put in too many files in too short time period... but there have been other such times when it has worked... and the pattern is not fully systematic, which renders it even weirder.

I am also assuming that if there were no failures, the queue would move significantly faster, because it seems to jump forward when multiple Y statuses are achieved and then halts to more or less complete stop upon hitting E.

I expect there isn't anything wrong with the files.. instead I expect there is a timing issue in Rollie's processing that blip in the data being read in and stored.. where either the PHP is getting or feeding faulty data or the database engine is getting confused and saying that it has seen data it hasn't really or isn't seeing the data it has been given.

When the process errors out there is time for the system to settle down.. and the next file processes fine. It is possible that rapid change of realm keys with the random cross-realm entries might be giving fits to the buffering (cache problems, memory fragmentation etc.) in the database. This is something that can be fixed by tweaking the configuration parameters but you can't do the tweaking until you see that limits you are hitting during processing.

If you remember Rollie wanted to upgrade his database hardware we don't know if he was able to do that before he opened the flood gates.

Which is another way of saying that only Rollie can know what is causing the problem.. and he won't know until he can take the time to analyze the database logs.

Rollie was looking at it earlier, don't know if he reached a final conclusion yet but he did mention that the file he was looking at had over 56k characters in it and took over an hour to process, so maybe size could be an issue there....

Admittably size could be an issue... but it's unlikely to be the only reason, as files in excess of 80000 characters were pretty much routine for me before site was updated. Also, with a quick glance... the previous time I had a file failure prior to it... was on May 15th... more looks... next failure... Sep 10th, 2011!

Overall, both the queue and my files have been pretty reliable, so I have a very hard time trying to understand why they are now encountering multiple errors in the space of just few days when preceding over a year contains only two, extremely far apart from each other, too.

Yeah, mine tend to be something similar, at least I've pruned chunks of 40k before now and never had trouble. Wondering if the database is struggling in another way, such that it can no longer handle such big files. If so, that's a headache, but maybe there's some other explanation, we'll see when Rollie investigates further I guess.

Unfortunately, it seems the situation is getting worse as I am now seeing multiple failures in row... :(

Nov 28, 12 02:12:37 0 0 E
Nov 28, 12 02:11:14 0 0 E
Nov 28, 12 02:05:16 0 0 E
Nov 28, 12 02:03:55 2871 9726 Y
Nov 28, 12 02:02:42 0 0 E
Nov 28, 12 02:01:15 1956 8662 Y
Nov 28, 12 01:59:57 0 0 E

Woot! This is lookin better 1974ER :)

Nov 28, 12 02:27:05 2631 8195 Y
Nov 28, 12 02:24:41 576 2035 Y
Nov 28, 12 02:19:45 1053 4294 Y
Nov 28, 12 02:16:40 1953 4991 Y
Nov 28, 12 02:15:41 409 2084 Y
Nov 28, 12 02:14:35 1929 6379 Y
Nov 28, 12 02:13:32 1964 6528 Y

It did... :/ Now it has hit another snag at:

Nov 28, 12 02:28:29 0 0 E

as it has been at that over 3 hours now.

Luckily, including that, only 14 files from 28th remain (from me, that is :D ).

Almost 12 hours later... there have been more failures and queue has barely moved...

Nov 28, 12 02:35:55 0 0 E
Nov 28, 12 02:34:26 2509 7691 Y
Nov 28, 12 02:32:58 0 0 E
Nov 28, 12 02:30:14 2846 8760 Y
Nov 28, 12 02:28:29 0 0 E

*deep sigh* At this rate, it will take an additional 24+ hours for the 28th to finish... :(

*shakes fist at 1974ER*

You have hogged the queue log enough.. Give it up, for the rest of us.. ;-)

*deep sigh* I am sorry Fuxie...

28th is almost done, but there has been yet another snag:

Nov 30, 12 14:22:14 0 0 N
Nov 28, 12 03:01:50 0 0 N
Nov 28, 12 02:51:40 0 0 E
Nov 28, 12 02:48:44 1317 5847 Y

After that, queue should reach 30th fairly quickly... but what happens when it gets there... remains to be seen. :/ *worries*

I really hope Rollie does something.. Failed processes and uber slow process is disrupting the fun..

Well, as far as I am concerned, 28th is done. Queue is now chewing on someone else's files for that day. It will be interesting to see how things go from here...

Fairly long time later... it seems the problem is not restricted to my files. The queue seems to have gotten stuck again at 2012-11-28 03:07:36.

That [i:d11053b733]may[/i:d11053b733] just be it stopping for a rest for a few hours after all the work it's had to do on your files ;) We'll see soon enough when it gets moving again, if it sticks on any more. I've got one at Nov 28, 12 09:19:35 which should be fairly small, but if I'm on when it reaches that I'll see how fast it processes.

28/11-12 22:56:00 processe fine...
0 new, 8 updates.. LOL..

Seems it was one of my morning runs..

Queue has now started going through my submission pile from the 30th, so far so good. :)

[quote:0b4d4f6fea="1974ER"]Queue has now started going through my submission pile from the 30th, so far so good. :)[/quote:0b4d4f6fea]15ish files processed, without errors..
Funny how the othes failed at such high rate...

Maybe some files were damaged during one of your crashes?
So you still have the files, so you can check is it's just garbage?

When a disk crash, sometimes, the files seems intact (timestamp, filesize, name etc all match up) at first glance, but when you open it, it's pure garbage..
Maybe this is the case here?

*awake again*

Nov 30, 12 15:20:42 0 0 E
Nov 30, 12 15:18:51 1006 4633 Y
Nov 30, 12 15:16:07 0 0 E
Nov 30, 12 15:14:35 751 3459 Y

Unfortunately, we have been hit by more errors.

As to your questions Fuxie... the data of 28th comes of the hard drive of my father's computer, which as far as I can tell is working just fine and the data of 30th comes of my repaired computer, which also appears to be working just fine.

The data that was on the previous hard drive of this machine was completely destroyed and never even submitted. The guarantee repair replaced the entire drive and everything that was on it, was lost... that everything included 6 .luas, about 30 MBs worth of census data, along with almost 3 weeks various personal notes and other things I lost permanently.

There might be a minor amount of data on the hard drive of my old machine, but it's now irrelevant, because WCR requires chronological order. Data, which for the most part has been updated over 100 times (literally, this is not a joke, without failures figure would already exceed 120 and with over 40 more rounds to go) since by now, is pretty worthless as things now stand.

I also took a look at the 15:16:07 file... to my non-expert eye and at a fairly quick glance... it looks utterly ordinary in all respects. It's not even direct size issue, because the file 15:18:51 is almost half a MB bigger and has Y status.

I fear that only Rollie, if anyone, can say what's going wrong. I obviously lack the required expertise. :(

Over 4,5 hours later...

It still stuck on:

Nov 30, 12 15:20:42 0 0 E

*deep sigh*

I guess we might get to December by (end of) next week... :(

The last large file I had was uploaded before yours that failed on 30 November. My next wasn't uploaded until 5 December.

[quote:d5305c9134="1974ER"]I fear that only Rollie, if anyone, can say what's going wrong.[/quote:d5305c9134]
Set up a /RollieSignal He'll be looking at it soon. (Life happens, and plus it's the holidays!)

Thank you, Hybuir. I am welll aware that the timing is really bad, but the queue is getting worse, not better, and it's not only bad for me and my figures, but the data and other people, too.

Nov 30, 12 15:32:07 0 0 E
Nov 30, 12 15:28:54 0 0 E
Nov 30, 12 15:26:36 1226 5793 Y
Nov 30, 12 15:24:32 0 0 E
Nov 30, 12 15:22:16 1414 6075 Y
Nov 30, 12 15:20:42 0 0 E
Nov 30, 12 15:18:51 1006 4633 Y
Nov 30, 12 15:16:07 0 0 E
Nov 30, 12 15:14:35 751 3459 Y

From my point of view, the Swiss cheese keeps piling on and the backlog gets worse by the minute... In fact, if I am not mistaken... in the past about 25 hours, queue has managed to move about 7,5 minutes... :(

Hmhrrr... some files went through, now the queue is stuck yet again at:

Nov 30, 12 15:44:18 0 0 E

Luckily, less than 20 files of 30th remain... but at this rate, they'll still take at least a couple of days. *yet another deep sigh*

Uhhh.. *yawn* I didn't sleep at all well and seems the queue didn't do well either... :(

Nov 30, 12 15:49:52 0 0 E
Nov 30, 12 15:47:49 0 0 E
Nov 30, 12 15:46:07 1045 4615 Y
Nov 30, 12 15:44:18 0 0 E

Less than 6 minutes in many hours and most of it failed... :(

Tuesday morning, sort of... :D It's a bit past my local midnight, that is.

Sadly, there were a few more errors... :(

Nov 30, 12 15:55:37 0 0 E
Nov 30, 12 15:53:55 1089 4739 Y
Nov 30, 12 15:51:47 0 0 E
Nov 30, 12 15:49:52 2476 9240 Y
Nov 30, 12 15:47:49 0 0 E
Nov 30, 12 15:46:07 1045 4615 Y
Nov 30, 12 15:44:18 0 0 E

...but now, only one file from 30th remains from me... and it's actually, if I recall correctly, already a post-stockpile file, so to speak. :)

OMG FuxieDK!!!!!

Nov 30, 12 18:31:13 0 1 Y

Now that is epic!! How? How? WoW! LOL

I think there is something wrong with the process, now one of my files errored.

Nevermind. I swear there was an E there, but now it shows a Y.

While a file is being processed, it shows an E, which is then changed to Y when processed completely..

Sorry for my question, but...
I'm not quiet familiar with the queue system.
Is the queue finally moving to december?
Is there any hope that the queue is getting better without Rollie's help?

greetings from Austria

As I write these words, it in fact already has and a few people have made it to the December top 10 lists. Now that my massive stockpiles are gone, I hope the queue will move forward more swiftly, as it appears to be doing. :) Even more importantly, I hope my files don't cause any further snags in it.

My apologies to Rollie for causing a lot of extra work. :( Thank yous to Bringoutyourdead, Balgair, Hybuir, Fuxie and others.

Epic Censi wishes everyone a more successful December, both in game and otherwise! :)

I do Censuses for selfish reasons... I want to be tracked. And I want to be #1, but I'll settle for staying in the top 10 :)

*very, very deep sigh*

It's my very sad duty to report that queue is still broken. :(

Dec 01, 12 18:16:44 0 0 E
Dec 01, 12 14:57:22 1650 7562 Y
Nov 30, 12 18:43:32 1405 6498 Y

The first two post-stockpile .luas succeeded... the 3rd failed. :( And no, it's not being worked on as the DB stats page says:

Last processed file was uploaded on 2012-12-01 23:33:13.

As I see no reports from anyone else having massive failures, I have to conclude that the problem is somehow, unexplainably, created by me. My current, rather sad conclusion is that I may have to quit in order not to ruin the site for others... HELP!!!

Until I hear otherwise, I am going back to "stockpile mode".

This is Epic Censi, signing off... :(

Epic Censi, how large are your files?

Size isn't the issue as such..
One of the first uploads when the DB was opened up again had over 12,000 new characters and more then 22,000 updated characters.

That was more then 3x Epic Censi's typical update size.

I think I also ran across an update of over 25,000.. but can't remember who the updater was.

They vary a lot... from about 1 MB to slightly below 10 MB, which is the upper limit of manual upload. But as I have stated before, the file size is not direct cause of the problem. That particular file is somewhat under 7 MB... but the point is that much smaller AND larger files have BOTH failed AND succeeded. If file size alone was indicative, I would be looking at a failure rate somewhere in the range of 80 to 95%... at the moment... out of last almost 180 files I have uploaded since I was able to resume uploading, 23 have not been gotten to yet and at a quick count, 31 have utterly failed.

As I posted earlier... I looked backward a lot... in the time preceeding the site update, I found TWO failures within a time period exceeding one year. Two failures in over a year versus over 30 failures in less than two weeks tells me something is seriously wrong... but what that something is, remains an utter mystery to me.

Just as quick example, look at this:

Nov 30, 12 15:55:37 0 0 E
Nov 30, 12 15:53:55 1089 4739 Y
Nov 30, 12 15:51:47 0 0 E
Nov 30, 12 15:49:52 2476 9240 Y
Nov 30, 12 15:47:49 0 0 E

If I am looking at this correctly, those submissions come from the same, incremental .lua. The smallest, the one in the middle and the biggest of the five failed. If there was a direct size comparison, files two and four should have automatically failed, but they didn't. Again, if it was file damage, they should have all failed, because they are all derivates of the smallest.

When I uploaded them, the system didn't report any problems. All of them gave me the usual messages of a successful upload. I am quite simply stumped and don't really have a choice in the matter before I hear something new. :( As I stated this morning, I have gone back to "stockpile mode" and wait. If I don't hear anything fairly soon (within the next few days), I have to conclude that I am inadvertently causing problems and need to quit uploading permanently in order to prevent constantly breaking the queue. :(

This is a very sad Epic Censi, signing off... :(

I don't think your files are the cause of the problem 1974ER.

I just looked through my submissions and had no errors
until Dec 28th and have had 2 failures since then. Not as
many failures but not as many files either.

Dec 04, 12 00:30:43 0 0 E
Dec 03, 12 01:03:33 0 0 E
Dec 01, 12 23:44:11 0 0 E

and now 3 failures in a row, doh :roll: lol. I'll never catch ya for updates but
I can try and match you for errors 1974ER, 20 something to go.

I am very sorry to hear that. For me, a loss a file or two (or even 30) is just a major setback. For someone who only makes submissions a few times a week, it's rather devastating. :( Though it doesn't help with queue backlog problem, you should probably try making at least 2 submissions on the days you do. That way, if you don't do full purges, most of data should eventually go through, as for at least as far my errors were concerned, the errors almost exclusively hit every second file at most.

Also, a few days have already passed and there is no further info... :( And the queue has now fallen almost 11 days behind! So, things on a very overall level have actually gotten even worse since my big stockpiles went away... which, from my point of view is pretty incredible, in the bad way. :(

I'm showing:
Dec 15, 12 18:24:47 N
Dec 15, 12 09:01:32 N
Dec 13, 12 22:33:53 N
Dec 13, 12 21:01:30 N
Dec 13, 12 18:55:40 N
Dec 11, 12 22:35:54 N
Dec 11, 12 22:16:12 N
Dec 11, 12 18:52:47 N
Dec 11, 12 18:35:17 N
Dec 04, 12 22:03:25 N
Dec 02, 12 22:37:05 Y
Dec 02, 12 00:03:08 Y
Dec 01, 12 22:59:44 Y

So nothing since Dec. 2nd.

Last processed file was uploaded on 2012-12-04 20:45:12.
Current server time is 2012-12-15 19:46:37

It's getting close to your file on the 4th Korwynd.

I think I started aggressively pruning about Dec 7th so I figure I will have a
few more errors at the least. By the 11th I was purging after each
submission and submitting 5 small files a day so we'll see if that has any
effect.

Dec 05, 12 16:30:10 0 0 E
Dec 05, 12 00:57:23 0 0 E
Dec 04, 12 17:22:03 0 0 E
Dec 04, 12 00:30:43 0 0 E
Dec 03, 12 01:03:33 0 0 E
Dec 01, 12 23:44:11 0 0 E
Nov 30, 12 23:54:55 4472 12847 Y
Nov 29, 12 23:36:39 4347 12797 Y
Nov 28, 12 23:50:31 0 0 E

/palm_to_forehead Doh! Hopefully the smaller files after I started
regularly purging will go better.

I've given Rollie another /poke, hate to bug the guy constantly so I tend to leave it a while after first contact on any issue but this clearly isn't fixing itself.

I've looked over everyone in top contributors this month and last, and it seems only ER and Padanfain have errors, both seem to be giving higher update totals than most (although nothing unusual; I've regularly had similar/larger totals per file in my more active moments with no trouble), but, there are two users who managed 20k+ submissions back in November, so clearly filesize isn't the only factor here.

Gave Rollie that info plus what I could get from the posts recently, hopefully he can come up with something, even if it's just "sorry, the database can't handle such large files any more, here's your new size limit" :?

Unlike Padanfain, my files from Dec 2 to first submission of Dec 8 have been successful. Though I am currently not submitting, I am heavily pruning my stockpile files, striving hard to keep them under 6 MB. Once we reach Dec 11 and the last of my current uploads is processed, we may or may not see some change in the queue speed, which is currently slow, but from my personal view, working better than during end of November, very early December.

This is what I have now
Dec 21, 12 21:47:05 0 0 N
Dec 21, 12 07:02:55 0 0 N
Dec 19, 12 22:26:11 0 0 N
Dec 19, 12 20:58:20 0 0 N
Dec 19, 12 08:02:25 0 0 N
Dec 18, 12 22:22:20 0 0 N
Dec 18, 12 21:44:53 0 0 N
Dec 18, 12 06:57:26 0 0 N
Dec 17, 12 22:54:26 0 0 N
Dec 16, 12 22:12:13 0 0 N
Dec 16, 12 11:05:25 0 0 N
Dec 15, 12 23:56:23 0 0 N
Dec 15, 12 21:22:46 0 0 N
Dec 15, 12 18:24:47 0 0 N
Dec 15, 12 09:01:32 0 0 N
Dec 13, 12 22:33:53 0 0 N
Dec 13, 12 21:01:30 0 0 N
Dec 13, 12 18:55:40 0 0 N
Dec 11, 12 22:35:54 0 0 N
Dec 11, 12 22:16:12 0 0 N
Dec 11, 12 18:52:47 0 0 N
Dec 11, 12 18:35:17 0 0 N
Dec 04, 12 22:03:25 513 1285 Y
Dec 02, 12 22:37:05 58 320 Y

A couple of successful files is an encouraging sign but the queue has slipped
a bit more, 13 days behind.

Dec 08, 12 21:23:14 0 0 N
Dec 08, 12 01:12:33 4673 13404 Y
Dec 07, 12 22:24:49 0 0 E
Dec 07, 12 00:16:12 18985 43036 Y
Dec 06, 12 21:25:37 0 0 E
Dec 05, 12 21:09:20 0 0 E

Some of the times that I've seen the [url=http://www.warcraftrealms.com/dbstats.php][u:764aaeb05b][color=darkblue:764aaeb05b]database stats page[/color:764aaeb05b][/u:764aaeb05b][/url] stuck for 2~4 hours
at a time it isn't choking on your files 1974ER or mine. If no other members
are showing errors, is Anonymous showing a ton of errors Balgair?

The current DB stats page hiccup is about 5 hours old so far. I'm certain, if it
is a file process, that it is not mine or 1974ER's. Not complaining, I'm still
taking census and uploading but I am curious about the details.

BTW so far the most lopsided realm I've taken a look at is Mug'thol.
800~1000 Horde to 50~70 Alliance every time I drop by. Brutal for
defending Stormwind LOL. Sorry. LOL.

I can't check anonymous submissions, only Rollie has that access; I've got the same data you guys do otherwise.

Btw Rollie says he was flooded with work last week, is hoping for better this week but if all else fails he'll have time over the holidays to have a look... hopefully! :)

I am new here , but pretty sure this isnt good ?
No updates processed after 8 dec up till 22 dec, some 120 files still in queu.

Dec 10, 12 05:53:44 0 0 N
Dec 10, 12 04:07:05 0 0 N
Dec 10, 12 03:01:01 0 0 N
Dec 10, 12 02:47:47 0 0 N
Dec 09, 12 17:09:52 0 0 N
Dec 09, 12 16:09:23 0 0 N
Dec 09, 12 14:05:23 0 0 N
Dec 09, 12 13:02:30 0 0 N
Dec 09, 12 12:16:20 0 0 N
Dec 09, 12 11:02:03 0 0 N
Dec 09, 12 09:41:51 0 0 N
Dec 09, 12 08:16:37 0 0 N
Dec 09, 12 06:10:18 0 0 N
Dec 09, 12 05:30:37 0 0 N
Dec 09, 12 05:08:48 0 0 N
Dec 09, 12 03:36:09 0 0 N
Dec 08, 12 19:23:55 0 0 N
Dec 08, 12 14:37:38 0 0 N
Dec 08, 12 03:53:04 18 85 Y
Dec 08, 12 03:13:07 45 200 Y

Welcome Rowilco,

DB queue is still crawling onwards, current status is:

Last processed file was uploaded on 2012-12-09 10:05:24.
Current server time is 2012-12-26 02:25:53

Oke thx

Right now I got:

Dec 25, 12 20:30:35 0 0 N
Dec 25, 12 18:34:12 0 0 N
Dec 21, 12 21:47:05 0 0 N
Dec 21, 12 07:02:55 0 0 N
Dec 19, 12 22:26:11 0 0 N
Dec 19, 12 20:58:20 0 0 N
Dec 19, 12 08:02:25 0 0 N
Dec 18, 12 22:22:20 0 0 N
Dec 18, 12 21:44:53 0 0 N
Dec 18, 12 06:57:26 0 0 N
Dec 17, 12 22:54:26 0 0 N
Dec 16, 12 22:12:13 0 0 N
Dec 16, 12 11:05:25 0 0 N
Dec 15, 12 23:56:23 0 0 N
Dec 15, 12 21:22:46 0 0 N
Dec 15, 12 18:24:47 0 0 N
Dec 15, 12 09:01:32 0 0 N
Dec 13, 12 22:33:53 0 0 N
Dec 13, 12 21:01:30 0 0 N
Dec 13, 12 18:55:40 0 0 N
Dec 11, 12 22:35:54 0 0 N
Dec 11, 12 22:16:12 0 0 N
Dec 11, 12 18:52:47 0 0 N
Dec 11, 12 18:35:17 0 0 N
Dec 04, 12 22:03:25 513 1285 Y

will they ever catch up?

On the 21st it was 13 days behind and now a week later it is 18 days behind so not looking so good.

I sent an email a few days ago to Rollie to see if I could get some clarification of the issue at hand, as of this time I haven't received a reply.

The site outage we saw from around 2am on the 24th thru 4pm on the 25th was caused by the servers being offline.. maybe power loss, maybe maintenance.

When the mySQL database is shutdown like that, proper start up requires a check of the database table for corruption issues. Depending on number of tables etc the background check can take awhile and slow down active transaction processing.

Another possibility (I hope, I hope) is that Rollie has added newer faster and larger drives and is doing an online mirror of the data from the one active drive set to the new one, this will also slow down processing for now with improved performance later after he finishes any upgrades.

We may never know what has been happening this week.. unless Rollie decides to respond the our outstanding queries.

Thanks bringoutyourdead, I hope he can find the time to keep the site cooking data.

Wondered about the site being down. I have my fingers crossed it was a mirror/backup preceding a
hardware swap. That would be great.

As of some time yesterday... the almost no processing speed ended and we seem to be processing at a much faster rate.. but I have not check to see if it is faster then real time clock .. i.e. are we finally catching up with the queue?

The speed seems to be fluctuating a lot... which is what it used to be like, long ago... So I am hoping things are improving. I'll copy the current situation here and take another look either late tomorrow or some time Tuesday.

Last processed file was uploaded on 2012-12-10 21:42:47.
Current server time is 2012-12-30 15:11:38

Also, due to personal reasons and since there have been no further errors, I am resuming uploads, though I'll try to keep a minimum 10 minutes interval while reducing my stockpile.

It does feel like it is picking up a bit though I am still seeing errors :(

Dec 10, 12 22:55:17 763 2661 Y
Dec 10, 12 21:45:51 3694 11938 Y
Dec 10, 12 20:09:45 0 0 E
Dec 10, 12 01:39:38 0 0 E
Dec 10, 12 00:47:06 0 0 E
Dec 09, 12 23:15:34 2124 7455 Y
Dec 09, 12 22:00:21 0 0 E
Dec 09, 12 20:12:31 795 3229 Y
Dec 09, 12 01:52:12 164 426 Y
Dec 09, 12 01:05:49 4365 10398 Y

The 9th was when I started pruning over a day old. File sizes were about
4~6 MB if I remember correctly and by the 11th or 12th I started purging
after each submission which rarely hit 1 MB.

Hopefully experimenting with purging smooths out that E/Y column.

It will be interesting to see what happens with my uploads.. after every upload I do a
/census serverprune on a low population server.

I'm doing about 20 servers both sides.. and I'm seeing locally around 16 - 20k on every serverprune.

Hopefully you won't see a replay of last Junes submissions. /FingersCrossed

Each of my submissions is both sides of 5 realms, upload, purge then rinse
and repeat 6 times. The 30 realms are a variety of low, medium and high
population realms and are PVE, PVP and RP and run between 5~9k per
group of 5.

The consistent lopsided population of play types is interesting.

A curiosity... Will the single toon submission from a cross realm sighting give
an update toward the 100 updates per 30 days needed to make that realm
census 'reliable data?'

Padanfain, I am sorry that you are still seeing errors. I have luckily had none since December 1st and my latest 20 look like this:

Dec 30, 12 23:38:25 0 0 N
Dec 30, 12 21:55:33 0 0 N
Dec 30, 12 17:46:15 0 0 N
Dec 30, 12 17:27:48 0 0 N
Dec 30, 12 17:11:30 0 0 N
Dec 30, 12 16:57:12 0 0 N
Dec 30, 12 16:44:30 0 0 N
Dec 30, 12 16:31:41 0 0 N
Dec 30, 12 16:17:58 0 0 N
Dec 30, 12 15:44:19 0 0 N
Dec 30, 12 15:21:45 0 0 N
Dec 11, 12 17:50:43 0 0 N
Dec 11, 12 10:19:01 0 0 N
Dec 10, 12 19:43:08 824 4078 Y
Dec 10, 12 14:34:48 1382 5632 Y
Dec 10, 12 08:33:21 860 3483 Y
Dec 10, 12 05:10:33 293 1293 Y
Dec 09, 12 21:53:30 89 436 Y
Dec 09, 12 15:34:55 354 884 Y
Dec 09, 12 14:00:40 1419 5250 Y

Which means the pre-pause files are almost done and I have a rather long wait ahead of me to see, how the next stockpile will do...

So close to that 20 million update mark, I hope you don't have any more errors.

After Balgair mentioned that you and I were the only members with errors I
started taking census on a second computer, one with a WoW install from
around BC release and the other install was just before the Wrath release. It
may be a couple weeks to see if a trend emerges from that.

Besides I need about 20 more errors to catch up with you on that stat ;) lol.

Close is somewhat relative... with the data lost to the errors and so on... If there are absolutely no further errors, end of February / early March looks reasonable for breaking the 20 million updates barrier as I am about 680k short of that goal and have been getting somewhere around 250k per month (on average, November balancing out October during which I only gathered data due to the site problems) lately.

About that curiosity question you posted... as far as I know, no. Only fully completed censuses count and only for the faction you are online with. CRZ data only updates individual characters seen.

Real close compared to where you were at that first update or that first
million. ;) Glass is half full kind philosophy here.

Was just peeking through a recent single realm census of both factions and I
believe you are right. Under one heading it lists the date and time census
was taken on that realm for both sides and in another heading was a realm
with a single toon. So it looks like only the realms in the one heading gets
an update credit.

Hehehe... with that philosophy the "glass" is about 97% full (if my estimate on stockpile results holds and there are no processing errors). It's just that each remaining 1% takes about 3 weeks. :D And when I started... I don't think anyone expected to see 20 million updates on the top of the all time board. Even now, over four years later, 10 million has not been broken by anyone else, though I expect Balgair will eventually get there. 5 million has been achieved by four people, 3 million by 8 (and Bringoutyourdead is very close to making that 9). In fact, due to the backlog, he probably already has, because he is currently only 28 005 short and has 26 unprocessed entries. :D

20 million is just insane, back in the day we all thought 1 million was a crazy goal lol :P I can see myself reaching 10 million given another year or two, but 20... nope. It's taken me over seven years to get to still some way short of 9 million so, 20? Will WoW/WCR even still exist in the 10+ years that's gonna take me? :P

I remember, desperately [url=http://www.warcraftrealms.com/forum/viewtopic.php?t=4617]trying to reach 1M[/url] before Cata pre-patch was released..

Oh, those were the days ;)

[quote="Padanfain"]Hopefully you won't see a replay of last Junes submissions. /FingersCrossed/quote]

Those were test submissions .. that were expected to fail

[quote:3bdfaeba06="Padanfain"]
A curiosity... Will the single toon submission from a cross realm sighting give
an update toward the 100 updates per 30 days needed to make that realm
census 'reliable data?'[/quote:3bdfaeba06]
that is a Rollie question but from what I've been told that might just be the case.
the site doesn't have any way to tell if you did full census runs or just started and killed right away and only picked up a few. as far as the cross realms are treated in the client file which is uploaded they are just standard realm/faction updates.

One of the questions I had for Rollie was the TimesPlus part of the upload, I also assumed that it somehow had a data checking feature which would be mucked up by the cross realms addition.
Rollie's response was that he only used that data for the activity graphs shown here on the site. But then he did qualified his response with "at least that is what I think I remember"

Bringoutyourdead...

Interestingly, site data conflicts with your replies...

I see people from from EU-Runetotem and EU-Thunderhorn Horde every day I play on EU-Stormrage Horde... and both of those realms list a big fat zero on faction updates for Horde on server submissions page.

And on the other end... EU-Scarshield Legion has 394 Horde updates despite the fact that there are rarely more than 60 characters online. If there was no counter of any sort... that value should be around 20 to 30 from 2 or 3 submission from me / currently available day (due to massive backlog we have).

Lastly, as this might well end as the first post of the year here... Happy new year to all! :)

well then that appears to indicate that Rollie remembered incorrectly and that might be a part (though I expect minor or nearly all submissions would end with E status) of the slow processing.

I haven't any data yet myself to work with.. since my uploads haven't been processed. (I kept renamed copies of the first 10 or so uploads.

And at the rate the uploads are processing.. my Dec15th upload won't get processed until mid January..sigh

Last processed file was uploaded on 2012-12-11 16:36:49.
Current server time is 2013-01-02 03:08:41

Actually... judging by that... we'll reach December 15th some time late February, early March... UGH. :( Maybe, as a stopgap of sorts, data expiry should be increased to 60 days? We are already close to a situation where data is "outdated" BEFORE it finishes processing. :(

I am not sure what to make of this:

Last processed file was uploaded on 2012-12-12 12:39:57.
Current server time is 2013-01-03 10:31:37

About 5 hours ago, the queue had not yet reached:

Dec 11, 12 17:50:43 0 0 E

Which is now failed. In the preceeding roughly 24 hours progress was minimal and now it has gone forward almost a whole day in less than 6.

I really wish I could find out what is causing the process failures.

Yours have stopped (we hope)
Of the other visible uploaders Padanfain appears to be the only one that is still getting hit with the problem.

Epic any ideas of what may have changed on your part to cause/allow processing to complete?

did you do a full delete of your CensusPlus and reload as a new install from the curse.com copy? grasping at straws here.

Actually... I am still using the "old" version (pre-TOC change)... and to my knowledge, there has not been any change on my part at all... many files are variants of previous ones, just pruned. Some of them work, others don't. As could be seen, there were many successes, but as noted:

Dec 11, 12 17:50:43 0 0 E

failed, reason yet again unknown. Now the queue has gone to the area where I have no submissions, so I can't offer direct input until queue manages to reach:

Dec 30, 12 15:21:45 0 0 N

which at current speed will take a couple of weeks, at least.

I am sorry for not being able to help you and Rollie... but to be honest, such random fail / succeed pattern is beyond my understanding... as is the fact that only I and Padanfain have reported failures. We submit a lot... but I have gone above 700 000 updates in a month in previous years and the system didn't break. Now many of the files are much smaller and they still fail. *puzzled*

Dec 12, 12 17:14:13 921 2278 Y
Dec 12, 12 16:15:00 1797 3966 Y
Dec 12, 12 14:53:30 549 1753 Y
Dec 12, 12 14:07:16 469 1895 Y
Dec 12, 12 13:11:17 6231 16796 Y <--- After this submission was when I started purging after each submission.
Dec 11, 12 21:44:48 0 0 E
Dec 11, 12 19:56:45 0 0 E
Dec 11, 12 17:51:06 0 0 E
Dec 11, 12 16:39:54 0 0 E
Dec 11, 12 14:54:08 0 0 E
Dec 11, 12 01:09:55 2208 5361 Y
Dec 11, 12 00:19:07 0 0 E
Dec 10, 12 22:55:17 763 2661 Y

Was looking bad with only 38% of Decembers processed files being successful. Up to 44%.

What I'm wondering is if there might be some incorrect left over data in the file from pre CP 6.0 version

But what that could be I haven't a clue..
the only difference other then the obvious new race/class info is
the version number and the number of fields in the TimesPlus data lines

If your submissions had any data from Pre-CP 6.0 the older TimesPlus data would have 2 fewer fields in the data line. This is something that was brought to Rollie's attention for his class activities reporting.

doing a serverprune doesn't if I remember clear the TimesPlus data.. so you should look to see if you have anything from Pre MoP days in your file.. if so you should erase the file after doing your upload and have CP recreate a clean file.

If you count the fields, you should find one for each class in the game and a trailing three for the original battlegrounds.

I wish I would have kept copies of files from around the 1st of Dec but didn't
start that until the 12th. I have all submissions from then on saved away if I
have more errors in case someone needs to look at them. I never did a
server prune but around the 5th I pruned to 7 days, then to 4 days and by
the 11th I had pruned to 1 day. I am certain the 5 errors in a row on the
11th were after a /CensusPlus prune 1.

Dec 15, 12 23:56:23 0 0 N
Dec 15, 12 21:22:46 0 0 N
Dec 15, 12 18:24:47 0 0 N
Dec 15, 12 09:01:32 0 0 N
Dec 13, 12 22:33:53 0 0 N
Dec 13, 12 21:01:30 0 0 N
Dec 13, 12 18:55:40 0 0 N
Dec 11, 12 22:35:54 0 0 E
Dec 11, 12 22:16:12 0 0 E
Dec 11, 12 18:52:47 0 0 E
Dec 11, 12 18:35:17 0 0 E

The system does not like me but yeah, I hadn't done any pruning so I'm not expecting anything.

I don't think it's data carryover:

Dec 11, 12 17:50:43 0 0 E
Dec 11, 12 10:19:01 1273 3776 Y

Those two are essentially the same file, the failed one is about 0,6 MB bigger and they are both under 5 MB. And I didn't spot any entries before December 10th... and I did spot both monks and pandaren, so data as such should be ok. Why one works and the other doesn't... simply isn't apparent. Size is not decisive either, because some of the files from, say, Dec 8th and 9th were bigger.

The only semi-obvious thing is that both Padanfain and Korwynd also had failures on the 11th. But like me, Padanfain's list also has a successful submission on 11th.

I am starting to lean in the direction of the problem being in the processing end. And that, unfortunately, is entirely Rollie's area. The only thing the failed fails seem to have in common is that they come from similar time points.

Ack... failed files, not failed fails. :D That's what one gets from sleeping poorly and paying attention. :(

Honestly, it wasn't that I was feeling left out, but now I have a failed file as well.

Dec 13, 12 02:25:19 1457 3854 Y
Dec 12, 12 21:55:11 0 0 E
Dec 12, 12 03:55:19 2587 6768 Y

That's the first fail that I've had.

On the up-side, I started keeping my files a while back so I have the file and already know which one it was. Nothing jumped up and screamed "this is the problem" when I looked at it, but if anyone else wants to take a look at it you're welcome to it.

As for old data, I used to always delete my files after uploading them, now I drag them over to a folder named with the date, but in either case the addon has to create a completely new data file next time I log in, so there is no chance of it being old data still hanging around in my case.

Thanks for the info Fett.
That pretty much confirms that the issue isn't in left over data..
so it is something that Rollie isn't cleanly processing.

Fett I'm sending you a PM with my email, I'll look the file over but I don't really expect to see anything that stands out either.

Dec 13, 12 00:23:25 0 0 E
Dec 12, 12 23:27:26 0 0 E
Dec 12, 12 21:32:09 0 0 E
Dec 12, 12 20:14:03 0 0 E
Dec 12, 12 18:56:45 553 2213 Y
Dec 12, 12 17:14:13 921 2278 Y
Dec 12, 12 16:15:00 1797 3966 Y
Dec 12, 12 14:53:30 549 1753 Y

Yep, I'm convinced it's a processing problem. Those were all taken after
purging. Of the errors the smallest file was about 300 KB and the largest
about 900 KB. 59% failure rate for the month so far is convincing me to take
a break.

At least, by the end of Decembers files I bet I will have passed 1974ER's
error total if not any other. lmao

I also had a file fail, uploaded on the 12th...
Dec 12, 12 21:59:15 0 0 E

So, nub-Sanna is level nub... is it better to not be uploading files? I still run censuses (censii?) but I haven't uploaded since the 21st of Dec, and today I pruned-7... am I piling straw on a poor camel's back by giving the server more files to deal with? Or is this a thing where the server will get to some point and then speed up and catch up and things will be fine again? Or do we not know? X D Well, the main question is, should I hold off on uploading files till things are fixed?

Add me to the failure list and I was doing such a good job.

It's mighty interesting how many fail which were uploaded at a similar time, from different users.

1974ER:
Dec 11, 12 17:50:43 0 0 E

Padanfain:
Dec 11, 12 17:51:06 0 0 E

[url=http://www.warcraftrealms.com/usersheet.php?id=1039544&c=u&p=9]Rowilco[/url] (found him on the top contributors list):
Dec 11, 12 18:07:18 0 0 E

Korwynd:
Dec 11, 12 18:52:47 0 0 E
Dec 11, 12 18:35:17 0 0 E

I'm so far failing to find any that actually succeeded in that timeframe (granted I've no idea who's active outside those who made the top 10 for December or have posted here so it's a small sample). I'm wondering if this may actually be completely unrelated to the contents of the files?

Although in that case, I'm baffled how it's hitting some more than others: it may of course be that Padanfain and ER are more prolific and so have a higher chance of being in the wrong place at the wrong time... but Rowilco I linked to above seems to upload many times every day and only has one error out of dozens of submissions; I think he submits more often than anyone.

Edit: Hmm, missed the new page and extra posts, ok:

SannaSK:
Dec 12, 12 21:59:15 0 0 E

Linxa:
Dec 12, 12 21:15:58 0 0 E

Padanfain:
Dec 13, 12 00:23:25 0 0 E
Dec 12, 12 23:27:26 0 0 E
Dec 12, 12 21:32:09 0 0 E
Dec 12, 12 20:14:03 0 0 E

Fett:
Dec 12, 12 21:55:11 0 0 E

Again, no successful uploads in that timeframe which I can find from known active users (last beforehand was Rowilco at 18:11:03, next one was Rowilco again at 02:34:49 on the 13th). Time to mail Rollie again, maybe the patterns might help (else he can look and tell me there's one I'm not seeing which worked fine :P)

Oh, damn.. I've just had my 2 first failed files..
And it seems one of them landed right on top of the times Belgair stated:

Dec 12, 12 15:00:12 28 174 Y
Dec 12, 12 22:51:51 0 0 E
Dec 13, 12 14:47:52 24 215 Y
Dec 13, 12 22:47:50 0 0 E

Okay it seems everything from 12/11 failed

Dec 13, 12 18:55:40 540 2333 Y
Dec 11, 12 22:35:54 0 0 E
Dec 11, 12 22:16:12 0 0 E
Dec 11, 12 18:52:47 0 0 E
Dec 11, 12 18:35:17 0 0 E
Dec 04, 12 22:03:25 513 1285 Y

Rollie says he's working on a new processing system which may improve things, and is hoping to have it live by the end of the weekend. He reckons it really needs new hardware, but it'll just cost too much; the site is barely breaking even with costs as it is, so the extra expense would be hard to manage, he's hoping his changes will help but isn't too sure as yet. He also says he'll rerun all failed submissions once it's sorted, so hopefully not too much will be lost.

Just to chime in: I haven't had any submission failures, but my submissions are being processed v-e-r-y s-l-o-w-l-y, like everyone else's. The last one of mine that got processed was from December 14th. My files aren't very big, since I'm currently mainly doing just my home realm; not sure if that makes a difference when it comes to errors.

Thank you Jenny. Well, the queue is shared so it's the same for everyone. Interestingly enough... as of right now, it seems to have picked up some speed and the backlog has been a fairly steady about 22 days for over a day... Right now it seems that it MAY have started to actually catch up... Not sure about that. I'll copy paste current state here so I / we can keep an eye on it:

Last processed file was uploaded on 2012-12-15 04:03:49.
Current server time is 2013-01-06 02:24:20

Balgair, thank you for keeping Rollie up-to-date. I wish I had extra money to lend a helping hand. Though I like the idea of reruns, but considering the chronological function of the site... Unless he forces a complete rollback to a state near November 20th or so, most of the lost data will stay that way. Rollback to December 11th around 15 o'clock might be more feasible, but I am not sure how high accuracy is even possible. It's not likely that massively huge database is backed up every hour... or even every day. Almost 100 million characters is simply huge. And it's that much despite restrictions, purges and the fact that we don't catch everyone to begin with.

Last processed file was uploaded on 2012-12-15 04:03:49.
Current server time is 2013-01-06 05:17:50

I think we got a candidate for the examination of a critical error with the file right after the one uploaded on 2012-12-15 04:03:49, which as of now is "in process" for almost 3 hours.

Whee, I've joined the error club, wondered how long it'd take :P

Dec 14, 12 22:01:52 0 0 E

Others from the evening of the 14th:

Padanfain:
Dec 14, 12 23:18:46 0 0 E
Dec 14, 12 22:18:24 0 0 E
Dec 14, 12 21:02:57 0 0 E

FuxieDK:
Dec 14, 12 18:16:31 0 0 E

Fett:
Dec 14, 12 17:33:02 0 0 E

Rowilco:
Dec 14, 12 17:47:06 0 0 E
Dec 14, 12 17:24:22 0 0 E

Linxa:
Dec 15, 12 00:01:25 0 0 E

Definitely seeing these coming in waves of failures now.

Let's see now

Dec 17, 12 22:54:26 0 0 N
Dec 16, 12 22:12:13 0 0 N
Dec 16, 12 11:05:25 0 0 N
Dec 15, 12 23:56:23 0 0 N
Dec 15, 12 21:22:46 0 0 N
Dec 15, 12 18:24:47 0 0 N
Dec 15, 12 09:01:32 0 0 N
Dec 13, 12 22:33:53 0 0 E
Dec 13, 12 21:01:30 0 0 E
Dec 13, 12 18:55:40 540 2333 Y
Dec 11, 12 22:35:54 0 0 E
Dec 11, 12 22:16:12 0 0 E
Dec 11, 12 18:52:47 0 0 E
Dec 11, 12 18:35:17 0 0 E

Everything from Dec 11 failed, 2 fails out of 3 on Dec 15 and it just drags along.

:(

Last processed file was uploaded on 2012-12-15 08:32:05.
Current server time is 2013-01-06 13:46:22

-> In about 11 hours of server running time, we've had 4:20ish hours of progress with data processing.

If a person had a couple extra dollars to spare, is there a method of donating those couple dollars, and would said donation help? :|

Took a minute to find lol, it's not obvious - http://www.warcraftrealms.com/donate.php - that's a Paypal donations link.

I can't tell you how much it'll help, if at all though, one-off donations will likely help a bit but I think it's sustained income the site needs to move onto bigger and better hardware, and that's tougher, since that needs more regular traffic seeing the ads, and the problems lately are likely to mean less traffic, not more. (Note that this paragraph is entirely my own speculation, and not Rollie's take, he may well find it more useful than I'm guessing, but I just don't know :))

It might help too if we post on our realm forums about this website, and help advertise lightly to increase traffic. :wink:

I am rather failing at the 'find information on the website' game these days, aren't I >_>.

Does a user clicking ads generate more revenue for him than just a user seeing ads?

YES

Last processed file was uploaded on 2012-12-16 10:23:02.
Current server time is 2013-01-08 01:41:42

Almost 48 hours have passed since my last comment, queue has moved forward about 30 hours, 20 minutes. So it's still falling behind, despite there being no files from me at the moment. *calculates* At current speed, if nothing can be done, site will "permanently" break on roughly January 28th at which point the processed data is directly discarded as outdated.

I hope whatever improvements Rollie is working on get done soonish...

[quote:a77befa131="1974ER"]January 28th at which point the processed data is directly discarded as outdated.[/quote:a77befa131]You paint a grimmer picture than it is..

The data *I* use, is the character history, and that NEVER outdate or discarded.

Yea, it's sub-optimal, that we're almost a month behind on processing, but since *MY* needs are met over time, it's not realy a problem for me.

[quote:d586bf0bce="FuxieDK"]Oh, damn.. I've just had my 2 first failed files..
And it seems one of them landed right on top of the times Belgair stated:

Dec 12, 12 15:00:12 28 174 Y
Dec 12, 12 22:51:51 0 0 E
Dec 13, 12 14:47:52 24 215 Y
Dec 13, 12 22:47:50 0 0 E[/quote:d586bf0bce]MEH.. Getting worse for me...
Dec 16, 12 22:52:25 0 0 E
Dec 16, 12 15:32:07 0 0 E
Dec 15, 12 19:55:12 0 0 E
Dec 15, 12 08:46:42 126 632 Y
Dec 14, 12 18:16:31 0 0 E
Dec 13, 12 22:47:50 0 0 E
Dec 13, 12 14:47:52 24 215 Y
Dec 12, 12 22:51:51 0 0 E
Dec 12, 12 15:00:12 28 174 Y

6 failed so far..

Getting worse for me too, hoping Rollie's changes help and he can get them in fairly quickly...

Dec 16, 12 20:18:44 0 0 E
Dec 15, 12 17:01:40 0 0 E
Dec 15, 12 09:46:31 434 1850 Y
Dec 14, 12 22:01:52 0 0 E
Dec 14, 12 17:04:57 343 1404 Y

Me three. That time of day submission theory is looking pretty good Balgair.

Dec 17, 12 21:19:41 0 0 N
Dec 17, 12 01:05:56 0 0 E
Dec 17, 12 00:17:40 0 0 E
Dec 16, 12 22:40:19 0 0 E
Dec 16, 12 21:30:32 0 0 E
Dec 16, 12 20:04:04 0 0 E
Dec 16, 12 01:53:49 644 1838 Y
Dec 16, 12 01:04:36 0 0 E
Dec 15, 12 23:27:28 0 0 E
Dec 15, 12 22:13:48 0 0 E
Dec 15, 12 20:50:40 0 0 E
Dec 15, 12 01:40:50 603 1760 Y
Dec 15, 12 00:55:27 0 0 E
Dec 14, 12 23:18:46 0 0 E
Dec 14, 12 22:18:24 0 0 E
Dec 14, 12 21:02:57 0 0 E
Dec 14, 12 01:26:29 537 1583 Y
Dec 14, 12 00:42:27 0 0 E
Dec 13, 12 23:18:38 0 0 E
Dec 13, 12 22:20:11 0 0 E
Dec 13, 12 20:57:10 0 0 E
Dec 13, 12 00:23:25 0 0 E
Dec 12, 12 23:27:26 0 0 E
Dec 12, 12 21:32:09 0 0 E
Dec 12, 12 20:14:03 0 0 E
Dec 12, 12 18:56:45 553 2213 Y
Dec 12, 12 17:14:13 921 2278 Y

Fuxie are your E submissions also of the same approximate size of characters updated?
and are you pruning your files?
I'm wondering if your files have a lot of older data vs. new update data?

If your files are purged/deleted and you have only or mostly new update data.. all I can say is "hmmn.. I haven't a clue what could be the processing issue."
And that bugs me.. I tend to spend much of my professional time, diagnosing IT problems and getting them fixed.

The one thing I do know is that so far as of the 12/15 data.. I'm batting .000% with 6 failures. Each submission I'm doing is with /census serverprune processed before I start doing my census runs on 40 to 50 factions each submission.

The fact that I'm not showing nearly 100,000 update isn't what bothers me.. as much as the fact that with all these E submissions from all of us, the most wanted and server submission pages here on the site are of no value.. no one can know which realm/factions need census runs more then others. sigh

oh well it is better then my other project (I run Wow in background, switching to it when I here the Census finished bell), that project has a program that randomly crashes on memory clean up when the program tells windows(win7 & win8) it no longer needs blocks of allocated memory... the problem seems to be a complied dll file which no one seems to have the source to any more.. it was lost during a hardware crash.. who needs backups .. sigh
So I get to try to figure out everything that dll was suppose to do and recreate it again from scratch.
Sort of like putting a picture puzzle together.. where there isn't a picture... and you haven't a clue if you have all the pieces.

Aw, me and my big mouth--now I have errors too :(

Dec 18, 12 01:55:26 0 0 N
Dec 16, 12 20:06:15 0 0 E
Dec 16, 12 01:52:22 111 482 Y
Dec 15, 12 21:02:31 0 0 E
Dec 14, 12 02:29:36 35 207 Y

I mostly haven't been pruning my files, since I know they're not that big (though I have pruned a few times lately).

@ bring....

Honestly, I can't remember, what I did, prune-wise, 3 weeks ago, but [i:fdd1579d35]most likely[/i:fdd1579d35], I used standard 30 days prune.
My files are NEVER over 4MB in size.. 3,5ish is the biggest files I have uploaded in MoP.

After the first failed file, a few days ago, I have started a prune-7 routine..

[quote:8f691b37af="FuxieDK"]After the first failed file, a few days ago, I have started a prune-7 routine..[/quote:8f691b37af]
To be honest: That won't help you. I'm doing the 7 day prune for years now and two of my files error'd.

Yep, I've been doing prune 5 the last few weeks with all the problems (only even played on 2 servers so won't be that high anyway), hasn't helped me much.

I suppose on the plus side, the backlog is progressing a lot faster than it was... I just think it's doing that by not processing many of the files :P

[quote:3f25555ba6="Djambo"][quote:3f25555ba6="FuxieDK"]After the first failed file, a few days ago, I have started a prune-7 routine..[/quote:3f25555ba6]
To be honest: That won't help you. I'm doing the 7 day prune for years now and two of my files error'd.[/quote:3f25555ba6]It's morelikely to be number of entries, rather than days.
However, you might do more censusing (and thus more entries) than me, and thus, it might work for me, but not you..

Strictly guessing, ofc... :shock:

Since I'm not interested in keeping a local history on the factions I census what I'm doing is - /census serverprune - which will prune ALL data except for the current realm you are on.

One of the side effects of that is that every time I do a census run all characters are new characters.. so what I see and the end of the run doesn't come close to what warcraftrealms.com sees.

So far from what has cleanly processed.. I see about 22000 characters... and WR sees 7000 - 9000 updates and about 1/2 of that as new.

It's starting to look like that there will not be a rollback to reprocess failed files and lost data will remain that way forever. No obvious news from Rollie and DB queue remains roughly 22 days behind schedule.

If there's not already a song about how lost data is gone forever, there ought to be.


I had two fails in a row from late on December 19. If our observed pattern of fails coming in bunches holds true, the rest of you should be seeing fails around the same time as well. How about it? Is the pattern holding true?

Dec 19, 12 18:41:19 0 0 E

Yep :(

I think the time range is expanding.

Dec 19, 12 23:57:30 0 0 E
Dec 19, 12 22:09:15 0 0 E
Dec 19, 12 21:02:25 0 0 E
Dec 19, 12 19:49:10 0 0 E
Dec 19, 12 16:41:15 0 0 E
Dec 19, 12 15:44:57 0 0 E
Dec 19, 12 14:18:13 0 0 E
Dec 19, 12 13:37:21 0 0 E
Dec 19, 12 12:51:21 0 0 E
Dec 19, 12 00:21:13 0 0 E

BTW five of the failures on the 19th were taken on a second computer with a
fresh install of WoW, the only addon installed and updated is CensusPlus via
curse client and a purge after each upload on both computers. I'm getting
the feeling the database just hates me. :cry: :lol:

[quote:748fe74891="Padanfain"]BTW five of the failures on the 19th were taken on a second computer with a fresh install of WoW, the only addon installed and updated is CensusPlus via curse client and a purge after each upload on both computers. I'm getting the feeling the database just hates me. :cry: :lol:[/quote:748fe74891]

Judging by the number of errors from pretty much everyone, I'm beginning to think it's not a problem with the uploads, but with the database itself.

I'm not into coding, but would it be possible that the database can't cope with the number of entries? After all, "Total character count: 98,373,879" and "Total guild count 6,580,801" does mean a really big lot of datasets...

Dec 25, 12 23:57:52 1787 4425 Y
Dec 25, 12 22:23:20 3519 7105 Y
Dec 25, 12 20:10:30 1281 3168 Y
Dec 25, 12 19:08:30 932 3018 Y
Dec 25, 12 18:13:16 607 2147 Y
Dec 25, 12 17:57:00 1343 4110 Y
Dec 25, 12 17:40:16 1154 2792 Y
Dec 25, 12 17:24:49 3302 6466 Y
Dec 25, 12 17:09:18 1331 3091 Y
Dec 25, 12 16:53:43 1007 3085 Y
Dec 25, 12 16:37:34 1467 3884 Y
Dec 25, 12 16:20:39 718 2353 Y
Dec 24, 12 02:02:46 1910 4276 Y
Dec 24, 12 01:18:56 1768 4446 Y

Looking promising along with the queue gaining 2 days toward being
caught up. Did Rollie make some changes or ??? Hope this trend
continues.

Not that I know anything for sure, and also not to be a total naysayer, but those dates were when the db was already known by the userbase to be having issues, plus it's the Christmas timeframe, so my first thought was that fewer files were getting submitted on those days, thus the DB might still be processing at its same slow speed, and only has less to chew through.

Although I'd much prefer to subscribe to the theory of 'the problems are fixed' X D.

This thread has inspired me to do a complete purge of data after every upload, because even when things are fine serverside, why make it chew through the same data twice?

Good points SannaSK, I prefer to subscribe to the theory too.

I purged after every submission for most of December and in hindsight I
wish I hadn't. Each file that errors is just gone, poof, recycled electrons
before they were counted. Pruning to, say, two days or maybe three would
have contributed something of those soon to be recycled electrons when
that one file out of five or ten was a success.

Jan 02, 13 20:10:19 0 0 N
Dec 27, 12 06:32:46 803 2534 Y
Dec 26, 12 13:34:41 4603 8626 Y
Dec 23, 12 20:33:10 0 0 E
Dec 23, 12 19:50:43 0 0 E
Dec 23, 12 13:24:47 0 0 E
Dec 23, 12 05:33:49 1676 3742 Y
Dec 22, 12 19:45:54 0 0 E

... we'll see if the bugs are gone. I'm afraid I don't subscribe to the "purge of data after every upload" method. I'd suggest a /prune to the date that the person uploading had the last confirmed successful upload...

Dec 31, 12 10:30:49 218 1368 Y
Dec 30, 12 17:16:30 850 3416 Y
Dec 29, 12 08:23:58 524 2225 Y
Dec 28, 12 16:58:21 0 0 E
Dec 26, 12 16:53:41 3172 9275 Y
Dec 21, 12 08:08:55 4696 8451 Y
Dec 20, 12 16:50:30 1193 4743 Y

Certainly improving for me, not perfect but definitely doing better than it was. I mailed Rollie a few days back asking if he'd done anything to cause the recent improvements, but no response as yet so I guess he's not available right now, it may just be lesser load with fewer uploads over the holiday period, we'll see.

So far, all files that I have submitted after the break I took have processed correctly. Queue has been fluctuating a lot and still has huge backlog, but I think things are finally getting better. Anyone out there who is still getting lots of errors? Or are others seeing things the way I currently am?

A big hooray from Austria, I've entered January. :) My last error was on th 28th Dec. But I think I will still get errors 'til the 17th of January (that's were I started purging my database).
I agree with the theory that during vacation the upload rate was not so high, therefore the queue speed has increased a little bit.

I'm error-free for several days now, it may not stay that way but it's nice while it lasts, thats for sure! Also seems to be catching up a little, or at the very least not falling further behind, hope that lasts.

One error at 28/12, otherwise last error at 23/12...

Am I miscalculating or are we picking up speed?

Last processed file was uploaded on 2013-01-03 05:58:33.
Current server time is 2013-01-24 13:33:42

(I could *swear* we were at more than 22 days behind just yesterday.)

For me it's still the 2nd or 3rd file on the second page as most of the last days. So most likely not.

Last processed file was uploaded on 2013-01-03 12:46:53.
Current server time is 2013-01-25 12:07:36

... and we're back to a speed of "within 22:30 hours, only about 6:45 hours advance for the date of the last uploaded file".

....and the errors are back..

Jan 07, 13 15:04:21 190 863 Y
Jan 07, 13 09:28:06 3736 7863 Y
Jan 06, 13 18:36:09 0 0 E
Jan 06, 13 16:35:45 0 0 E
Jan 05, 13 20:37:47 0 0 E
Jan 05, 13 13:13:43 0 0 E
Jan 04, 13 13:59:29 349 1091 Y

Meh.. Only had one error at 28/12, otherwise last eror was 23/12 :(

Last processed file was uploaded on 2013-01-07 18:54:29.
Current server time is 2013-01-29 16:08:09

so if I'm mathing right, in 4 irl days, the server has processed 4 days of data. I still feel bad uploading new files... Nobody likes having their inbox be slammed when it's already full >_>.

Jan 07, 13 19:17:20 464 2227 Y
Jan 07, 13 16:27:59 1125 4670 Y
Jan 07, 13 05:53:16 4884 18086 Y
Jan 07, 13 00:58:51 0 0 E
Jan 06, 13 22:02:00 0 0 E
Jan 06, 13 16:16:18 0 0 E
Jan 05, 13 19:36:05 0 0 E
Jan 05, 13 17:24:59 0 0 E
Jan 05, 13 11:47:06 0 0 E
Jan 04, 13 18:46:38 1077 4795 Y
Jan 04, 13 06:45:40 474 2386 Y
Jan 03, 13 18:03:52 1017 4768 Y
Jan 03, 13 10:02:01 615 2822 Y
Jan 02, 13 17:39:17 938 4556 Y
Jan 02, 13 10:03:10 693 3343 Y
Jan 01, 13 18:16:21 797 4127 Y
Jan 01, 13 11:05:27 1087 5281 Y

Seems queue decided to jump forward by fast failing piles of files... :( And not only mine... I would not be overtly surprised if just about everything from 5th and 6th failed. :(

Jan 08, 13 08:59:30 0 0 N
Jan 06, 13 18:33:07 0 0 E
Jan 04, 13 22:02:22 626 3092 Y

Yep, my only entry from those days failed. Sigh :( No response from Rollie yet, I'll give it a bit longer then try another subtle poke... problem is, if he's busy, he's busy and mails don't really achieve anything, just have to be patient... even more patient :P

Jan 12, 13 03:15:26 0 0 N
Jan 06, 13 17:13:32 0 0 E
Jan 06, 13 05:09:23 0 0 E
Jan 05, 13 09:25:29 0 0 E
Jan 03, 13 14:43:31 1576 3674 Y

Just for my tuppence worth of upload data.

Looks like we've had a major problem with the data of January 5th through January 7th, and only now getting solid processing.

Last processed file was uploaded on 2013-01-08 16:14:23.
Current server time is 2013-01-31 13:18:26

We're now at 22d 21h between processing and current timestamp, and sadly, it looks as if the successfully processed data is slowing down the processing, compared to what looks like quick "processing" of errors.

Jan 11, 13 10:25:42 0 0 N
Jan 08, 13 12:42:42 133 348 Y

Last processed file was uploaded on 2013-01-10 07:41:35.
Current server time is 2013-01-31 23:50:50

Looks like we've jumped ahead 40 hours in data in 10 hours of realtime, but I've no idea how much of that is actual processing and how much is error drops. We're now 21d 16h behind (assuming I've done my math right xD)

(Man, I know I only just started uploading, and I'm on low-pop servers, but my numbers are so pitiful next to everybody else's!)

Seems to be no errors in the time range between the last two quoted "last processed file" times, I found one from just before 16:14 on the 8th but everything on the 9th seems to have gone through fine on the people I've checked. I was about to say it's only moved 10 minutes since your post but it just jumped further on, guess another one finished then:

Last processed file was uploaded on 2013-01-10 08:32:23.
Current server time is 2013-02-01 00:08:20

Don't worry about your numbers btw, every submission helps, no matter how big :) It may be that your server is also being censused by someone else, who if they uploaded before you, might beat you to some updates, which would explain the lower numbers, but even if that's the case, you're still helping to create a bigger and more detailed picture so everything helps :)

Jan 15, 13 22:54:05 0 0 N
Jan 11, 13 10:25:42 576 1527 Y
Jan 08, 13 12:42:42 133 348 Y

Last processed file was uploaded on 2013-01-13 22:25:56.
Current server time is 2013-02-02 14:04:27

So I'm looking at the numbers, since it looks like we're trucking along at a pretty good pace; we're less than 20 days behind now. If I'm right, about 2 hours of datatime is being processed every hour of realtime, so if we keep going along at this pace, we should be caught up in about 10 days or so :O

Last processed file was uploaded on 2013-01-24 10:46:12.
Current server time is 2013-02-08 20:27:43

four days and six hours irl = 10 and a half days of data...

so maybe by... the 15th or 16th things might be caught up? and when they do, are we going to have correct stats and things? cross all the fingers X D

Ruh Roh Reorge.....

Jan 25, 13 21:20:33 0 0 N
Jan 25, 13 01:10:24 0 0 E
Jan 24, 13 23:33:20 0 0 E
Jan 24, 13 22:32:17 701 2815 Y

Last processed file was uploaded on 2013-01-25 04:22:24.
Current server time is 2013-02-09 22:18:37

Last processed file was uploaded on 2013-01-31 13:19:44.
Current server time is 2013-02-13 03:09:54

Queue backlog has reduced to about 12 days, 14 hours from the roughly 13 days, 4 hours I saw before going to bed late last night. I have had no recent failures and latest of Padanfain's seem alright, too. I didn't check for anyone else, but I am hoping things are finally starting to return towards normal operations.

January SHOULD, in my estimate, get finished later today. This despite the fact that there are a lot of files left, many of them pretty big (mine :D ) At the current rate, we will not be fully up-to-date by the 16th as Sanna hoped, but I expect us to be able to see some life in the "past 7 days" part of database stats by then and something resembling up-to-date by late 19th / 20th.

well, 'hope' is kind of a strong word X DD maybe more like 'speculate', 'theorize', or 'throw wildly out there to see what happens' X DD although it will be nice to be back to 'normal'... upload a file and get it crunched in like 12 hours instead of three months later.... : x

Last processed file was uploaded on 2013-01-31 16:09:24.
Current server time is 2013-02-14 23:32:06

*deep sigh* For me, it's already Friday morning. Almost two days have passed and queue has crawled on... less than 3 hours. :( If the rate doesn't (re)improve and quickly, we are headed for another disaster. At current rate it will take an additional WEEK for the January files to finish and we'll back to about 22 days of backlog.

Last processed file was uploaded on 2013-01-31 16:24:02.
Current server time is 2013-02-15 09:57:53

*even deeper sigh* Barely any progress and my files have started to fail (again). :(

Last processed file was uploaded on 2013-01-31 17:17:41.
Current server time is 2013-02-16 01:28:12

Jan 31, 13 17:20:33 0 0 E
Jan 31, 13 16:58:58 0 0 E
Jan 31, 13 16:34:53 0 0 E
Jan 31, 13 16:17:20 0 0 E

*another sigh* Queue has almost entirely choked and more files have failed. And not only mine... at least Bringoutyourdead also has a file failure, lots of other people I checked either don't have files on 31st (after about 14 hours WCR time) or the queue hasn't gotten to their's yet.

I've been having better luck.

I also have an answer to a question we already knew the answer to.

A while back, I took half a dozen files that had failed and re-uploaded them.
Every single one of them processed successfully this time.

We were already pretty sure the file itself had nothing to do with the failures, but this pretty well seals it.

I give up. This site hasn't worked properly almost since the release of MoP, and uploads have been backlogged for months now.

I don't know what's going on behind the scenes, but if the owner can't handle site/database maintenance, maybe it's time for someone else to take over.

I realize we aren't paying anything to use this service, but the site is seriously broken, and has been for a long time.

Oddly enough, no one who has the skills to maintain the site has volunteered to step up and take it over. All that's involved is someone with the necessary skills who wants to invest a lot of time and money, and put up with abuse from users. Surprisingly, people just aren't beating down the door looking for that opportunity.

Well, Jan seems to be complete and Feb already started. Gets better...slowly.

Feb 26, 13 18:00:52 0 0 N

Feb 26, 13 10:50:15 0 0 N

Feb 26, 13 08:32:09 0 0 N

Feb 21, 13 16:43:13 0 0 N

Feb 21, 13 12:40:13 0 0 N

Feb 20, 13 15:48:03 0 0 N

Feb 19, 13 15:37:48 0 0 N

Feb 19, 13 12:10:05 0 0 N

Feb 17, 13 19:09:24 0 0 N

Feb 12, 13 09:54:23 0 0 N

Feb 10, 13 14:06:30 0 0 N

Feb 08, 13 15:22:30 218 912 Y

Jan 31, 13 13:40:20 119 583 Y

Last processed file was uploaded on 2013-02-17 21:04:11.
Current server time is 2013-03-08 00:23:01

Feb 17, 13 15:58:31 0 0 E
Feb 17, 13 12:24:05 4204 15424 Y
Feb 16, 13 18:08:24 0 0 E
Feb 16, 13 16:24:30 0 0 E

*deep sigh* More errors, big ones. The incremental filing saved something, but things are not well. Backlog isn't really reducing and I can see that the failures and delays have already destroyed data as some of my own characters have been updated by other people and levelling info has been skipped, though I know I have it in my files. :(

We were catching up until the server stopped responding and processing. I'm hoping that when it gets to that same time in processing it will jump ahead and get back to closing in on current.
But we are pushing this poor old machine to the limit.
With Rollie having to be out of town for his RL job it makes it hard to do quick fixes, and with the economy the way it has been I expect he doesn't have much money for doing updates.
That is why I used paypal to send him $20, I hope enough people have and will do much the same, and maybe Rollie will be able to upgrade the hardware and get the site back into shape.

I assume the uploads are not working, as I've uploaded multiple times, and been told that my upload was successful, but when I look at the servers i uploaded data for it still shows like 36 ally and 830 horde.. when i have uploaded data showing 1400 alliance players

If you click here you can see when you've uploaded files and if they've been processed:
http://www.warcraftrealms.com/usersheet.php?id=1040074&c=u
As you can see, you've uploaded files on March 24 & 25 and they have not yet been processed.

If you click here you can see where the server is in processing files:
http://www.warcraftrealms.com/dbstats.php
As you can see, it is currently processing files that were uploaded on March 7.

There is quite a backlog, and unfortunately it is not going away as quickly as anyone had expected. All we can do at this point is wait, but the files you uploaded will be processed in time, barring any errors. The short of it is, uploads are working, but processing is painfully slow right now.