WarcraftReamls.com
  FAQFAQ    SearchSearch    MemberlistMemberlist    UsergroupsUsergroups   RegisterRegister 
  ProfileProfile    Log in to check your private messagesLog in to check your private messages    Log inLog in 
Upload processing...

 
Post new topic   Reply to topic    WarcraftRealms.com Forum Index -> WarcraftRealms.com Bugs
View previous topic :: View next topic  
Author Message
Padanfain
Updater Extraordinaire


Joined: 18 Dec 2010
Posts: 173
Location: Shadar Logoth
WR Updates: 40,168,446
Padanfain WR Profile

PostPosted: Fri Jan 27, 2017 7:55 pm    Post subject: Upload processing... Reply with quote

...looks like something broke on about the 25th. Looking at a bunch of user updates tabs and everyone I could see has E's since the 25th.
_________________
"Water? .... I'm thirsty not dirty." - Prince Kheldar of Drasnia
"I've looked at the world for quite a few years now and I've found that if I don't laugh, I'll probably end up crying." - Prince Kheldar of Drasnia
Back to top
View user's profile Send private message
piratejenny
Census Taker


Joined: 05 Oct 2011
Posts: 6

WR Updates: 4,133,105
piratejenny WR Profile

PostPosted: Sat Jan 28, 2017 10:28 am    Post subject: Reply with quote

I made a successful upload last night (a bit later than your post); if somebody fixed something, it worked Smile
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Sat Jan 28, 2017 1:19 pm    Post subject: Reply with quote

Metalbeast did some underlying software upgrades.. including the version of PHP that is used to handle most of the site pages.
Back to top
View user's profile Send private message
FuxieDK
Census Taker


Joined: 22 May 2008
Posts: 616
Location: Copenhagen, DK
WR Updates: 4,395,489
FuxieDK WR Profile

PostPosted: Sat Jan 28, 2017 2:30 pm    Post subject: Re: Upload processing... Reply with quote

Padanfain wrote:
...looks like something broke on about the 25th. Looking at a bunch of user updates tabs and everyone I could see has E's since the 25th.
are you able to salvage the uploads?

I also had an E on 20th...
_________________
Doing census mainly on Draenor; Raluf - Nimsay - Lusmo - Quixx - Sosyan - Garthog - Trubin - Zalistra - Zesmi and Djaang
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Sat Jan 28, 2017 3:39 pm    Post subject: Reply with quote

I am looking at the logs to see what might have gone wrong..
more later after I learn more.
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Sat Jan 28, 2017 7:45 pm    Post subject: Reply with quote

hmm.. the problem is not something really broken.
I have been able to get both web upload and UniUpload files to process.
And I have seen that both fail.
A first level look at the processing log shows the failed files to not pass the initial Lua parsing (to verify that the file is a clean CensusPlus uploaded file.
Unfortunately the next step is never invoked and things appear to just hang until the timer limit is hit.
This could be (and probably is ) related to the PHP upgrade, but it might also be a systems level configuration issue as other code at that level was also replaced.

We may need to wait until Monday before Metalbeast can check what went wrong.

What I think is happening is that PHP processes are hitting default cut off timers when the upload files are too big for the default settings.
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Sun Jan 29, 2017 6:48 pm    Post subject: Reply with quote

Further investigation discovers this

Uploads of about 1.5MB and smaller process fine.

It appears that two of the control parameters for the PHP processor may have been reset back to default levels.

Correcting these parameters and making them active would require a restart of the PHP processor.. and may impact other domains on the server.. so I have informed Metalbeast of what I have found.

I expect the problem to be fixed shortly after he reads my email sometime Monday morning.. probably shortly after he starts his business day.

After he has informed me of the system level fix I will go into the control table for uploaded files and reset the failed uploads as unprocessed and they will process again.
By Monday Evening (PST) we should be back to normal with all files correctly processed.
Back to top
View user's profile Send private message
Padanfain
Updater Extraordinaire


Joined: 18 Dec 2010
Posts: 173
Location: Shadar Logoth
WR Updates: 40,168,446
Padanfain WR Profile

PostPosted: Sun Jan 29, 2017 7:40 pm    Post subject: Reply with quote

Shiny Very Happy TY
_________________
"Water? .... I'm thirsty not dirty." - Prince Kheldar of Drasnia
"I've looked at the world for quite a few years now and I've found that if I don't laugh, I'll probably end up crying." - Prince Kheldar of Drasnia
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Mon Jan 30, 2017 5:55 pm    Post subject: Reply with quote

Metalbeast discovered that his newest version of server control software had broken out some of the PHP control variables into a separate setup function... so they had been missed on setup.. love new software version changes.

I reset about 150 failed uploads back to unprocessed and kicked the processor into action.. of those, 12 failed again.. ( I knew some would because they failed initially with unknown or corrupted LUA data.)

I am looking at those 12 to see if I can spot the problem as these failures stopped the current active processing cycle.
This means it might be 12 hours before all the problem files were removed from the to be processed list (if I did nothing), not a real big problem... but something that Metalbeast might want to look into himself.

NOTE:
The reprocessing was in the order received so you should have gotten credit for updates as normal.
Except if someone including yourself, did a small file update on the same server/faction they might have slipped in and gotten credit for a small number of updates..
Just the way it goes.
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Mon Jan 30, 2017 6:50 pm    Post subject: Reply with quote

Looks like we have a new old problem
files larger then 8MB or so are failing to process..
I am not sure exactly what the problem is.. and have passed it on to Metalbeast.

Either we need to change some parameters to handle the larger files or we need to change the process for verifying that the files are clean CensusPlus upload files.

FuxieDK and Kanegasi have the most failed uploads.. having hit the size limit.. that shouldn't have happened.

After Metalbeast determines the fix.. I will again reset those 12 uploads and process them again.. maybe tomorrow... or maybe much later if we have to rewrite code.
Back to top
View user's profile Send private message
FuxieDK
Census Taker


Joined: 22 May 2008
Posts: 616
Location: Copenhagen, DK
WR Updates: 4,395,489
FuxieDK WR Profile

PostPosted: Tue Jan 31, 2017 1:58 am    Post subject: Reply with quote

I upload, solely, with UniUpload, where size shouldn't matter.. Mad

The uploads from 25th-27th are now flagged as Y, but my latest upload is E...
_________________
Doing census mainly on Draenor; Raluf - Nimsay - Lusmo - Quixx - Sosyan - Garthog - Trubin - Zalistra - Zesmi and Djaang


Last edited by FuxieDK on Wed Feb 01, 2017 8:54 am; edited 1 time in total
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Tue Jan 31, 2017 12:30 pm    Post subject: Reply with quote

The problem wasn't the actual upload process where UniUpload gets around the 8MB Apache web server limit.
The problem was (mostly) the maximum memory limit for running a php process.
Metalbeast increased the limit again last night just before he left work.
I am going to check why the last few E results still exist.. it just may be I missed marking them as unprocessed.
Back to top
View user's profile Send private message
bringoutyourdead
Forums Admin & general flunky


Joined: 07 Nov 2008
Posts: 1214
Location: Silicon Valley, CA, USA
WR Updates: 9,182,216
bringoutyourdead WR Profile

PostPosted: Tue Jan 31, 2017 3:34 pm    Post subject: Reply with quote

All the E status results for the last 10 days that are still E status, are valid failures.
The all failed the clean lua parsing.

Either the files were corrupted in creation or in transmission.

If you continue to have E status on uploads with those files.. delete the data file and restart your data collection.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    WarcraftRealms.com Forum Index -> WarcraftRealms.com Bugs All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
WarcraftRealms.com  


Powered by phpBB © 2001, 2005 phpBB Group