Hi,
from today (about 9.35 a.m.) the wu queue is empty.
I'll "switch" my pc for some hours.
Thanks
K.
Reply
Quote
valtercProject administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 2
Hi,
from today (about 9.35 a.m.) the wu queue is empty.
I'll "switch" my pc for some hours.
Thanks
K.
We have to move 'slowly' during this weekend. Until we fix the 'too big output file' issue the system needs a lot of micro-management. I hope that at beginning of the next week we'll be able to solve this problem.
Thanks all for your understanding.
Reply
Quote
Hi, will you post here when the issue is fixed?
I'd move my crunch boxes back to tn-grid then...
Cheers
Holger
____________
My stats
Reply
Quote
valtercProject administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 2
Hi, will you post here when the issue is fixed?
I'd move my crunch boxes back to tn-grid then...
Cheers
Holger
Yes, certainly, I will make an official announcement (news)
Reply
Quote
valtercProject administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 2
We are almost done writing the new application. We had to make a new one because changing the output file format implies that there would be no cross-validation between results made by this and the old one (a simple version change don't work in this cases)
We are testing it locally, we will keep you all updated.
Reply
Quote
We are almost done writing the new application. We had to make a new one because changing the output file format implies that there would be no cross-validation between results made by this and the old one (a simple version change don't work in this cases)
We are testing it locally, we will keep you all updated.
I recall that some projects which wanted to update app to new version which was not compatible with old one decided to stop generating new WUs and wait until all existing WUs were validated, then rolled out new app version. Your WUs have quite short deadline, maybe it would be worth to wait a few days more until all (or most) of existing WUs will be returned?
____________
Reply
Quote
valtercProject administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 2
We are almost done writing the new application. We had to make a new one because changing the output file format implies that there would be no cross-validation between results made by this and the old one (a simple version change don't work in this cases)
We are testing it locally, we will keep you all updated.
I recall that some projects which wanted to update app to new version which was not compatible with old one decided to stop generating new WUs and wait until all existing WUs were validated, then rolled out new app version. Your WUs have quite short deadline, maybe it would be worth to wait a few days more until all (or most) of existing WUs will be returned?
yes, this is also a solution, but the best one (without waiting times) is to have for some days both applications running (validator and assimilator are the same), stopping the 'old' work generator. When all the 'old' workunits are back I can simply remove the old application.
This way is also easier to deal with people running the 'old' anonymous platform, they will simply receive no new work (and will notice it) until they switch back to their default platform.
Reply
Quote
So the reason it is hard to get new WU is not that there are no new ones, but that there are not enough because the work creator is too slow?
Reply
Quote
valtercProject administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 2
So the reason it is hard to get new WU is not that there are no new ones, but that there are not enough because the work creator is too slow?
We have plenty of work to do and workunits to distribute. The work generator is slow, we may solve this problem either rewriting it or upgrading our hardware (this isn't so easy). Right now we are able to prepare around 170 workunits (times two, for redundancy) in about 8 minutes. A workunit may return in around 20-25 minutes, so it's almost impossible to build up a large queue. A fix would be to build up larger workunits (with average running time of more than 60-90 minutes) but we cannot do this until we fix the 'too big' output file issue. This is a problem for users with limited bandwidth but also for us having to micro-manage our whole storage system.
We are working to find a reliable solution. Thank you for your understanding.
Reply
Quote
...
We have plenty of work to do and workunits to distribute. The work generator is slow, we may solve this problem either rewriting it or upgrading our hardware (this isn't so easy). Right now we are able to prepare around 170 workunits (times two, for redundancy) in about 8 minutes. A workunit may return in around 20-25 minutes, so it's almost impossible to build up a large queue. A fix would be to build up larger workunits (with average running time of more than 60-90 minutes) but we cannot do this until we fix the 'too big' output file issue. This is a problem for users with limited bandwidth but also for us having to micro-manage our whole storage system...
Very nice and clear explanation. Thank you! And best of luck in finding the solution soon! :-)
Reply
Quote
... A workunit may return in around 20-25 minutes, so it's almost impossible to build up a large queue. A fix would be to build up larger workunits (with average running time of more than 60-90 minutes)...
On my i7 7700k it takes around 10-14 minutes per WU with HT on so one WU per 1-2 minutes if all threats are used. The 2600K is still half as fast. realy looking forward to the "new" WU´s ;)
keep up the good work. The increase of efficiency was huge over the last month so its no surprise the WU´s are "eaten" faster than produced at the moment.
Reply
Quote
Is there any particular reason why there are no wus?
I know there was an explanation that they may be longer and I can understand that.
We are dedicated so it shouldn't be a problem. Why not more credit if they are longer?
Reply
Quote
valtercProject administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 2
Is there any particular reason why there are no wus?
I know there was an explanation that they may be longer and I can understand that.
We are dedicated so it shouldn't be a problem. Why not more credit if they are longer?
Very briefly, I have to limit the number of wu because of storage problems. We are waiting for a new application with a way smaller output file, this will hopefully solve the problem...
Reply
Quote
Is there any particular reason why there are no wus?
I know there was an explanation that they may be longer and I can understand that.
We are dedicated so it shouldn't be a problem. Why not more credit if they are longer?
Very briefly, I have to limit the number of wu because of storage problems. We are waiting for a new application with a way smaller output file, this will hopefully solve the problem...
Do you have an approximate time frame for the new app? To many of my machines are idle :)
Reply
Quote
We need some communication here, please.
Reply
Quote
See the news section : http://gene.disi.unitn.it/test/forum_thread.php?id=162
Reply
Quote