Biting off too much to crunch!
log in

Advanced search

Message boards : Number crunching : Biting off too much to crunch!

Author Message
Gunnar Hjern
Send message
Joined: 29 Mar 17
Posts: 11
Credit: 6,749,875
RAC: 8,720
Sweden
Message 1513 - Posted: 1 Mar 2019, 23:48:17 UTC

There seems to have been a whole lot of excessive WU downloading lately. My pending bin is steadily growing and has more than doubled the last day! :-(

When taking a closer look, several computers have as much as 512 WUs each in progress (without doing much crunching) and there is little chance they will be ready in time.

Apparently the number of cores per computer can be manipulated, and the worst I've seen so far is a couple of computers that state they got over 3000 cores each!!!
(I'd love to really see one of those machines - they must be quite impressive... ;-)

I recommend the project admins to put up some harder limits for how many WUs that can be held in progress, so that the servers don't run completely dry.

Kindest regards,
Gunnar Hjern

Jim1348
Send message
Joined: 29 Dec 16
Posts: 27
Credit: 2,151,208
RAC: 10,571
United States
Message 1514 - Posted: 2 Mar 2019, 2:30:21 UTC - in response to Message 1513.

There seems to have been a whole lot of excessive WU downloading lately. My pending bin is steadily growing and has more than doubled the last day! :-(

I have also seen that behavior a year or two ago. It is very strange. However, things are all OK for me now, and I have the expected number of work units (18). That is reasonable for my buffer size of 0.1 + 0.5 days, which is the default.

Obviously, something gets messed up once in a while. Why don't you detach from the project, reset the buffer to the default size, and try again? It might straighten it out.

MPG
Send message
Joined: 15 Jan 17
Posts: 1
Credit: 8,490,965
RAC: 10
Netherlands
Message 1515 - Posted: 2 Mar 2019, 10:20:03 UTC

Apparently the number of cores per computer can be manipulated


Indeed a Ryzen Threadripper 2990WX doesn't have >3000 cores !
Or AMD has really special procs out there ;-)

Some kind of limit of maximum WU's (in progress) for each computer to avoid WU's not being processed before the deadline.

Profile Buro87 [Lombardia]
Send message
Joined: 23 Nov 16
Posts: 33
Credit: 1,737,992
RAC: 1,928
Italy
Message 1516 - Posted: 2 Mar 2019, 16:18:04 UTC - in response to Message 1515.
Last modified: 2 Mar 2019, 16:27:46 UTC

All theese wu are only bunker fo the upcoming challenge on the project, issued by TSBT, starting March 4th
http://gene.disi.unitn.it/test/forum_thread.php?id=241

You can avoid the wu limit (10wu per core) increasing the value "ncpus" in the cc_config.xml (only in the bunker construction phase)

Gunnar Hjern
Send message
Joined: 29 Mar 17
Posts: 11
Credit: 6,749,875
RAC: 8,720
Sweden
Message 1517 - Posted: 2 Mar 2019, 17:55:44 UTC - in response to Message 1516.

Such hoarding is indeed a very blatant violation of fair competition rules and should lead to that all of them being disqualified!!!

For what good is it anyway???
The competition starts at March 4th, and a whole lot of the hoarded WUs will timeout early at 6 o'clock the very same day! :-O

It is the rest of us serious Boinc:ers that will have to pay the price for this malpractice, not to speak of the science project itself!
I've just PM:ed the admin, and hope that he will put a stop to this and let the project survive.

A really nice weekend to all of you fair working crunchers!!!

//Gunnar

Profile Buro87 [Lombardia]
Send message
Joined: 23 Nov 16
Posts: 33
Credit: 1,737,992
RAC: 1,928
Italy
Message 1518 - Posted: 2 Mar 2019, 22:16:05 UTC - in response to Message 1517.
Last modified: 2 Mar 2019, 22:17:49 UTC

It is a normal practice for each challenge issued on every project.
When starting date and hour is set (in this case March 4th 0:00 UTC) you calculate when you can begin download wu to return after the challenge is started (in this case 5 or les days). The user is interest to execute wu an upload them, before deadline to recieve credit.

Personally, doing bunker, i do some maths to calculate the right amount of wu needed in this period
However many user don't do this


By the way, theese challenge are kind of boost for the project, because maybe some partecipant decide to stay on the project
It's also a server' stability test

Gunnar Hjern
Send message
Joined: 29 Mar 17
Posts: 11
Credit: 6,749,875
RAC: 8,720
Sweden
Message 1519 - Posted: 3 Mar 2019, 0:07:42 UTC - in response to Message 1518.

Maybe Zioriga could finish his tasks in due time - if he really have those monster machines that the listing says - but I think Cruncher Pete must have miscalculated a bit:
He has a couple of computers that should finish 512 tasks each until about 8 o'clock in the morning on Monday. Another of his machines have 994 tasks to finish in 24 hours!

No way on this earth he will manage with that!! All my own computers put together needs almost a whole week to eat up 994 WU:s, and I know that pretty well as I have been running mine 24/7 the last three months.
(Maybe he's working on his bunch of WUs already this weekend, and waiting to upload them all until Monday morning. Wouldn't that be called cheating? ;-)

I do agree that some competition will boost interest for the project, but I hope it will not bring the servers down or cause some other troubles along the way.
Hopefully, we will receive all the credits for our pending WUs once the fighting is over next week. :-)

Have a nice weekend!!!

//Gunnar

Alez
Send message
Joined: 7 Feb 16
Posts: 9
Credit: 2,426,453
RAC: 213
Anguilla
Message 1520 - Posted: 4 Mar 2019, 0:57:18 UTC - in response to Message 1519.

Gunnar, watch and learn as most of those units are returned within time for the challenge. I have already posted that the challenge is taking place, the project is aware. The servers will not fail and the project will have a big boost in computing power.
Try to have a nice day ... :)

Gunnar Hjern
Send message
Joined: 29 Mar 17
Posts: 11
Credit: 6,749,875
RAC: 8,720
Sweden
Message 1522 - Posted: 4 Mar 2019, 1:06:41 UTC - in response to Message 1520.

Hi Alez!

Well it's time to hit the bed now, but I'm quite satisfied to see that more than 11,000 creds already have returned from the pending bin - all in a single hour!! :-)

Have a good competition week!!

//Gunnar

Alez
Send message
Joined: 7 Feb 16
Posts: 9
Credit: 2,426,453
RAC: 213
Anguilla
Message 1523 - Posted: 4 Mar 2019, 1:17:37 UTC - in response to Message 1522.

Thank you Gunnar. I hope that it is a good competition, a bit of fun had by all taking part and the project is the big winner at the end of the week.
Sleep well.

Profile Cruncher Pete
Send message
Joined: 13 May 14
Posts: 2
Credit: 24,329,609
RAC: 39,605
Anguilla
Message 1524 - Posted: 4 Mar 2019, 5:58:44 UTC - in response to Message 1519.

Maybe Zioriga could finish his tasks in due time - if he really have those monster machines that the listing says - but I think Cruncher Pete must have miscalculated a bit:
He has a couple of computers that should finish 512 tasks each until about 8 o'clock in the morning on Monday. Another of his machines have 994 tasks to finish in 24 hours!

By way of protecting myself since you have implicated me (I changed my name from Cruncher Pete to PeterJ.Shane for security reasons) Let me explain a couple of things.

1. I can assure you that it is legitimate to set your cache to 10Days for I need a number of WU's downloaded to me that will last that long as I am not always available to check my machines every day. Since the maximum number of WU's that is allowed as maximum on each machine is 512 I realized that that is not enough to last me 10 days so I created another instance of BOINC Client on the same machine. I have no idea where you got the idea that I needed to complete these WU's in 24 hours. As you can see now that the Challenge has started they were all completed, uploaded and credited after the challenge started.

2. I have a number of 64 core/thread computers and a horde of others. In order to be a fair competition, do you expect me to switch off some of my machines to match yours thereby slowing the results of the project? And I suppose you would than call this a fair competition. I have paid for my computers out of my own pocket and they are all dedicated to helping Scientific Research. I am sure that the Admin of this project will vouch for my efforts to help this project for I have been crunching it long before the challenge started.

3. I can assure that what I am doing is not cheating and I take offense that you insinuate that I am. Please read up on how BOINC works, what is allowed and what is not. Further more what I am doing is legal and is used by many users and the method is available to all users.

I hope that explains the reasons for my high scores. Should you wish to find more information on it, please PM me and I will send you the links where you can find more information on how to use BOINC.

No way on this earth he will manage with that!! All my own computers put together needs almost a whole week to eat up 994 WU:s, and I know that pretty well as I have been running mine 24/7 the last three months.
(Maybe he's working on his bunch of WUs already this weekend, and waiting to upload them all until Monday morning. [/i]
I do agree that some competition will boost interest for the project, but I hope it will not bring the servers down or cause some other troubles along the way.
Hopefully, we will receive all the credits for our pending WUs once the fighting is over next week. :-)

Have a nice weekend!!!

//Gunnar

Profile Cruncher Pete
Send message
Joined: 13 May 14
Posts: 2
Credit: 24,329,609
RAC: 39,605
Anguilla
Message 1525 - Posted: 4 Mar 2019, 6:05:53 UTC - in response to Message 1519.

@Gunner Hjern.By way of protecting myself since you have implicated me (I changed my name from Cruncher Pete to PeterJ.Shane for security reasons) Let me explain a couple of things.

1. I can assure you that it is legitimate to set your cache to 10Days for I need a number of WU's downloaded to me that will last that long as I am not always available to check my machines every day. Since the maximum number of WU's that is allowed as maximum on each machine is 512 I realized that that is not enough to last me 10 days so I created another instance of BOINC Client on the same machine. I have no idea where you got the idea that I needed to complete these WU's in 24 hours. As you can see now that the Challenge has started they were all completed, uploaded and credited after the challenge started.

2. I have a number of 64 core/thread computers and a horde of others. In order to be a fair competition, do you expect me to switch off some of my machines to match yours thereby slowing the results of the project? And I suppose you would than call this a fair competition. I have paid for my computers out of my own pocket and they are all dedicated to helping Scientific Research. I am sure that the Admin of this project will vouch for my efforts to help this project for I have been crunching it long before the challenge started.

3. I can assure that what I am doing is not cheating and I take offense that you insinuate that I am. Please read up on how BOINC works, what is allowed and what is not. Further more what I am doing is legal and is used by many users and the method is available to all users. May I suggest that you do a bit of studying before you open your mouth and call people cheaters.

I hope that explains the reasons for my high scores. Should you wish to find more information on it, please PM me and I will send you the links where you can find more information on how to use BOINC.

davidBAM
Send message
Joined: 23 Feb 19
Posts: 3
Credit: 2,042,146
RAC: 2
United Kingdom
Message 1526 - Posted: 4 Mar 2019, 9:30:55 UTC

Seems to me that Competitive crunching is about getting the maximum number of WU validated during the timespan of the Challenge AND before the WU expire. For example, I am unaware of any 'rules' which forbid bunkering.

Peter and others in The Scottish Boinc Team have educated me into a whole range of techniques to help me maximise output from my own machines.

Like Pete, I pay for my own machines and my own electricity but let me point out that Pete is rated #9 in the all-time, cross project total points scored on Boincstats. Modesty will have prevented him from telling that he has accumulated over 650 BILLION points to advance medical and scientific research.

All in all, it is hard to see where anyone is losing out - perhaps the projects not chosen to be in the challenge? So your pending WU take a few days longer to clear - not really a big deal is it?

Gunnar Hjern
Send message
Joined: 29 Mar 17
Posts: 11
Credit: 6,749,875
RAC: 8,720
Sweden
Message 1527 - Posted: 4 Mar 2019, 9:31:39 UTC - in response to Message 1525.

Hi Peter!

I have never ever stated - and nor have I believed - that you weren't using your own computers, or that your were doing something illegal!!!

When I saw that my pending bin was very rapidly growing, I started to investigate the reasons, and then I found several computers with 512 and even more WUs in progress, and many of them also seemed to be inactive, not having completed any WUs in several days. On top of this, some of these computers was listed as having over 3000 cores, something I didn't thought ever existed. All this certainly seemed rather odd to me, to say the least, and it was this that made me react the way I did.

As the competition started at midnight and all the deadlines for the tasks I saw on your computers had deadlines before midnight to Tuesday, it isn't that far off to believe that they have to be completed in 24 hours?

To my great satisfaction I can confirm that a great part of the WUs in my pending bin now have been credited! :-)

At the time of writing I can only find a dozen of WU's that seem to have timed out:
18702265, 18702546, 18702672, 18703499, 18703675, 18703751, 18703765, 18705100, 18705192, 18705375, 18706313, 18706596
(Maybe some or even all of them will be reported as completed after this is posted?)

I am certainly not an attorney trying to nail anyone participating in different competition! From the beginning I was not even aware that this whole issue was due to any competition, and nor am I aware of the exact rules for such competitions. Therefore I will not further argument about what is regarded as competition cheating or fair play.

I hope you do not feel further offended by me, Peter. That is certainly not my intention!

I hope that in the end all this competing will lead to the benefit of the project and the science behind it - that's what we're in it for.

So continue to crunch!, you sure have some mighty computers to do it with! :-)

Have a nice day, and a good week to come!!!

Kindest regards,
Gunnar

Gunnar Hjern
Send message
Joined: 29 Mar 17
Posts: 11
Credit: 6,749,875
RAC: 8,720
Sweden
Message 1528 - Posted: 4 Mar 2019, 9:49:42 UTC - in response to Message 1526.

Hi David!

No I'm not that worried about the WUs in my pending bin! I'm quite convinced that they will return as credited in the end.
I saw a lot of very strange things going on, and wanted the security and functionality of the project to prevail.

I certainly salute the scientific calculation work that you all are doing!!!

Although I have not been into Boinc:ing for more than three years, and have not the most powerful computers, I am running the 16 old cans I have both day and night. I really like to be a part of the good science that can be done this way!

Have a good day, and Happy crunching to you all!

//Gunnar

davidBAM
Send message
Joined: 23 Feb 19
Posts: 3
Credit: 2,042,146
RAC: 2
United Kingdom
Message 1529 - Posted: 4 Mar 2019, 11:48:50 UTC - in response to Message 1528.

Hi Gunnar

Many thanks for your message & I totally agree with the sentiments you express.

You would be most welcome to visit us on www.tsbt.co.uk and, if I can point out, you don't need to be based in Scotland to join our team :-)

Happy crunching

Profile Buro87 [Lombardia]
Send message
Joined: 23 Nov 16
Posts: 33
Credit: 1,737,992
RAC: 1,928
Italy
Message 1530 - Posted: 4 Mar 2019, 14:11:27 UTC - in response to Message 1529.
Last modified: 4 Mar 2019, 14:12:43 UTC

Hi David,
are you running all 64 thread on this monster? http://gene.disi.unitn.it/test/show_host_detail.php?hostid=44901
15700s is impressive for 64 thread

AMD is doing grat job for BOINC users

davidBAM
Send message
Joined: 23 Feb 19
Posts: 3
Credit: 2,042,146
RAC: 2
United Kingdom
Message 1531 - Posted: 4 Mar 2019, 14:24:20 UTC - in response to Message 1530.
Last modified: 4 Mar 2019, 14:27:41 UTC

Yes, sir, all 64 threads crunching hard on TN-Grid. Same machine is additionally running PrimeGrid on three nVidia 1080 incidentally so it is earning well. Keeps me nice and warm here in Aberdeenshire

I am not overclocking the Ryzen but I am running only fma work units

ETA: under Linux by the way

Alez
Send message
Joined: 7 Feb 16
Posts: 9
Credit: 2,426,453
RAC: 213
Anguilla
Message 1532 - Posted: 5 Mar 2019, 1:56:39 UTC - in response to Message 1528.
Last modified: 5 Mar 2019, 1:59:36 UTC

Hi David!

No I'm not that worried about the WUs in my pending bin! I'm quite convinced that they will return as credited in the end.
I saw a lot of very strange things going on, and wanted the security and functionality of the project to prevail.

I certainly salute the scientific calculation work that you all are doing!!!

Although I have not been into Boinc:ing for more than three years, and have not the most powerful computers, I am running the 16 old cans I have both day and night. I really like to be a part of the good science that can be done this way!

Have a good day, and Happy crunching to you all!

//Gunnar


Gunnar,
As david said, if you wish to learn a bit more about optimizing your systems, or simple wish to say hi, then do visit our forum. You will be more than welcome.
regards
Alez

mikey
Avatar
Send message
Joined: 28 Nov 16
Posts: 7
Credit: 4,574,501
RAC: 19,366
United States
Message 1586 - Posted: 25 Jun 2019, 12:30:16 UTC - in response to Message 1519.
Last modified: 25 Jun 2019, 12:30:41 UTC

Maybe Zioriga could finish his tasks in due time - if he really have those monster machines that the listing says - but I think Cruncher Pete must have miscalculated a bit:
He has a couple of computers that should finish 512 tasks each until about 8 o'clock in the morning on Monday. Another of his machines have 994 tasks to finish in 24 hours!

No way on this earth he will manage with that!! All my own computers put together needs almost a whole week to eat up 994 WU:s, and I know that pretty well as I have been running mine 24/7 the last three months.
(Maybe he's working on his bunch of WUs already this weekend, and waiting to upload them all until Monday morning. Wouldn't that be called cheating? ;-)

I do agree that some competition will boost interest for the project, but I hope it will not bring the servers down or cause some other troubles along the way.
Hopefully, we will receive all the credits for our pending WUs once the fighting is over next week. :-)

Have a nice weekend!!!

//Gunnar


One thing to know is that not all projects allow "bunkering' of the workunits for 'challenges', some do and some don't. For those that do it's normal practice to fill up your cache and turn off the returning of the wu's until after the challenge starts so you can get a head start on those people who don't take advantage of "bunkering". For instance if you can crunch 1000 wu's in 3 days and the challenge starts in 3 days then you could download 1000 wu's and return them just after the challenge starts giving yourself a large lead, after that you are just like everyone else in that you crunch a wu and return it.

The 'problem' with bunkering, as your journey down this path showed, is that "bunkering" does delay the credits for those of us who don't do it. It can make us take a look to see if we have a problem with our computers or not, but now that you understand about "bunkering" and challenges it should help the next time.

I almost never do "bunkering" or challenges as even though I have 16 pc's here at my house they can't compete with the large numbers of pc's that people bring to the challenges, so instead I just concentrate on goals of my own and continue doing my thing. To me it's a blip in my stats and then it's back to normal again as the people who did the challenge move on to the next one.
____________


Post to thread

Message boards : Number crunching : Biting off too much to crunch!


Main page · Your account · Message boards


Copyright © 2019 CNR-TN & UniTN