WU / task creation flow
log in

Advanced search

Message boards : Number crunching : WU / task creation flow

Author Message
JaviF
Send message
Joined: 14 May 19
Posts: 41
Credit: 0
RAC: 0
Spain
Message 1630 - Posted: 31 Oct 2019, 16:42:55 UTC

Hi team,

Would it be possible to see WU creation, task sent in a day...all that kind of stats /graphs?

Which would be the limit server could deal with,using current set up?
200 CPU cores more? (Example).

Have a nice day.

Javi

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 6
Italy
Message 1632 - Posted: 4 Nov 2019, 10:15:28 UTC - in response to Message 1630.
Last modified: 4 Nov 2019, 10:16:11 UTC

Hi team,

Would it be possible to see WU creation, task sent in a day...all that kind of stats /graphs?

Which would be the limit server could deal with,using current set up?
200 CPU cores more? (Example).

Have a nice day.

Javi

Ok, we don't have fancy graphs showing the workflow. The only informational page is https://gene.disi.unitn.it/test/gene_science.php, which shows some statistics about the currently running experiment.
BTW, the algorithm for generating workunits is rather complicated, i.e. it takes some time to generate a workunit and add it to the queue. With our current setup (hardware/software) we are able to create 294 workunits (588 results, because of the replication) every ~15 minutes, I let you do the math ;). This is our theoretical limit, when reached the queue will start to dry up.

JaviF
Send message
Joined: 14 May 19
Posts: 41
Credit: 0
RAC: 0
Spain
Message 1633 - Posted: 4 Nov 2019, 13:38:08 UTC - in response to Message 1632.

Interesintg, less is nothing:)

But I was also wondering tasks received and all that kind of "flows" in Wu/ tasks here and there.

I just added cpu to perform aprox 100 tasks a day and was wondering how "much" is that :)

Profile Buro87 [Lombardia]
Send message
Joined: 23 Nov 16
Posts: 100
Credit: 4,000,541
RAC: 0
Italy
Message 1656 - Posted: 18 Dec 2019, 14:29:24 UTC - in response to Message 1633.
Last modified: 18 Dec 2019, 14:35:51 UTC

Hi Valterc,
If i remember right, 1 gene expansion is composed by 294wu (588 vith replication. For every 15minutes = 96 gene/day and 56448 wu/day generated

There is a queue's top limit? why wu "ready to send" are alays around 8-9k and dont raise up?
According to science page we are currently process around 30 gene/day

It would help during challenges

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 6
Italy
Message 1657 - Posted: 19 Dec 2019, 10:19:38 UTC - in response to Message 1656.

Hi Valterc,
If i remember right, 1 gene expansion is composed by 294wu (588 vith replication. For every 15minutes = 96 gene/day and 56448 wu/day generated

There is a queue's top limit? why wu "ready to send" are alays around 8-9k and dont raise up?
According to science page we are currently process around 30 gene/day

It would help during challenges

Yes, the work generator starts doing its job if the number of available workunits is lower than a threshold, to optimize server's memory. Currently this number is 9000 and this is usually enough in 'normal' times. I will set the cap to 10000 and see what happens.

JaviF
Send message
Joined: 14 May 19
Posts: 41
Credit: 0
RAC: 0
Spain
Message 1813 - Posted: 8 May 2020, 6:40:49 UTC

Hi team,

It seems from server_status, we are running out of tasks...

Maybe is just a display issue, I can't say :)

Thx for checking¡
Javi F

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 623
Credit: 34,677,535
RAC: 6
Italy
Message 1814 - Posted: 8 May 2020, 9:24:55 UTC - in response to Message 1813.

Hi team,

It seems from server_status, we are running out of tasks...

Maybe is just a display issue, I can't say :)

Thx for checking¡
Javi F

Someone (with a large number of computers) is getting a lot of tasks, the number of "Tasks in progress" is much larger than yesterday. Usual problem here, slow server and slow work generator.....

JaviF
Send message
Joined: 14 May 19
Posts: 41
Credit: 0
RAC: 0
Spain
Message 1815 - Posted: 8 May 2020, 9:47:30 UTC - in response to Message 1814.

I see.

Flow of new tasks is really low. As per below Numbers, is 588 each 15 mins, around 2350 tasks / hour.

Even a tiny contributor like me, who has 16 cpu cores, I require 4 tasks / hour average.
4 / 2350 = 0,17 %.

Current flow could not even support 1000 contributors like me, of course as soon as someone comes with 200 cores, server wont be able to feed all users :(

Aurum
Send message
Joined: 18 Jul 18
Posts: 97
Credit: 291,386,295
RAC: 0
United States
Message 1816 - Posted: 8 May 2020, 17:03:06 UTC - in response to Message 1814.
Last modified: 8 May 2020, 17:03:36 UTC

Someone (with a large number of computers) is getting a lot of tasks
Sorry, crunchers gotta crunch :-)
You do know you're the Formula-BOINC Sprint this week. League One is devouring them:
http://formula-boinc.org/sprint.py?sprint=6&lang=&year=2020
____________

Jan Vaclavik
Send message
Joined: 7 Apr 20
Posts: 2
Credit: 57,748
RAC: 0
Czech Republic
Message 1818 - Posted: 9 May 2020, 19:46:19 UTC - in response to Message 1815.
Last modified: 9 May 2020, 19:47:50 UTC

I see.

Flow of new tasks is really low. As per below Numbers, is 588 each 15 mins, around 2350 tasks / hour.

Even a tiny contributor like me, who has 16 cpu cores, I require 4 tasks / hour average.
4 / 2350 = 0,17 %.

Current flow could not even support 1000 contributors like me, of course as soon as someone comes with 200 cores, server wont be able to feed all users :(

Aurum makes a good point about the project having increased demand due to an ongoing event.
That being said, the project has 944 users with recent credit. While some may have multiple machines with many cores, others compute only few WUs per week. Under normal circumstances, the project may generate enough WUs to supply all its crunchers.


Post to thread

Message boards : Number crunching : WU / task creation flow


Main page · Your account · Message boards


Copyright © 2024 CNR-TN & UniTN