Curious
log in

Advanced search

Message boards : Number crunching : Curious

Previous · 1 · 2 · 3 · Next
Author Message
Dave J
Send message
Joined: 3 Mar 22
Posts: 2
Credit: 65,600
RAC: 0
United Kingdom
Message 2709 - Posted: 6 Jun 2022, 19:04:20 UTC

Still got one refusing to complete but another four seem to have uploaded without isue.

TLD
Send message
Joined: 16 Feb 22
Posts: 3
Credit: 9,252,686
RAC: 0
United States
Message 2710 - Posted: 7 Jun 2022, 15:36:53 UTC - in response to Message 2703.

I have two finished tasks that I haven't been able to get uploaded to the server all day for some reason.

Tried all the tricks that I know, but nothing has worked. Just keep getting retried. Counts are 11 and 14 attempts so far.



I also have two tasks from different computers that refuse to upload is there a way to rectify this or should they just be aborted?

Aurum
Send message
Joined: 18 Jul 18
Posts: 97
Credit: 291,237,804
RAC: 6,671
United States
Message 2711 - Posted: 7 Jun 2022, 16:21:28 UTC - in response to Message 2710.

I have two finished tasks that I haven't been able to get uploaded to the server all day for some reason.

Tried all the tricks that I know, but nothing has worked. Just keep getting retried. Counts are 11 and 14 attempts so far.

I also have two tasks from different computers that refuse to upload is there a way to rectify this or should they just be aborted?

Don't abort them. They can be aborted from the server if that becomes necessary. They do not interfere with new WUs DLing so we can keep on crunching.

TLD
Send message
Joined: 16 Feb 22
Posts: 3
Credit: 9,252,686
RAC: 0
United States
Message 2712 - Posted: 7 Jun 2022, 16:30:45 UTC

Thanks for reply.

Profile Keith Myers
Send message
Joined: 26 Jun 20
Posts: 63
Credit: 15,280,600
RAC: 672
United States
Message 2713 - Posted: 8 Jun 2022, 2:29:11 UTC

So what magic rabbit did you pull out of your hat @valterc?

I see that both my stalled uploads cleared out sometime today after I last checked this morning.

!Yay! In time to before deadline.

Aurum
Send message
Joined: 18 Jul 18
Posts: 97
Credit: 291,237,804
RAC: 6,671
United States
Message 2714 - Posted: 8 Jun 2022, 4:31:39 UTC

Nice work Valter! Mine have all ULed and back to normal :-)

entity
Send message
Joined: 20 Jul 20
Posts: 20
Credit: 31,435,274
RAC: 17,248
United States
Message 3111 - Posted: 20 Mar 2023, 14:09:39 UTC - in response to Message 2714.

Why is the number of Queued genes/isoforms greater than the number of genes/isoforms? Was something added to the project? Source: science status page

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 614
Credit: 34,508,279
RAC: 1,041
Italy
Message 3114 - Posted: 21 Mar 2023, 18:32:10 UTC - in response to Message 3111.

Why is the number of Queued genes/isoforms greater than the number of genes/isoforms? Was something added to the project? Source: science status page

Well, there is something wrong with it.... I will check this out.

entity
Send message
Joined: 20 Jul 20
Posts: 20
Credit: 31,435,274
RAC: 17,248
United States
Message 3118 - Posted: 29 Mar 2023, 2:47:38 UTC - in response to Message 3114.

It is looking like the original count of genes/isoforms was/is wrong. I believe we still have at least 5 days if not 7 more days of work left to crunch. The queued number seems to be the more accurate count. Work seems to be coming in descending alphabetical order based on gene name and it takes about 1 day to 1 1/2 days per letter of the alphabet, currently finishing up the 'F' series. The status page will show over 100% (probably 101% to 102%) by the time all the genes have been completed.

Retvari Zoltan
Send message
Joined: 31 Mar 20
Posts: 41
Credit: 50,629,651
RAC: 12,221
Hungary
Message 3120 - Posted: 30 Mar 2023, 6:53:33 UTC - in response to Message 3118.

It is looking like the original count of genes/isoforms was/is wrong. I believe we still have at least 5 days if not 7 more days of work left to crunch. The queued number seems to be the more accurate count.
Provided that this is right, there are 888 genes left ("queued"-"executed"=88374-87486). At the present rate of 90.40/day it will take 9.823 days (9 days, 19 hours, 45 minutes and 8 seconds) to complete. It will take much longer than that, given that the timeout is 5 days, and there will be some workunits stuck on inactive hosts. The work availability will be sporadic after 10 days from now.

The status page will show over 100% (probably 101% to 102%) by the time all the genes have been completed.
The percentage is the fraction of the "executed" and the "# genes / isoforms".
Now it's 87486 / 87554 = 0.999223336... = 99.92%
Provided that the "executed" will go over the "# genes / isoforms" to reach the "queued" it will be:
88374 / 87554 = 1.0093656486... = 100.94%

Technologov
Send message
Joined: 27 Jan 22
Posts: 36
Credit: 301,861,450
RAC: 2,941
Ukraine
Message 3121 - Posted: 30 Mar 2023, 9:51:28 UTC - in response to Message 3120.
Last modified: 30 Mar 2023, 9:54:08 UTC

One "gene" / "isoform" is how many work units ?
Also to collect the last few days of results, I recommend to up the parallel sending of work units from 2 to 3 workers, reducing the amount of stale work units, that get stuck on unresponsive hosts.

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 614
Credit: 34,508,279
RAC: 1,041
Italy
Message 3122 - Posted: 30 Mar 2023, 10:04:04 UTC - in response to Message 3120.

Yep, there is obviously something wrong in the table. Some genes, in the past, were "expanded" with different parameters (so there are duplicates in the gene list), other ones were expansions with no real gene inside (no gene seed, we call them "empties"). The overhead should be something around 1000.
Anyway the situation right now is that ALL the genes were put into the queue and yes, the order is lexicographical descending, the last one will be A2LD1

Retvari Zoltan
Send message
Joined: 31 Mar 20
Posts: 41
Credit: 50,629,651
RAC: 12,221
Hungary
Message 3123 - Posted: 30 Mar 2023, 11:14:28 UTC - in response to Message 3122.

The number of queued is rising (it's 88856 now), so we may reach 102% in the end :)

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 614
Credit: 34,508,279
RAC: 1,041
Italy
Message 3124 - Posted: 30 Mar 2023, 11:19:25 UTC - in response to Message 3123.

The number of queued is rising (it's 88856 now), so we may reach 102% in the end :)

Yes, this counts also the "other" genes, it would be rather complicated to fix it now, that 2% more should be considered as a "necessary overhead" ;)

Technologov
Send message
Joined: 27 Jan 22
Posts: 36
Credit: 301,861,450
RAC: 2,941
Ukraine
Message 3125 - Posted: 2 Apr 2023, 23:14:27 UTC

Can you accelerate "gene_work_generator" anyhow ?

The amount of tasks up for grabs is always zero, meaning that people have more workers than your TN-GRID servers can generate.

This is especially important now, where WCG project is down and Rosetta is out-of-work-units (again).

Bryn Mawr
Send message
Joined: 23 Jun 20
Posts: 43
Credit: 14,256,442
RAC: 188
United Kingdom
Message 3126 - Posted: 3 Apr 2023, 9:56:12 UTC - in response to Message 3125.

Can you accelerate "gene_work_generator" anyhow ?

The amount of tasks up for grabs is always zero, meaning that people have more workers than your TN-GRID servers can generate.

This is especially important now, where WCG project is down and Rosetta is out-of-work-units (again).


But that would just mean that TN-Grid would go dormant in (say) 8 days instead of 10.

Better the effort was put into bringing forward the next set of gene expansions than accelerating this set.

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 614
Credit: 34,508,279
RAC: 1,041
Italy
Message 3127 - Posted: 4 Apr 2023, 10:53:33 UTC - in response to Message 3125.
Last modified: 4 Apr 2023, 16:47:44 UTC

Can you accelerate "gene_work_generator" anyhow ?

The amount of tasks up for grabs is always zero, meaning that people have more workers than your TN-GRID servers can generate.

This is especially important now, where WCG project is down and Rosetta is out-of-work-units (again).

At this time, with our hardware resources, this is the best we can do...

arcturus
Send message
Joined: 18 May 22
Posts: 17
Credit: 5,806,368
RAC: 0
United States
Message 3128 - Posted: 4 Apr 2023, 19:32:23 UTC

So what's the situation? Can we expect a limited amount of new work from this point on? Or are we left to guess what's to become?

entity
Send message
Joined: 20 Jul 20
Posts: 20
Credit: 31,435,274
RAC: 17,248
United States
Message 3129 - Posted: 6 Apr 2023, 13:10:36 UTC

Looks like we will finish up this group of work in the next 12 to 24 hours. I hope Denis has a group to distribute while waiting on new work here.

Technologov
Send message
Joined: 27 Jan 22
Posts: 36
Credit: 301,861,450
RAC: 2,941
Ukraine
Message 3130 - Posted: 6 Apr 2023, 15:16:23 UTC - in response to Message 3129.

You be interested to know that I can't access Denis at Home from Ukraine, it's website is blocked

Previous · 1 · 2 · 3 · Next
Post to thread

Message boards : Number crunching : Curious


Main page · Your account · Message boards


Copyright © 2024 CNR-TN & UniTN