| log in |
|
1)
Message boards :
Number crunching :
World Gene Expansion Challenge 2021
(Message 2261)
Posted 10 Mar 2021 by xii5ku Michael H.W. Weber wrote: It would be nice if you could ensure that the server is loaded with tasks when announcing a challenge. Through the last several years at least, almost all contests at TN-Grid worked like this. It is well known and documented in several forum threads that the rate of work generation is very limited. What we don't know (yet) is whether or not a large number of WUs could be generated in advance during a period of low demand. (E.g. if new WUs depend on results of old WUs, then this might not be possible.) The project admin could certainly answer this. But if work generation in advance would be possible, then the immediate next question would be how many to produce in advance. This would be impossible to answer well, because (1.) participation in past contests changed from one to another obviously, and (2.) in those contests with good participation, the available computer capacity cannot be determined in retrospect because total contest throughput was capped by work generator throughput. |
|
2)
Message boards :
Number crunching :
World Gene Expansion Challenge 2021
(Message 2248)
Posted 8 Mar 2021 by xii5ku xii5ku wrote: Bunkering is not the reason for work scarcity. Aurum wrote: If you say so, but then why have my Validation Pending WUs grown to 6600??? I didn't say that there was no bunkering going on. I said that the reason for work scarcity is (in other words) that work requests are more frequent than WU generation. BTW, tasks in progress normally at TN-Grid: 25...35 k during this month, 25...55 k during the past several months, right before the start of the current challenge: < 80 k, right now: 53 k These less than 80 k in progress do not look like a massive bunkering effort to me. Bunkering would contribute to work scarcity if the bunker builders would issue frequent work requests over a prolonged time, and it would cause work scarcity if the number of tasks in progress would inflate the database so much that the work generator stopped producing. Buro87 wrote: For those with jobs that would go beyond the deadline, not already started,it is preferable to abort them BEFORE deadline. The last part is not correct. TN-Grid's workunits are configured with . . . . minimum quorum = 2 . . . . initial replication = 2 . . . . max # of error tasks = 8 . . . . max # of total tasks = 16 . . . . max # of success tasks = 8 which among else means that a workunit fails only if there were 8 tasks ending in an error (which includes timeout, among other error types) or if there were 16 tasks created but still no two correct result reports coming in before the tasks' respective reporting deadline. (This 16 tasks limit can never be reached in practice, and the 8 error tasks limit is extremely unlikely to happen.) As long as there failed less than 8 tasks, a failed (e.g. timed out) task will merely cause the server to make another task out of the same already existing workunit. Buro87 wrote: Another thing: he set for the first time, a "grace period" of 8hours after deadline, within which, THEORETICALLY, wus will be considered valid. At TN-Grid, there is no difference between the reporting deadline listed at the website (which is what the server should take as deadline) and the reporting deadline listed locally on the hosts (which the clients were told to obey as deadline). However: When a host "Y" does not report a result at the respective task's reporting deadline, the server replicates another task from the same workunit and assigns this task to another host "Z" at the next opportunity. If host "Y" then reports a valid result before host "Z" does, host "Y" gets credit for this result despite the passed deadline, and the server notifies host "Z" that his task is no longer needed. This notification will be delivered at the next point in time when host "Z" happens to emit a scheduler request. If host "Z" receives this notification before it started the now superfluous task, it marks it as 'error/ cancelled by server' and reports it. But if host "Z" already started the task before this notification, it completes the task as if nothing happened, reports the result to the server, and will get credit if the result is valid, even though the corresponding task was not needed anymore for the server to finish the workunit. (The above assumes that there was already another host "X" reporting a valid result in time, such that either X&Y or X&Z would satisfy the minimum quorum of 2.) To summarize, there are regular situations in which a result is validated and gets credit even if the reporting deadline was exceeded. (This is my understanding; somebody correct me if I got something wrong.) |
|
3)
Message boards :
Number crunching :
World Gene Expansion Challenge 2021
(Message 2240)
Posted 7 Mar 2021 by xii5ku Magiceye04 wrote: Will there be some WUs ready to be send tomorrow? Maybe. But only if there are enough people who give up and stop requesting work. The pace of TN-Grid's work generator is limited. Aurum wrote: Just Say No to Bunkering. The limit is 8 tasks in progress per active CPU AFAIK, counting up to 64 CPUs per host. This is already a low limit. Bunkering is not the reason for work scarcity. |
|
4)
Message boards :
Number crunching :
Formula Boinc Sprint 2019
(Message 1613)
Posted 20 Sep 2019 by xii5ku Links: 2019 sprint results (to be finalized on Sunday, 16:00 UTC) 2018 sprint results, forum thread 2017 sprint results, forum thread |
|
5)
Message boards :
Number crunching :
No tasks
(Message 1365)
Posted 16 Sep 2018 by xii5ku PS, message 1362 is not a complaint, merely a correction of what I wrote in message 1356. I understood that WU generation for gene@home is not trivial and cannot be increased by the flip of a switch. |
|
6)
Message boards :
Number crunching :
No tasks
(Message 1364)
Posted 16 Sep 2018 by xii5ku [H]auntjemima, until Saturday 18:00 UTC, server status always showed "tasks ready to send" to be zero, or almost zero. During this time, those who updated their clients more frequently than it would update by itself, got enough tasks to keep their hosts busy. Those who let their clients run freely, received almost no tasks. (Over the course of 12 hours from Saturday 18:00 UTC on, "tasks ready to send" slowly climbed up and are at practically normal level now. This is of course because the sprint ends at Sunday 16:00 UTC, and TN-Grid's quota of 8 tasks in progress per logical CPU gives about a day worth of tasks -- less on quicker hosts, more on slower hosts. Meaning, those who participate in this sprint but will switch elsewhere after the sprint stopped requesting new tasks about a day before the sprint ends.) My guess that the clients which remained idle make up roughly half of the entire computing capacity that was intended to run TN-Grid this weekend (for the sprint or otherwise) comes from the observation of how many points the various teams have by now, how points were distributed in sprints in which there was no scarcity of work, and from reading at a few forums. -------- Edit: It is a fact that "tasks ready to send" were at or very near zero constantly from Wednesday 16:00 UTC until some time after Saturday 18:00 UTC. (In addition, "tasks in progress" climbed from 35,000 to 65,000 between announcement and start of the sprint, and further to a peak of 79,000 during the sprint. It is now coming down slowly.) So, it is also a fact that demand for new work was higher than supply of new work, constantly from the announcement of the sprint until less than a day before the end of the sprint. IOW, there were hosts without enough work, as a matter of fact. The only point to disagree on is how many hosts had not enough work; this one point was really just guesswork on my part. But it were certainly many. -------- Edit 2: I disagree on the other hand that bunkering was impossible. The slowness of the work generator does not prevent bunkering. It only prevents some bunkering methods from being effective, while other bunkering methods still work. (Bunkering = receiving new work but not reporting completed work; you can control the latter, only the former depends on the server having work available.) |
|
7)
Message boards :
Number crunching :
No tasks
(Message 1362)
Posted 15 Sep 2018 by xii5ku xii5ku wrote:
@valterc, at this point it looks like you meant 488 new tasks, not WUs, per 15-18 minutes. ~150 credits/task * 488 tasks / 15...18 minutes * 1440 minutes/day = ~6...7 M credits/day This aligns with the total production of the project today and yesterday, as seen on free-dc or boincstats. Unfortunately this means, as a very rough guesstimation, that perhaps 1/2 of the computer capacity which volunteers would like to put to task during this sprint remains without work. The number of tasks ready to send remain near zero. IOW even though less than a day remains in this sprint, there is still more demand for new work than there is supply. |
|
8)
Message boards :
Number crunching :
No tasks
(Message 1360)
Posted 14 Sep 2018 by xii5ku Number of tasks in progress: - was constantly at 30,000 for many days until Tuesday - went to 35,000 on Wednesday for no apparent reason - steadily climbed to almost 70,000 since the sprint announcement until now, and is still rising |
|
9)
Message boards :
Number crunching :
No tasks
(Message 1358)
Posted 14 Sep 2018 by xii5ku @Marko Sango, the project which is chosen for a Formula BOINC sprint is announced exactly 24 hours before the start of each sprint. (Except when Sebastien forgets to update the site.) A reminder at when exact date and time the next sprint project will be announced is appearing at the Formula BOINC site a week in advance of each sprint. |
|
10)
Message boards :
Number crunching :
No tasks
(Message 1356)
Posted 13 Sep 2018 by xii5ku valterc wrote: For your reference the current pace of the work generator is actually around 488 new workunits every 15-18 minutes. ~145 credits/task * 2 tasks/WU * 488 WUs / ~18 minutes * 1440 minutes/day = ~11 M credits/day This is looking good! The project is seeing ~3 M credits/day normally. |
|
11)
Message boards :
Number crunching :
No tasks
(Message 1352)
Posted 12 Sep 2018 by xii5ku Orange Kid wrote: The server is out of work and a Formula Boinc sprint is in progress. Link to Formula Boinc sprint page: http://formula-boinc.org/sprint.py?sprint=15&year=2018 H|Skillz wrote: We need tasks! As far as I can tell, recent Formula Boinc sprints cause about more than double (quite possibly triple) the production compared to normal days at projects which (a) are whitelisted by Gridcoin = already have a relatively high daily baseline of participation and (b) are not constrained by work generator pace or validator problems or bandwidth or whatnot. TN-Grid is whitelisted by Gridcoin, but it has a notoriously low rate of work generation. Long story short: "We need tasks!" -> Good luck with that. :-( Edit, quoting from post 1133: On 19 October 2017 valterc wrote: The way we produce work is rather complicated (computationally speaking), at the moment this is the maximum pace we can achieve. |
|
12)
Message boards :
News :
Another experiment on E. coli
(Message 1148)
Posted 23 Oct 2017 by xii5ku Felix, work is being generated, but it is downloaded immediately by hosts which are asking for work often enough. Hence the server status page shows 0 tasks ready to send practically all the time. In other words, computer capacity available from contributors currently exceeds the pace of the work generator. |
|
13)
Message boards :
Number crunching :
Formula Boinc sprint, October 20-23
(Message 1143)
Posted 23 Oct 2017 by xii5ku As far as I observed, work fetching was indeed limited by the pace of the work generator during the day before the actual start of the sprint and during maybe half a day after the start, due to sprint participants wanting to load up with work. Nevertheless I managed to use a large part of this period for actual crunching with a bit of checking back on my hosts. But after that, there was a plentiful supply of work. Also, result uploads etc. worked without a hitch. So, thanks to the TN-Grid team for AFAICT a well working project infrastructure. |
|
14)
Message boards :
Number crunching :
Formula Boinc sprint, October 20-23
(Message 1132)
Posted 19 Oct 2017 by xii5ku Hi, just a heads-up that TN-Grid has been selected as a sprint project for the next 3 days. http://formula-boinc.org/sprint.py?sprint=17&year=2017 Not sure if Formula Boinc admins routinely keep project admins informed about their project choices and schedule, so at least now you know. :-) There seems to be a high rate of requests for work to your server now... |