Posts by valterc
log in
1) Message boards : Number crunching : Server Status - No Work Available (Message 1217)
Posted 6 days ago by Profile valterc
Our work generator, for the current experiment, is capable to prepare a batch every 4.5 minutes, any batch is made by 97 workunits, times two for redundancy. The calculation is therefore (60/4.5)*97*2*24 = 62080, this is the maximum number of tasks that we can distribute daily. Other experiments has different numbers.

Another point is that any workunit, with the dataset and parameters of the current experiment, is relatively fast, ie. results come back fast. This got "worse" with the recent application update.
2) Message boards : News : New application (v 1.10) (Message 1214)
Posted 7 days ago by Profile valterc
Despite the shortage of wus, i'm crunching some 1.10 on my win10.
Also the win version has the same optimizations??

Same version (source code) for all the platforms.
3) Message boards : News : New application (v 1.10) (Message 1212)
Posted 8 days ago by Profile valterc
A new application version will be distributed soon, starting with the Linux platforms. The new application contains some major speed enhancements and bug fixes. Look at the following thread for further info http://gene.disi.unitn.it/test/forum_thread.php?id=135.
4) Message boards : Number crunching : No work for newly attached machines? (Message 1209)
Posted 12 days ago by Profile valterc
looking at the scheduler log I found a lot of lines like
Bad authenticator '1140_42646555bcaff2fadd
I really don't know what it is....

... But looking at the server source code (handle_request.cpp) I noticed that this is something related with the 'weak authentication key' (the one used by the gridcoin pools, I guess....)
5) Message boards : Number crunching : No work for newly attached machines? (Message 1208)
Posted 12 days ago by Profile valterc
One of my teammates is reporting:

"Having a heck of a time getting any work on the machines I added here, they request new tasks and event log just says the request was completed, got 0 new tasks. No error, no out of work, just nothing. Machines that were attached several weeks ago seem to work fine."

What's going on?

We didn't change anything recently. We still have the issue of the work generator not being able to make enough workunits to satisfy user's requests, so sometime there is no work available. Anyway, you may check the newest attached computers (hostid >= 19874) and see that they were able to download some work.

If you give me the hostid of one of the computers you are talking about I may check the server log and see if there is something strange.

[edit] looking at the scheduler log I found a lot of lines like
Bad authenticator '1140_42646555bcaff2fadd
I really don't know what it is....
6) Message boards : Wish List : Future requests (Message 1203)
Posted 19 days ago by Profile valterc
Francesco (one of our collaborators) is making it, he told me it would probably be ready at the beginning of the next week.

Any update on this? It's a shame to be unnecessarily wasting our CPU time.

I'm in contact with Daniel in order to solve some minor issues, please be patient....
7) Message boards : Wish List : Future requests (Message 1200)
Posted 27 days ago by Profile valterc
Francesco (one of our collaborators) is making it, he told me it would probably be ready at the beginning of the next week.
8) Message boards : Wish List : Future requests (Message 1196)
Posted 28 days ago by Profile valterc
Can we possibly make the latest apps from Daniel the official versions? Haven't they been out for about 6 months?

That's something I wanted to do after a system upgrade (operating system), I still haven't find the time (for instance, right now, we are focusing on the work generator) to do this... I'll try my best, maybe the next week...

Thanks, might bug you again next week :-)

Next week is here :-P

Thanks ;-). Just came to do the reminder too!

Yep, this is starting to become 'my apologies thread'.... Notice the 'maybe' in the former sentence ;) I'll wait until we have all the needed versions (the one missing is the one for MacOS)
9) Message boards : News : Another experiment on E. coli (Message 1190)
Posted 11 Nov 2017 by Profile valterc
2:15 h on my R7 1700. :)

better than average!
10) Message boards : News : Another experiment on E. coli (Message 1188)
Posted 10 Nov 2017 by Profile valterc
There is another experiment on Ec floating around, this one uses *very* computing intensive search parameters. An average workunit should last 3-6 hours. As usual we started just with a few batches, the next Monday, if everything goes well, we will put them all into the system.
One side effect of this is that the work generator should be able, during this experiment, to keep the queue full.
11) Message boards : Web site : Statistics export broken (Message 1187)
Posted 6 Nov 2017 by Profile valterc
The statistics XML export (for BoincStats and the like) seems to be broken. Last update is over 2 days ago, where the export used to be done twice a day. Can you look into this?

I also noticed this, now it seems working again, I checked boincstats. (btw I didn't do anything server-side....)
12) Message boards : Number crunching : IMPORTANT: App flagged as Ransomware (Message 1183)
Posted 2 Nov 2017 by Profile valterc
Hi,

The fma app has been flagged up as ransomware and removed by MalwareBytes.

I've now added an exclusion for the BOINC folder.

https://www.imageupload.co.uk/image/EKlb

I compiled it by myself from our source code, many months ago. I'm quite sure that it doesn't contain any malicious code. Anyway, a scan using VirusTotal gave the following results (https://www.virustotal.com/it/file/5e90d85043a707be0d38f657d703d09c90b63dd944b742820085220203873e5a/analysis/1509635813/)

SHA256: 5e90d85043a707be0d38f657d703d09c90b63dd944b742820085220203873e5a Nome del file: gene_pcim_v1.02_win64__fma.exe Rapporto rilevamento: 0 / 67 Data analisi: 2017-11-02 15:16:53 UTC ( 0 minuti fa )

Just (another) false positive, do not worry!
13) Message boards : Wish List : Future requests (Message 1182)
Posted 2 Nov 2017 by Profile valterc
Can we possibly make the latest apps from Daniel the official versions? Haven't they been out for about 6 months?

That's something I wanted to do after a system upgrade (operating system), I still haven't find the time (for instance, right now, we are focusing on the work generator) to do this... I'll try my best, maybe the next week...
14) Message boards : News : Another experiment on E. coli (Message 1178)
Posted 30 Oct 2017 by Profile valterc
But i continue to download and crunch Vitis Vinifera....

...as explained a few posts above... The E. coli experiment is almost finished (see the 'science' link from the home page)


Ok, thank you.

P.S. i thinked that the new experiment on e.coli could bring new work. My misunderstanding

Always waiting for Pseudomonas :-P

We are still doing experiments on Pa (in direct contact with some biologists here), but not using the BOINC system, A single gene expansion on this organism is *really* fast, a simple dedicated (not too powerful) workstation is enough for now.
15) Message boards : News : Another experiment on E. coli (Message 1177)
Posted 30 Oct 2017 by Profile valterc

Simply speaking the algorithm implemented here is able to find (causal) relations between variables (on various domains). We apply it on 'expression data' obtained from different organisms. A typical computational result is the information that gene X has some kind of relationship with gene Y (this knowledge will help biologists to better understand the biological processes associated with the specific genes). We apply the algorithm (with different parameters) to the organisms of interest of the biologists we are working with. If you want to know more about please look at the 'science' section of the forum.


Thanks. The openness and the friendly replies are exactly why I'm crunching for you. :)

One more thing, I've noticed that the current VV database has only 3611 genes to compute, right? Does that mean these are "cherry picked" genes that need further expansion?

Also, if that's the case, the science meter on the page is not working properly, as it reports 1212 genes as being 4.33%. Sorry if I'm being dumb here.

The number 3611 (DB column) that you see right now means that 3611 out of 28013 genes are inserted into our queuing system, ie. ready to be used for creating workunits. The other ones will be inserted later, when the need arises. We do this in order to keep our db small.
Maybe the word 'QUEUED' is more explanatory than 'DB' (I just changed it)
16) Message boards : News : Another experiment on E. coli (Message 1174)
Posted 30 Oct 2017 by Profile valterc
I've just added another experiment on Vitis vinifera (same dataset as the last one but different algorithm's parameter). This is computationally faster but the work generator will take the same time as before to create a work unit. This means that it would be difficult to build up a large queue of ready-to send workunits...


Could you please explain what is the difference here in more layan terms? I Think a lot of people would like to know exactly what's happening with our submitted results. :) I see that it's marked with the same algo as the E. Coli dataset, but I couldn't find where are the differences or what the end goal of them is.

Thanks!

Simply speaking the algorithm implemented here is able to find (causal) relations between variables (on various domains). We apply it on 'expression data' obtained from different organisms. A typical computational result is the information that gene X has some kind of relationship with gene Y (this knowledge will help biologists to better understand the biological processes associated with the specific genes). We apply the algorithm (with different parameters) to the organisms of interest of the biologists we are working with. If you want to know more about please look at the 'science' section of the forum.
17) Message boards : News : Another experiment on E. coli (Message 1173)
Posted 30 Oct 2017 by Profile valterc
But i continue to download and crunch Vitis Vinifera....

...as explained a few posts above... The E. coli experiment is almost finished (see the 'science' link from the home page)
18) Message boards : Number crunching : I am getting lots of computation errors (Message 1169)
Posted 27 Oct 2017 by Profile valterc
The 'computation error' problems should slowly be resolved (when all the bad workunits that were distributed error out).

The 'queue' problem is another issue (see http://gene.disi.unitn.it/test/forum_thread.php?id=195&postid=1146). Simply speaking the work generator is not able to satisfy all the users request (we are working on rewriting/optimizing it but it will take some time)

The speed of the work generator depends on the organism dataset (the number of genes, ~3000 Ec, ~28000 Vv), the time needed to complete a workunit depends on the 'structure' of the dataset (Ec is much more 'complicated' than Vv).
19) Message boards : Number crunching : I am getting lots of computation errors (Message 1165)
Posted 26 Oct 2017 by Profile valterc
Like 860 of them so far
http://gene.disi.unitn.it/test/result.php?resultid=22183628

Yes, I know, it was a bad batch (with a too low disk space limit for the output file), now I should have correct it, new replicas and new workunits should be okay. I really apologize for this.
20) Message boards : Number crunching : Error: disk limit exceeded (Message 1163)
Posted 26 Oct 2017 by Profile valterc
I've got today three units with this error with IDs out of the range you stated. Wingmen have completed them successfully in some cases so I'm not sure of the conclusion. In my host they have been processed with sse2 instructions and in the case of the successful completion with avx, do they generate different output file sizes?.

The content and the size of the output files should match regardless of the application version (the validation of a couple of results is a simple byte-by-byte comparison), so this shouldn't be the origin of the error. (I have to further investigate)


Next 20

Main page · Your account · Message boards


Copyright © 2017 CNR-TN & UniTN