Posts by paolomorettin
log in
1) Message boards : Science : Parameter tuning, phase 1 is over! (Message 404)
Posted 8 May 2014 by Profile paolomorettin
Hi everyone!
First of all, thank you for your support. As you probably know, our current goal is to find the best parameters for our algorithm and to discover meaningful relations between them. The first phase of this experiment is over and your efforts have paid off!
Here you can see the similarity in the results with respect to different parameters:



These heatmaps highlight a strong influence of the tile_size parameter on the similarity of the results. Also, there's a kind of 'stable' zone with tile_size >= 750. Without going into details, this is an important hint on how to split the computational work while ensuring consistent results. Our current research work is aimed at validating the results by comparing them with the latest biological knowledge.

If you have any question, feel free to ask!


Moreover, as Nadir stated on a different post (http://gene.disi.unitn.it/test/forum_thread.php?id=84#403) a new phase started, so please keep crunching numbers :)

Have a nice day!
2) Message boards : Unix/Linux : glibc version problem (Message 273)
Posted 8 Jan 2014 by Profile paolomorettin
Hi!
Finally I found some time to run some of your wus :)
All wus failed because a glibc version problem. Your tngrid_expansion_v3_linux64 executable need glibc 2.14 but on my computer I have 2.13. My system is the last stable debian (Version 7.3 (wheezy) 64 bit) with Kernel Linux 3.2.0-4-amd64.
I see that the binary is dynamically linked, so maybe if you build a static executable the problem could be solved.
Bye!


Hello and welcome! We will solve this issue as soon as possible, thank you!
3) Message boards : Development : Validation problems (roadmap) application+server (Message 266)
Posted 7 Jan 2014 by Profile paolomorettin
However, after the console request to delete the two files (just before the "query" line) the validator seems to hang for 4-5 seconds before outputting things again. What is it doing? (the overall check seems much slower than expected).


Is it the new validator?
4) Message boards : Development : Application group request (compact stderr) (Message 241)
Posted 3 Jan 2014 by Profile paolomorettin
Please try to make the stderr output more compact. From
#132 Tue Dec 24 02:05:01 2013
Graph created File: complete_gene_net read Standard deviations computed Correlations computed l: 0 1 2 3 4 5 6 7 8 9 # of arcs: 2223
Writing checkpoint's file
to something like
132|13-12-24 02:05:01|G F S|C(9:2223)
CHK
I mean, all the needed info squeezed in few bytes.


I did not work on that, but I think that the new version we already sent has a more compact stderr.

Also, do you need to read the complete_gene_net at every iteration? Shouldn't be more efficient to just read it at start or returning from checkpoint and keep it (or a copy of it) in memory?


When we developed the first versions of the application, we had no bounds on the complete_gene size, so the best strategy was to read on-the-fly the lines needed.
I think that we should discuss this with professor Blanzieri, reading it once should be faster, but I suspect that working with different species could lead to far bigger input files, compared to the Arabidopsis. Splitting those files may be a solution, though..
5) Message boards : Number crunching : Calculation error (Message 232)
Posted 3 Jan 2014 by Profile paolomorettin
Did you upgrade the .NET library recently? This error seems to be related to a broken system environment, caused by some missing or corrupted library files.

Another user had the same error few weeks ago, I'll ask him if he managed to solve this is some way.
6) Message boards : Number crunching : Calculation error (Message 231)
Posted 3 Jan 2014 by Profile paolomorettin
Today i've a lot of calculation error after few seconds, like this
38103

Risultato Errore di elaborazione
Stato client Errore di computazione
Stato di uscita -1073741515 (0xffffffffc0000135) Unknown error number


Thank you! I'll check this one :)
7) Message boards : News : New work available (intermittently) (Message 230)
Posted 3 Jan 2014 by Profile paolomorettin
The app running is version 2, Dec 22. If you have developed a new one you should probably contact the server group again (or install it by yourself...:)

BTW: I'm very curious about the changes you made to fix the 'won't stop' problem... any info about?


If you mean the suspension problem, I think it was already solved in the previous version. There was an implementation detail (rather counterintuitive, imho) that we weren't aware of. That is, a boinc_time_to_checkpoint call, if successful, opens a critical section (incrementing a counter). Even though we surrounded the checkpoint phase with critical sections, there was no boinc_checkpoint_completed. So at each successful checkpoint, the counter was increased twice and decreased once, stucking the application in a neverending critical section and not permitting the boinc client to suspend it.
8) Message boards : Number crunching : Disk space (Message 211)
Posted 27 Dec 2013 by Profile paolomorettin
Ah, i forget:
Congratulations to everyone for the great work you are doing!!


Thank you for your valuable advices! ;)
9) Message boards : Development : Meeting proposal - Dec 30, 15:00, at my office (Message 199)
Posted 27 Dec 2013 by Profile paolomorettin
Sorry, I will not be in Trento until 2014..
10) Message boards : Development : validation issues in 0.02 (Message 186)
Posted 25 Dec 2013 by Profile paolomorettin
For detail, take Wu 23056 as an example. We have three output results.


    1. Expansion_At2_work-1387833585.xml_pn19926_0_0 runs on GenuineIntel Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz [Family 6 Model 42 Stepping 7] with x86_64-pc-linux-gnu platform.

    2. Expansion_At2_work-1387833585.xml_pn19926_1_0 runs on GenuineIntel Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz [Family 6 Model 42 Stepping 7] with windows_x86_64 platform.

    3. Expansion_At2_work-1387833585.xml_pn19926_2_0 runs on AuthenticAMD AMD FX(tm)-6300 Six-Core Processor [Family 21 Model 2 Stepping 0] with windows_x86_64 platform



This first and the second run on same Intel core i7, different platform (Windows & Linux) and two results have small differences. The first one have 3 new edges: 252411_at,255604_at, 253467_at,262159_at, 253467_at,262159_at. While the second one have 1 new edge: 252002_at,253467_at

If we consider two results are all correct. Please give us a threshold of difference to judge. For example: If total of new edges < 1% total of edges, we will give both result credit.

The third is totally different. The size is much bigger (38 Mb vs 27 Mb), and have 661 lines in format "#... list_gene" compare to 500 lines of the first and the second.

Thanks and Merry Christmas.




IMHO the best idea is to develop a parametric validator and ask to Professor Blanzieri and to the post-processing team which threshold you should put in.

The third result is insanely wrong and surely caused by an application bug.
11) Message boards : Development : validation issues in 0.02 (Message 183)
Posted 25 Dec 2013 by Profile paolomorettin
Chau already sent to me the output files. I will investigate the differences as soon as possible.
12) Message boards : Development : validation issues in 0.02 (Message 175)
Posted 24 Dec 2013 by Profile paolomorettin
Hi,

The current validator is checking files by md5. That's why when you receive two files with different check_sum, they will mark as different. I'm improving it with removing \r (Carriage-Return) before comparing it. Thanks

Wait a sec: the md5sum validator is the one provided by boinc or is it something you have written?
Please mind that it seems there are still some slight difference among the WU computed by linux and the ones computed by windows, hence we should use something as the method discussed in one of the meeting with the professors. An md5um, instead, would keep failing


I agree, comparing output files by their md5s is even less informative than comparing them bitwise.

By the way, Paolo, when you output results to files, can you please take a look at the format of output? I mean, can you set the mode to "binary" instead of "text"? In windows, the new line is '\r\n' and in unix, the new line is '\n'. Thanks


I'm not sure I understood. What should we change in the syntax?
13) Message boards : Development : validation issues in 0.02 (Message 163)
Posted 24 Dec 2013 by Profile paolomorettin
Hi everyone, we are trying to get rid of the validation issues we're having with version 0.02. Is it possible to see the source code of the validator?
14) Message boards : Development : suspending/resuming/aborting the computation (Message 141)
Posted 21 Dec 2013 by Profile paolomorettin
Hi guys, I'm working on the issue pointed out by Valter here:

http://gene.disi.unitn.it/test/forum_thread.php?id=31#133

The function in boinc_api.cpp boinc_end_critical_section() used in our code actually checks for boinc_status.quit_request and boinc_status.abort_request, also if options.direct_process_action is true, it should suspend the work, by calling the boic_api.cpp function static int suspend_activities(bool called_from_worker).

What I got by reading the api source is that the option.direct_process_action flag tells boinc to handle the suspend requests. Calling boinc_init() hides a boinc_init_options(..) call with default values defined in boinc_api.h. That flag is true by default.

Also, another interesting flag is options.handles_process_control that enables the 'timer thread' to call handle_process_control_msg(). This function handles the status changes as the boinc_end_critical_section does. As for the previous one, also this flag is true by default.

I have no clue why it doesn't suspend.








15) Message boards : Development : Different results on the same WU (Message 108)
Posted 20 Dec 2013 by Profile paolomorettin
We found the bug, it was actually related to the checkpoint mechanism. We will fix it asap :)
16) Message boards : Number crunching : Weird redundancy (Message 99)
Posted 19 Dec 2013 by Profile paolomorettin
I made a thread on the development board:

http://gene.disi.unitn.it/test/forum_thread.php?id=27#98
17) Message boards : Development : Different results on the same WU (Message 98)
Posted 19 Dec 2013 by Profile paolomorettin
Ok so I checked the wu 21058, that were run 2 times on the same client (this is another issue that will not be discussed here) giving different results.

First I ran in standalone mode, getting the same results the client got once, then I found that the difference between the 2 client's outputs is that one of the two files have a single pcalg-run output repeated (the one starting in line 60584).

This is definitely an application issue, I guess is somewhat related to the checkpoint mechanism.
18) Message boards : Number crunching : Weird redundancy (Message 86)
Posted 19 Dec 2013 by Profile paolomorettin
we're checking it right now :)
19) Message boards : Web site : Problems... a lot of them! (Message 70)
Posted 18 Dec 2013 by Profile paolomorettin
In the home page:
- some images weight more than 5 megabytes.

In the client group page:
- the stretched images look ugly.. in particular Nadir and Luca are kinda scaring!




Main page · Your account · Message boards


Copyright © 2024 CNR-TN & UniTN