Another experiment on E. coli
log in

Advanced search

Message boards : News : Another experiment on E. coli

1 · 2 · Next
Author Message
Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1127 - Posted: 18 Oct 2017, 9:53:02 UTC
Last modified: 18 Oct 2017, 12:50:03 UTC

We are ready to start another experiment on Escherichia coli, using the same expression dataset of the previous one but changing one of the algorithm's parameters.
Execution times should slightly faster than the last one on Vitis vinifera.

PC Gaming Master Race
Send message
Joined: 26 Jul 17
Posts: 6
Credit: 13,629
RAC: 0
Italy
Message 1128 - Posted: 18 Oct 2017, 14:25:29 UTC

Confirming I received them, they are much quicker than the Vitis vinifera WUs.
Will the science page on the homepage updated with the % of work done?

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1130 - Posted: 18 Oct 2017, 14:39:00 UTC - in response to Message 1128.

Confirming I received them, they are much quicker than the Vitis vinifera WUs.
Will the science page on the homepage updated with the % of work done?

We need some time in order to find the 'optimal' generation parameters (the target execution time should be around one hour for average cpus), I will update the science page in the next days.

Col323
Send message
Joined: 23 Nov 16
Posts: 6
Credit: 1,095,813
RAC: 712
Angola
Message 1136 - Posted: 19 Oct 2017, 15:34:42 UTC

Thanks for the new "science meter". I really appreciate that you created a separate meter for the new experiment instead of lumping it all into one and recalculating the percent complete based on total work done. With separate meters, it feels like we are going forward. Recalculating the percentage would feel like we're going backward. Plus, we can track the progress of each experiment individually.

Keep up the good work!

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1146 - Posted: 23 Oct 2017, 16:40:20 UTC - in response to Message 1136.

I've just added another experiment on Vitis vinifera (same dataset as the last one but different algorithm's parameter). This is computationally faster but the work generator will take the same time as before to create a work unit. This means that it would be difficult to build up a large queue of ready-to send workunits...

Felix_M_
Send message
Joined: 18 Oct 17
Posts: 3
Credit: 25,793
RAC: 57
Germany
Message 1147 - Posted: 23 Oct 2017, 18:24:58 UTC

there is nothing to sent when i look at serverstatus

xii5ku
Send message
Joined: 20 Jul 17
Posts: 3
Credit: 2,864,642
RAC: 622
Message 1148 - Posted: 23 Oct 2017, 20:04:45 UTC - in response to Message 1147.

Felix, work is being generated, but it is downloaded immediately by hosts which are asking for work often enough. Hence the server status page shows 0 tasks ready to send practically all the time.

In other words, computer capacity available from contributors currently exceeds the pace of the work generator.

Felix_M_
Send message
Joined: 18 Oct 17
Posts: 3
Credit: 25,793
RAC: 57
Germany
Message 1150 - Posted: 24 Oct 2017, 4:45:29 UTC

oh okay thank you

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1151 - Posted: 24 Oct 2017, 9:11:41 UTC - in response to Message 1150.

At this moment we are able to create 97*2 workunits every ~4 minutes (2910 per hour, 69840 per day). The average computing time is around one hour.

Felix_M_
Send message
Joined: 18 Oct 17
Posts: 3
Credit: 25,793
RAC: 57
Germany
Message 1152 - Posted: 24 Oct 2017, 14:01:13 UTC

well...
i would say thats to less new created work
can you do sth about this?

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 80
Credit: 2,202,886
RAC: 0
Poland
Message 1153 - Posted: 24 Oct 2017, 14:25:30 UTC

Could I take a look on code of your work generator and some sample input data? I wonder if I could optimize it a bit.
____________

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1154 - Posted: 24 Oct 2017, 15:43:39 UTC - in response to Message 1153.

Could I take a look on code of your work generator and some sample input data? I wonder if I could optimize it a bit.

Thank you for this, I will contact Francesco (who is the main author of the program) and let you know about his comments. Beware that the core of the program is written in Python, I'm planning to rewrite it in C++.

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 80
Credit: 2,202,886
RAC: 0
Poland
Message 1155 - Posted: 24 Oct 2017, 16:22:59 UTC - in response to Message 1154.

Could I take a look on code of your work generator and some sample input data? I wonder if I could optimize it a bit.

Thank you for this, I will contact Francesco (who is the main author of the program) and let you know about his comments. Beware that the core of the program is written in Python, I'm planning to rewrite it in C++.

No problem, I know Python too :)

BTW, have you tried to use PyPy (https://pypy.org/) or something like this?
____________

Aladar
Send message
Joined: 28 Oct 17
Posts: 3
Credit: 0
RAC: 0
Message 1171 - Posted: 28 Oct 2017, 16:25:08 UTC - in response to Message 1146.

I've just added another experiment on Vitis vinifera (same dataset as the last one but different algorithm's parameter). This is computationally faster but the work generator will take the same time as before to create a work unit. This means that it would be difficult to build up a large queue of ready-to send workunits...


Could you please explain what is the difference here in more layan terms? I Think a lot of people would like to know exactly what's happening with our submitted results. :) I see that it's marked with the same algo as the E. Coli dataset, but I couldn't find where are the differences or what the end goal of them is.

Thanks!

Profile [VENETO] boboviz
Send message
Joined: 12 Dec 13
Posts: 130
Credit: 910,431
RAC: 1,432
Italy
Message 1172 - Posted: 30 Oct 2017, 10:29:07 UTC

But i continue to download and crunch Vitis Vinifera....

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1173 - Posted: 30 Oct 2017, 14:32:41 UTC - in response to Message 1172.
Last modified: 30 Oct 2017, 14:33:01 UTC

But i continue to download and crunch Vitis Vinifera....

...as explained a few posts above... The E. coli experiment is almost finished (see the 'science' link from the home page)

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1174 - Posted: 30 Oct 2017, 14:48:52 UTC - in response to Message 1171.

I've just added another experiment on Vitis vinifera (same dataset as the last one but different algorithm's parameter). This is computationally faster but the work generator will take the same time as before to create a work unit. This means that it would be difficult to build up a large queue of ready-to send workunits...


Could you please explain what is the difference here in more layan terms? I Think a lot of people would like to know exactly what's happening with our submitted results. :) I see that it's marked with the same algo as the E. Coli dataset, but I couldn't find where are the differences or what the end goal of them is.

Thanks!

Simply speaking the algorithm implemented here is able to find (causal) relations between variables (on various domains). We apply it on 'expression data' obtained from different organisms. A typical computational result is the information that gene X has some kind of relationship with gene Y (this knowledge will help biologists to better understand the biological processes associated with the specific genes). We apply the algorithm (with different parameters) to the organisms of interest of the biologists we are working with. If you want to know more about please look at the 'science' section of the forum.

Profile [VENETO] boboviz
Send message
Joined: 12 Dec 13
Posts: 130
Credit: 910,431
RAC: 1,432
Italy
Message 1175 - Posted: 30 Oct 2017, 16:49:20 UTC - in response to Message 1173.
Last modified: 30 Oct 2017, 16:49:56 UTC

But i continue to download and crunch Vitis Vinifera....

...as explained a few posts above... The E. coli experiment is almost finished (see the 'science' link from the home page)


Ok, thank you.

P.S. i thinked that the new experiment on e.coli could bring new work. My misunderstanding

Always waiting for Pseudomonas :-P

Aladar
Send message
Joined: 28 Oct 17
Posts: 3
Credit: 0
RAC: 0
Message 1176 - Posted: 30 Oct 2017, 17:45:51 UTC - in response to Message 1174.


Simply speaking the algorithm implemented here is able to find (causal) relations between variables (on various domains). We apply it on 'expression data' obtained from different organisms. A typical computational result is the information that gene X has some kind of relationship with gene Y (this knowledge will help biologists to better understand the biological processes associated with the specific genes). We apply the algorithm (with different parameters) to the organisms of interest of the biologists we are working with. If you want to know more about please look at the 'science' section of the forum.


Thanks. The openness and the friendly replies are exactly why I'm crunching for you. :)

One more thing, I've noticed that the current VV database has only 3611 genes to compute, right? Does that mean these are "cherry picked" genes that need further expansion?

Also, if that's the case, the science meter on the page is not working properly, as it reports 1212 genes as being 4.33%. Sorry if I'm being dumb here.

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,279,861
RAC: 3,910
Italy
Message 1177 - Posted: 30 Oct 2017, 18:06:06 UTC - in response to Message 1176.
Last modified: 30 Oct 2017, 18:09:43 UTC


Simply speaking the algorithm implemented here is able to find (causal) relations between variables (on various domains). We apply it on 'expression data' obtained from different organisms. A typical computational result is the information that gene X has some kind of relationship with gene Y (this knowledge will help biologists to better understand the biological processes associated with the specific genes). We apply the algorithm (with different parameters) to the organisms of interest of the biologists we are working with. If you want to know more about please look at the 'science' section of the forum.


Thanks. The openness and the friendly replies are exactly why I'm crunching for you. :)

One more thing, I've noticed that the current VV database has only 3611 genes to compute, right? Does that mean these are "cherry picked" genes that need further expansion?

Also, if that's the case, the science meter on the page is not working properly, as it reports 1212 genes as being 4.33%. Sorry if I'm being dumb here.

The number 3611 (DB column) that you see right now means that 3611 out of 28013 genes are inserted into our queuing system, ie. ready to be used for creating workunits. The other ones will be inserted later, when the need arises. We do this in order to keep our db small.
Maybe the word 'QUEUED' is more explanatory than 'DB' (I just changed it)

1 · 2 · Next
Post to thread

Message boards : News : Another experiment on E. coli


Main page · Your account · Message boards


Copyright © 2017 CNR-TN & UniTN