Posted 9 Dec 2018 by
Beyond
The output of the 'problematic' TCGA workunits were, in some cases, different if calculated on Windows or Linux. There are some validation errors because of this so I wrote down some code in order to give credits even for the TCGA invalids. See this workunit as an example: http://gene.disi.unitn.it/test/workunit.php?wuid=17810593
Very nice and much appreciated.
Posted 9 Dec 2018 by
Beyond
Here's the longest one I've seen yet:
http://gene.disi.unitn.it/test/workunit.php?wuid=17810256
I suspect the 4 "Timed out - no response" instances are still crunching away. My example is on a pretty slow machine. It's at 137 hours and 12%. It looks like these aren't checkpointing often. This WU is showing 117 hours since the last checkpoint (in BoincTasks).
Posted 1 Dec 2018 by
Beyond
I have a Ryzen 2700 that is only sent avx WUs. Everything is fine there. Recently I brought back my 2 Ryzen 1700 machines and after failing a few sse2 WUs I started aborting them. After a couple days they started getting only avx WUs (for a long time). Now all of a sudden they're being sent large numbers of sse2 WUs again. The sse2 WUs are unreliable and often fail after wasting a LOT of CPU time.
Could you PLEASE either give us a choice in preferences to receive only avx WUs or set the server to only send avx to Ryzen machines. As an aside there's obviously a flaw in either the sse2 programming or compiling as Ryzens run all other programs flawlessly.
The 3 Ryzens are flawless on avx:
gene@home PC-IM 1.11 windows_x86_64 (avx)
Consecutive valid tasks 1491
gene@home PC-IM 1.11 windows_x86_64 (avx)
Consecutive valid tasks 1440
gene@home PC-IM 1.11 windows_x86_64 (avx)
Consecutive valid tasks 1382
Posted 29 Nov 2018 by
Beyond
>> Have another one that's getting close to 75% and 7.6 hours on a very fast machine. That one's still running.
The one above finished in 9:10.
Now I have one on the same very fast machine that's run for 9:06 hours and is still at 0.5%.
Posted 29 Nov 2018 by
Beyond
Just had one that was stuck on 16+ hours and still at 17% on a relatively fast and previously reliable machine. Aborted it before I came over here to check...
Have another one that's getting close to 75% and 7.6 hours on a very fast machine. That one's still running.
Posted 27 Dec 2017 by
Beyond
Thanks for the new version!
Posted 30 Nov 2017 by
Beyond
One of my teammates is reporting:
"Having a heck of a time getting any work on the machines I added here, they request new tasks and event log just says the request was completed, got 0 new tasks. No error, no out of work, just nothing. Machines that were attached several weeks ago seem to work fine."
What's going on?
Posted 23 Nov 2017 by
Beyond
Francesco (one of our collaborators) is making it, he told me it would probably be ready at the beginning of the next week.
Any update on this? It's a shame to be unnecessarily wasting our CPU time.
Posted 17 Nov 2017 by
Beyond
Francesco (one of our collaborators) is making it, he told me it would probably be ready at the beginning of the next week.
Cool! Thanks, might bug you again next week :-)
Posted 16 Nov 2017 by
Beyond
Hi
any news regarding the Mac OS version ?
Any news yet? valterc is waiting for the Mac version before he makes the new optimized app the default (have no idea why that's a condition).
Posted 15 Nov 2017 by
Beyond
Can we possibly make the latest apps from Daniel the official versions? Haven't they been out for about 6 months?
That's something I wanted to do after a system upgrade (operating system), I still haven't find the time (for instance, right now, we are focusing on the work generator) to do this... I'll try my best, maybe the next week...
Thanks, might bug you again next week :-)
Next week is here :-P
Thanks ;-). Just came to do the reminder too!
Yep, this is starting to become 'my apologies thread'.... Notice the 'maybe' in the former sentence ;) I'll wait until we have all the needed versions (the one missing is the one for MacOS)
>> I'll wait until we have all the needed versions (the one missing is the one for MacOS)
Daniel?
Posted 14 Nov 2017 by
Beyond
Can we possibly make the latest apps from Daniel the official versions? Haven't they been out for about 6 months?
That's something I wanted to do after a system upgrade (operating system), I still haven't find the time (for instance, right now, we are focusing on the work generator) to do this... I'll try my best, maybe the next week...
Thanks, might bug you again next week :-)
Next week is here :-P
Thanks ;-). Just came to do the reminder too!
Posted 2 Nov 2017 by
Beyond
Can we possibly make the latest apps from Daniel the official versions? Haven't they been out for about 6 months?
That's something I wanted to do after a system upgrade (operating system), I still haven't find the time (for instance, right now, we are focusing on the work generator) to do this... I'll try my best, maybe the next week...
Thanks, might bug you again next week :-)
Posted 1 Nov 2017 by
Beyond
Can we possibly make the latest apps from Daniel the official versions? Haven't they been out for about 6 months?
Posted 8 Feb 2017 by
Beyond
The Ec workunits were easy to manage: relatively long computation time and small output file, the opposite happens with Vv. Ec (Escherichia coli) has only ~4000 genes while Vv (Vitis Vinifera) more than 28000. So we are trying to balance this in order to achieve the optimal workunit size. (the upload error was obviously not expected, our fault, sorry).
Could you make a choice in project preferences to choose the WU type? Some of us (read: 1/2 of the USA) have stone age broadband (a misnomer) speeds and can't handle these large uploads for short runtime WUs. I've had to cut way back because of this.
Posted 25 Jan 2017 by
Beyond
Impressive work. Thanks! It looks like the sse2+fma is only 0.659% faster. Is that even worth having another version?
I am going to modify code a bit to use AVX for div/sqrt calculations and SSE for the rest. This should improve performance a bit, so finally it should be a bit faster than this SSE+FMA version. Will see when I will have it ready how much it is faster. Also keep in mind that WUs sent by server now are 100 times longer and we can expect that they will be 200 times longer, so actual time reduction per WU will not be so tiny.
Looking at the user reporting for his AMD X8, his results show that the new fma app is actually running around 11% faster than the sse2 version. This is also what I'm seeing on my four AMD X8 CPUs. A useful increase. Once again, THANKS!
Posted 24 Jan 2017 by
Beyond
I found Haswell Xeon machine where I could perform some tests. Here are results of running app on test data, averaged over 10 runs:
SSE2 20.600 SSE2+FMA 20.465
Impressive work. Thanks! It looks like the sse2+fma is only 0.659% faster. Is that even worth having another version?
Posted 24 Jan 2017 by
Beyond
This app was limited by memory speed, so SSE version may be faster. Older version was executing more slow calculations (square roots, divisions) plus loops for AVX were executing less times because of longer vectors, so AVX and FMA were faster. Now with reduced number of these slow calculations and with unrolled loops it may be that SSE is faster. I could test it on SandyBridge CPUs only which have slow AVX division and square roots, so AVX app was slower there too. On newer CPUs with faster AVX and memory things may work differently and AVX/FMA versions may be faster. Will see, I hope people will post their results for new apps here.
I am seeing a little less than 34 min/workunit on win 7 64 with the newest AVX version. On an i3-4330 (Haswell) running two instances
Would it be possible to test the newest sse2 app on your machine? I bet it's faster.
Posted 23 Jan 2017 by
Beyond
The new sse2 app is faster than the fma app on this machine (AMD FX-8320E), a reversal from the earlier optimized app.
Newest optimized sse2: 51:08 to 53:32
Newest optimized fma: 56:10 to 58:12
The sse2 app is about 9% faster on this CPU, while the older fma app was faster. Again, strange...
This app was limited by memory speed, so SSE version may be faster. Older version was executing more slow calculations (square roots, divisions) plus loops for AVX were executing less times because of longer vectors, so AVX and FMA were faster. Now with reduced number of these slow calculations and with unrolled loops it may be that SSE is faster. I could test it on SandyBridge CPUs only which have slow AVX division and square roots, so AVX app was slower there too. On newer CPUs with faster AVX and memory things may work differently and AVX/FMA versions may be faster. Will see, I hope people will post their results for new apps here.
Thanks for the great explanation about what's probably going on. I was scratching my head over this one and it was starting to hurt. ;-)
Posted 23 Jan 2017 by
Beyond
I'm draining the queue on one of my AMD FX-8320E machines now. Will install the fixed fma version and test.
So far the new fma version is running but seems to be slower on this CPU than the sse2 app.
On the previous optimized app the fma was a little faster than the sse2 on this CPU. Strange.
Edit: The new sse2 app is faster than the fma app on this machine, a reversal from the earlier optimized app.
Newest optimized sse2: 51:08 to 53:32
Newest optimized fma: 56:10 to 58:12
The sse2 app is about 9% faster on this CPU, while the older fma app was faster. Again, strange...