Optimization
log in

Advanced search

Message boards : Number crunching : Optimization

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 10 · Next
Author Message
Profile Beyond
Avatar
Send message
Joined: 2 Nov 16
Posts: 24
Credit: 5,400,568
RAC: 24,487
United States
Message 746 - Posted: 13 Jan 2017, 16:19:04 UTC - in response to Message 740.

- stats on downloads page shows that 32-bit windows non-SSE version of my app was downloaded 12 times, so there is some need for it.

All that means is that 12 people might have tried it, not that anyone uses it. Some download all the versions and try them, or just DL them all in case they might want to later. I've tried all the 64bit versions on various CPUs but only the sse2 and fma were useful and even the difference between those was small.
BTW, THANKS MUCH for all the hard work!

Profile Beyond
Avatar
Send message
Joined: 2 Nov 16
Posts: 24
Credit: 5,400,568
RAC: 24,487
United States
Message 747 - Posted: 13 Jan 2017, 16:20:07 UTC - in response to Message 745.

So, with sse2, avx, fma, any modern computer will get the three applications and eventually decide which one is the best one...

Potential problem: some machines error the WUs with avx and fma while sse2 seems to work with everthing I've tried.
I've found that fma can be slightly faster than the sse2 version on some machines but the difference is small.

What OS do you use? AVX needs support on OS side too. List of supported OSes is here: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Operating_system_support

All machines are Win7-64.

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 79
Credit: 2,202,886
RAC: 0
Poland
Message 748 - Posted: 13 Jan 2017, 16:21:53 UTC - in response to Message 747.

So, with sse2, avx, fma, any modern computer will get the three applications and eventually decide which one is the best one...

Potential problem: some machines error the WUs with avx and fma while sse2 seems to work with everthing I've tried.
I've found that fma can be slightly faster than the sse2 version on some machines but the difference is small.

What OS do you use? AVX needs support on OS side too. List of supported OSes is here: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Operating_system_support

All machines are Win7-64.

Do you have SP1 installed? AVX support was added in it.
____________

Profile Beyond
Avatar
Send message
Joined: 2 Nov 16
Posts: 24
Credit: 5,400,568
RAC: 24,487
United States
Message 750 - Posted: 13 Jan 2017, 16:26:49 UTC - in response to Message 748.

So, with sse2, avx, fma, any modern computer will get the three applications and eventually decide which one is the best one...

Potential problem: some machines error the WUs with avx and fma while sse2 seems to work with everthing I've tried.
I've found that fma can be slightly faster than the sse2 version on some machines but the difference is small.

What OS do you use? AVX needs support on OS side too. List of supported OSes is here: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Operating_system_support

All machines are Win7-64.

Do you have SP1 installed? AVX support was added in it.

Yes and all patches and updates.

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 79
Credit: 2,202,886
RAC: 0
Poland
Message 751 - Posted: 13 Jan 2017, 16:42:43 UTC - in response to Message 750.

So, with sse2, avx, fma, any modern computer will get the three applications and eventually decide which one is the best one...

Potential problem: some machines error the WUs with avx and fma while sse2 seems to work with everthing I've tried.
I've found that fma can be slightly faster than the sse2 version on some machines but the difference is small.

What OS do you use? AVX needs support on OS side too. List of supported OSes is here: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Operating_system_support

All machines are Win7-64.

Do you have SP1 installed? AVX support was added in it.

Yes and all patches and updates.

I checked your computers and found few validations errors, but no error reported by app or BOINC client that something was wrong. Maybe these are caused by that mysterious error mentioned by valterc before? I also saw this few times but on SSE version. That was on machine running 24/7 and BOINC configured with long task switch time, so these WUs were crunched from start to end without interruptions. One of them was also crunched by someone with my app and that for that person it was validated successfully, so it is even more interesting.
____________

Profile Beyond
Avatar
Send message
Joined: 2 Nov 16
Posts: 24
Credit: 5,400,568
RAC: 24,487
United States
Message 752 - Posted: 13 Jan 2017, 17:35:18 UTC - in response to Message 751.

I checked your computers and found few validations errors, but no error reported by app or BOINC client that something was wrong. Maybe these are caused by that mysterious error mentioned by valterc before? I also saw this few times but on SSE version. That was on machine running 24/7 and BOINC configured with long task switch time, so these WUs were crunched from start to end without interruptions. One of them was also crunched by someone with my app and that for that person it was validated successfully, so it is even more interesting.

The only 3 validation errors I had were all at once on 1 machine when there was a power glitch. The problem I had with the non sse2 versions were on certain CPU types. They errored all WUs immediately, but it was a while ago when I was testing and the WUs are no longer in the database. The sse2 version runs fine on every machine I've tried so far (15-20 boxes for myself and friends).

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 79
Credit: 2,202,886
RAC: 0
Poland
Message 753 - Posted: 13 Jan 2017, 17:50:07 UTC - in response to Message 752.
Last modified: 13 Jan 2017, 17:50:25 UTC

I checked your computers and found few validations errors, but no error reported by app or BOINC client that something was wrong. Maybe these are caused by that mysterious error mentioned by valterc before? I also saw this few times but on SSE version. That was on machine running 24/7 and BOINC configured with long task switch time, so these WUs were crunched from start to end without interruptions. One of them was also crunched by someone with my app and that for that person it was validated successfully, so it is even more interesting.

The only 3 validation errors I had were all at once on 1 machine when there was a power glitch. The problem I had with the non sse2 versions were on certain CPU types. They errored all WUs immediately, but it was a while ago when I was testing and the WUs are no longer in the database. The sse2 version runs fine on every machine I've tried so far (15-20 boxes for myself and friends).

Yeah, power outage is a problem for many BOINC apps, usually they assume they will be able to always write checkpoint successfully, and do not take into account that this may be interrupted by power outage or another sudden app termination.

You mentioned errors with AVX app. Are they problems with starting/running app, or validation errors? If with starting/running, could you provide me link to example failed WU? I would like to check if there are some details which may be helpful.
____________

[AF>Le_Pommier] Jerome_C2005
Send message
Joined: 12 May 14
Posts: 4
Credit: 176,072
RAC: 1
Mexico
Message 754 - Posted: 13 Jan 2017, 23:19:54 UTC

Hi

you mention the deployment of the new version 0.10 of the application "for all platforms" (this says the news in the boinc manager) however Mac OS is still the old version.

Is this temporary or won't you consider Mac OS (X/macOS) ?


On my Mac I have

Ven 13 jan 23:56:51 2017 | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clfsh ds acpi mmx fxsr sse sse2 ss htt tm pbe pni dtes64 mon dscpl vmx smx est tm2 ssse3 cx16 tpr pdcm sse4_1 sse4_2 popcnt


Would it be compatible with the new version ?

Thanks

Profile Beyond
Avatar
Send message
Joined: 2 Nov 16
Posts: 24
Credit: 5,400,568
RAC: 24,487
United States
Message 755 - Posted: 14 Jan 2017, 0:58:25 UTC - in response to Message 753.

You mentioned errors with AVX app. Are they problems with starting/running app, or validation errors? If with starting/running, could you provide me link to example failed WU? I would like to check if there are some details which may be helpful.

Immediate failure. I tested on 12/25 and those WUs are out of the database. On most machines the avx app ran but wasn't any faster (often slower) than the sse2 app. In fact I found no machines on which the avx app was superior. The fma app was a little bit faster than the other 2 versions on my AMD X8 boxes.

rattorosso [Marche]
Volunteer developer
Send message
Joined: 19 Dec 13
Posts: 25
Credit: 1,400,165
RAC: 2,202
Italy
Message 756 - Posted: 14 Jan 2017, 1:30:13 UTC - in response to Message 755.


Immediate failure. I tested on 12/25 and those WUs are out of the database. On most machines the avx app ran but wasn't any faster (often slower) than the sse2 app. In fact I found no machines on which the avx app was superior. The fma app was a little bit faster than the other 2 versions on my AMD X8 boxes.


Be aware you need a capable cpu to run those applications. Looking at your computers, ony AMD FX have both avx and fma, Athlon 5350 have only avx. however, they are not known to be excellent in these capabilities :)
____________

Profile Beyond
Avatar
Send message
Joined: 2 Nov 16
Posts: 24
Credit: 5,400,568
RAC: 24,487
United States
Message 757 - Posted: 14 Jan 2017, 3:19:20 UTC - in response to Message 756.
Last modified: 14 Jan 2017, 3:20:02 UTC


Immediate failure. I tested on 12/25 and those WUs are out of the database. On most machines the avx app ran but wasn't any faster (often slower) than the sse2 app. In fact I found no machines on which the avx app was superior. The fma app was a little bit faster than the other 2 versions on my AMD X8 boxes.

Be aware you need a capable cpu to run those applications. Looking at your computers, ony AMD FX have both avx and fma, Athlon 5350 have only avx. however, they are not known to be excellent in these capabilities :)

Thanks. Pretty much to the point I was making. Looks like you're in the same boat. What CPUs actually use avx effectively? Comparative times?

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,276,743
RAC: 4,520
Italy
Message 759 - Posted: 14 Jan 2017, 11:24:19 UTC - in response to Message 754.
Last modified: 14 Jan 2017, 11:24:36 UTC


Is this temporary or won't you consider Mac OS (X/macOS) ?

We will continue to support Mac OS (sse, avx and fma). We just need more time for building the applications (lack of hardware is the main problem).

Profile Daniel
Volunteer developer
Send message
Joined: 19 Oct 16
Posts: 79
Credit: 2,202,886
RAC: 0
Poland
Message 761 - Posted: 14 Jan 2017, 14:41:24 UTC - in response to Message 759.


Is this temporary or won't you consider Mac OS (X/macOS) ?

We will continue to support Mac OS (sse, avx and fma). We just need more time for building the applications (lack of hardware is the main problem).

valters, I tried to google for "linux mac cross compiler" and found few interesting discussions on StackOverflow on this topic. Looks that there is such crosscompiler ready to use (see https://stackoverflow.com/a/10341443). There is even VM with Apple's system, although I am not sure what their legal status is.
____________

rattorosso [Marche]
Volunteer developer
Send message
Joined: 19 Dec 13
Posts: 25
Credit: 1,400,165
RAC: 2,202
Italy
Message 762 - Posted: 14 Jan 2017, 16:17:41 UTC - in response to Message 757.


Thanks. Pretty much to the point I was making. Looks like you're in the same boat. What CPUs actually use avx effectively? Comparative times?


Of mine only the FX have avx. I posted my test on this machine above and there was a little advantage over sse2 but I verified later that running the application on all cores the advantage gets negligible.
Anyway we no more need to care about this. Optimized applications are distributed by the project, just delete app_info.xml and restart boinc (better if you abort and report workunits first) and you will get the right app.

[AF>Le_Pommier] Jerome_C2005
Send message
Joined: 12 May 14
Posts: 4
Credit: 176,072
RAC: 1
Mexico
Message 766 - Posted: 15 Jan 2017, 21:24:06 UTC - in response to Message 759.


Is this temporary or won't you consider Mac OS (X/macOS) ?

We will continue to support Mac OS (sse, avx and fma). We just need more time for building the applications (lack of hardware is the main problem).


Thanks a lot for your answer.

I'll be patient then :)

No.15
Send message
Joined: 2 Feb 16
Posts: 13
Credit: 64,229,764
RAC: 2,328
United States
Message 767 - Posted: 16 Jan 2017, 1:38:02 UTC

when I am resetting the project it is not downloading the app to run the WU's. It is just downloading WU's and the erroring out with computational error.

I have been able to get it to download the apps with a detach/reattach. Initially it was running avx @ ~1hr45min a wu but now it is running ver9 at 4 hours a wu. Is there a reason it back leveled to the old app?

Profile valterc
Project administrator
Project tester
Send message
Joined: 30 Oct 13
Posts: 320
Credit: 16,276,743
RAC: 4,520
Italy
Message 768 - Posted: 16 Jan 2017, 10:04:50 UTC - in response to Message 767.

when I am resetting the project it is not downloading the app to run the WU's. It is just downloading WU's and the erroring out with computational error.

Did you have the app_info? In this case the safest way I know to 'go back' is a) make sure the wu cache is empty, stop boinc, delete the app_info.xml file, start boinc. (A simple reset is not enough)

I have been able to get it to download the apps with a detach/reattach. Initially it was running avx @ ~1hr45min a wu but now it is running ver9 at 4 hours a wu. Is there a reason it back leveled to the old app?

That's probably my fault, I forgot to 'deprecate' the old 0.09 apps. It should be fixed now, sorry, you can abort the 0.09 workunits.

No.15
Send message
Joined: 2 Feb 16
Posts: 13
Credit: 64,229,764
RAC: 2,328
United States
Message 770 - Posted: 16 Jan 2017, 16:36:41 UTC

app_info was deleted. I did not stop the boinc service though, I just hit reset after deleting app_info. I will try again on a different machine and see if it makes a difference.

Thanks for the work everyone did on this app, I really appreciate it.

No.15
Send message
Joined: 2 Feb 16
Posts: 13
Credit: 64,229,764
RAC: 2,328
United States
Message 771 - Posted: 17 Jan 2017, 15:52:58 UTC

Funny how when I follow the directions it works correctly ;)


Thanks for your help

Thyme Lawn
Send message
Joined: 22 Nov 16
Posts: 1
Credit: 430,505
RAC: 330
United Kingdom
Message 775 - Posted: 21 Jan 2017, 12:27:53 UTC

My Windows 10 i7-6700K @ 4.00GHz system is hyper-threaded to 8 cores. When Gene Network Application 0.10 windows_x86_64 was released it was initially only running the avx version (65 tasks). That was followed by a spell running the sse2 version (38 tasks) and its current work allocation has 29 tasks running the fme version. Performance figures so far are:


  • 65 completed avx tasks:

    • Average run time 43:11 (min 41:09, max 49:50)
    • Average CPU time 42:10 (min 40:54, max 43:53)


  • 38 completed sse2 tasks:

    • Average run time 45:04 (min 43:42, max 46:36)
    • Average CPU time 44:25 (min 43:01, max 46:13


  • 9 completed fme tasks:

    • Average run time 41:31 (min 40:02, max 42:33)
    • Average CPU time 40:48 (min 39:45, max 41:43)


CPU time gives a more realistic indication of the performance (the avx task with run time 49:50 had CPU time 42:18), and at the moment the timings are pointing towards the fme version being the fastest on that computer (the sample size is too small to say that it's statistically significant, but the minimum and maximum CPU times suggest that's likely to be the case).
____________
"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 . . . 10 · Next
Post to thread

Message boards : Number crunching : Optimization


Main page · Your account · Message boards


Copyright © 2017 CNR-TN & UniTN