Posted 16 Jan 2014 by
danicampa90
12 GB free: It should last for quite a bit
Side note: We are almost finishing the workunits to distribute!
Once we validate the results that we received when the validator was not running, we should get all the results we need for the post-processing!
Posted 15 Jan 2014 by
danicampa90
I'm having some issues in connecting with the VPN... It does not seem to work anymore.
I will try to fix it tomorrow morning.
If you are in desperate need of space, delete the big file in my home...
Posted 15 Jan 2014 by
danicampa90
Will look into the issue soon.
I think I will
- Delete the old_things (if they are really old)
- Compress sample_results (I don't know if they are going to be used, but lzma usually does the job in a good way - we may delete them later if needed)
- Check for older files in download and staging directory.
One home folder may be freed in the process ;)
Posted 4 Jan 2014 by
danicampa90
Updated :)
It will (slowly) go into effect with the new workunits that are being generated.
Posted 4 Jan 2014 by
danicampa90
I forgot to upload them.. I will do it now :)
Posted 27 Dec 2013 by
danicampa90
Once the upload of the results are completed you can delete everything,
Even if you don't do it, Boinc should automatically take care of deleting things if it's running low on the space reserved to him. Our project take up a lot of space because we work with big files. I agree that we need to try to reduce the file size, maybe by keeping them compressed.
Posted 27 Dec 2013 by
danicampa90
I should be able to come.
Marco is not in Trento, so will not be able to be there.
Posted 26 Dec 2013 by
danicampa90
A quick correction: I'm stopping the validator/task generator now, so that the queue is consumed a bit during the night.
[edit] Stopped right now.
Posted 26 Dec 2013 by
danicampa90
In short: Soon the task generator and the validator will be stopped, and they will remain down until we find a fix the validation problems we are having.
Long version: The validation issue is prolonging for too much time and we don't want you to waste your cpu cycles on something that will probably result in a validation error, so we decided to temporarily stop the task generator and the validator of the project.
Stopping the task generator will allow us to clear our queue cleanly, while stopping the validator should allow us to freeze the currently running tasks, and process them with the working validator later on. The credits for the tasks should remain freezed until we will start up the project again.
Sorry for the problems we are having in this period, we are working hard in order to fix them.
Thank you all for helping in this odissey :)
Posted 26 Dec 2013 by
danicampa90
Ok, I will do a post in the news section now and disable the task generator/validator tomorrow morning.
I still suggest to continue working on the new validator. From what I heard, I think it's very difficult if not impossible to fix the differences in the output in the application side.
Posted 26 Dec 2013 by
danicampa90
I will try to stop only the validator of the project, so that maybe the workunits won't be sent multiple times on validation errors.
I think that when we have a good validator we can start it up again, and hopefully it will assign credits also for old workunits.
Is it a good idea, or is it better to stop the entire project?
I will wait your answer before doing anything, valterc.
Posted 25 Dec 2013 by
danicampa90
@Chau: The difference was expected, and we spent meetings discussing this. If I recall correctly, in the meetings we have said that the differences (if small) would need to be "merged down", and the workunits accepted nevertheless. If the difference is "high" the workunits need to be rejected (probably someone was cheating, or an error occurred)
The small differences are OK, and totally expected (due to errors in floating point computations).
I have a big question: Why did you compare the results by md5? I don't remember we spoke about md5 in the meetings. If we were to accept workunits only when equal we would have used the standard bitwise validator. Why did you implemented again the standard bitwise validator using md5?
@Paolo: He was asking if you could in the windows executables end the lines only with \n and not by \r\n. The conversion \n -> \r\n is done automatically by windows unless you open the file in "binary mode". The way to open a file in binary mode changes depending on the API you use.
I think that this should simplify the work the validator has to do.
Merry Christmas to all.
Posted 24 Dec 2013 by
danicampa90
I received a mail from DanT, in which he reported a problem: ALL the workunits on his pc are resulting in a computation error (for example see result 36806).
He is running our application on Microsoft Windows 7 Home Premium x64 Edition, Service Pack 1,
CPU is an i5, with 4 GB of RAM.
We don't have any log of the crash, maybe he is missing some library?
We also have other users on win7 x64 fine, but they seem to work correctly.
If you have access to the control panel I suggest having a look at the results generated by his PC (pc 12)
Posted 24 Dec 2013 by
danicampa90
I noticed that too, and I was going to post a new thread right now.
I don't have the source code of the validator, nor I know how it works.
I examined two results that were different (of WU 23056), and I noticed
- files are gzipped (this is expected)
- files, when unzipped, present differences in newlines.
- when correcting the newlines there were only a few lines of differences, so the workunits should have been accepted. (WU 23056)
There is also another issue, for which I will create a separate topic....
Posted 23 Dec 2013 by
danicampa90
Server is now up again.
This time with longer tasks :).
Should be approximately 5 times harder, but the some may vary, depending on other factors (the expected time should be correct, nevertheless)
Posted 23 Dec 2013 by
danicampa90
The server will be down for approximately 10 minutes in order for me to upload the new longer workunits with the "final" round of experiments.
I will post here when everything is done.
Posted 22 Dec 2013 by
danicampa90
We just updated the application, and deprecated the old one.
The new version should hopefully fix the problems we were having with checkpoints and suspensions.
After a short period of testing we will increase the workunit length (I planned this for tomorrow).
Posted 21 Dec 2013 by
danicampa90
I want to add an approximate calculation:
Currently the average of time needed for a workunit max 25 minutes, This means that each core is able to compute 57 workunits/day.
If you have 2 computers with i7 processors you could theoretically reach the limit of 500/day (8 virtual cores * 2 pcs * 57 WU/day > 500 WU/day).
If that's the case, increasing the difficulty of workunits to about an hour may mitigate this problem.
I was planning to increase in difficulty when we get the new application and the new input files from the preprocessing group.
If you agree I can also increase it now (to about 1 hour - 1 hour and half).
Posted 21 Dec 2013 by
danicampa90
I examined the log and it seems that at the same when bobovitz didn't get any WU, other requests (from other users) are served correctly, so it is probably a problem with the limits.
We may try to increase a bit?
I would also have a look at this:
<min_sendwork_interval>120</min_sendwork_interval>
<max_wus_in_progress>5</max_wus_in_progress>
<max_wus_to_send>5</max_wus_to_send>
and see what happens....
Valter, what do you think?
Posted 21 Dec 2013 by
danicampa90
Hi!
I would like to have some feedback on the length of the workunits:
- How much time do they take on your machine?
- Would you like to have longer workunits?
The overhead of dowload/upload may decrease a bit with longer workunits (wrt the total time taken by the workunit),
Thank you.