I noticed that too, and I was going to post a new thread right now.
I don't have the source code of the validator, nor I know how it works.
I examined two results that were different (of WU 23056), and I noticed
- files are gzipped (this is expected)
- files, when unzipped, present differences in newlines.
- when correcting the newlines there were only a few lines of differences, so the workunits should have been accepted. (WU 23056)

There is also another issue, for which I will create a separate topic....

OK. I will check it. Thanks

Hi,

The current validator is checking files by md5. That's why when you receive two files with different check_sum, they will mark as different. I'm improving it with removing \r (Carriage-Return) before comparing it. Thanks

By the way, Paolo, when you output results to files, can you please take a look at the format of output? I mean, can you set the mode to "binary" instead of "text"? In windows, the new line is '\r\n' and in unix, the new line is '\n'. Thanks

Hi,

The current validator is checking files by md5. That's why when you receive two files with different check_sum, they will mark as different. I'm improving it with removing \r (Carriage-Return) before comparing it. Thanks

Hi,

After checking files from WU 23056 and removed the character CR(\r), I still see the difference between two files( result_id 36654 and 36655) even though they are running on the CPU Intel core-i( result_id 36786, after removing the CR \r, they have a big different size with the rest files). Can you please check it again? Thanks

I mean, when you write to a file, do you set the mode to write files as binary mode or text mode?

@Chau: The difference was expected, and we spent meetings discussing this. If I recall correctly, in the meetings we have said that the differences (if small) would need to be "merged down", and the workunits accepted nevertheless. If the difference is "high" the workunits need to be rejected (probably someone was cheating, or an error occurred)
The small differences are OK, and totally expected (due to errors in floating point computations).
I have a big question: Why did you compare the results by md5? I don't remember we spoke about md5 in the meetings. If we were to accept workunits only when equal we would have used the standard bitwise validator. Why did you implemented again the standard bitwise validator using md5?

@Paolo: He was asking if you could in the windows executables end the lines only with \n and not by \r\n. The conversion \n -> \r\n is done automatically by windows unless you open the file in "binary mode". The way to open a file in binary mode changes depending on the API you use.
I think that this should simplify the work the validator has to do.


Merry Christmas to all.

Hi Daniel,

The results before I test them are the same, that's why I used md5. I will improve it but the problem is how much difference between WUs would probably be marked as unaccepted or accepted. How much difference you will consider as small difference and high difference? What kind of difference will be considered as different or similar. What is the case if we marked them in a wrong way?. I sent an email to Paolo to ask more detail about that. Thanks and Merry Christmas.

For detail, take Wu 23056 as an example. We have three output results.

1. Expansion_At2_work-1387833585.xml_pn19926_0_0 runs on GenuineIntel Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz [Family 6 Model 42 Stepping 7] with x86_64-pc-linux-gnu platform.

2. Expansion_At2_work-1387833585.xml_pn19926_1_0 runs on GenuineIntel Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz [Family 6 Model 42 Stepping 7] with windows_x86_64 platform.

3. Expansion_At2_work-1387833585.xml_pn19926_2_0 runs on AuthenticAMD AMD FX(tm)-6300 Six-Core Processor [Family 21 Model 2 Stepping 0] with windows_x86_64 platform

This first and the second run on same Intel core i7, different platform (Windows & Linux) and two results have small differences. The first one have 3 new edges: 252411_at,255604_at, 253467_at,262159_at, 253467_at,262159_at. While the second one have 1 new edge: 252002_at,253467_at

If we consider two results are all correct. Please give us a threshold of difference to judge. For example: If total of new edges < 1% total of edges, we will give both result credit.

The third is totally different. The size is much bigger (38 Mb vs 27 Mb), and have 661 lines in format "#... list_gene" compare to 500 lines of the first and the second.

Thanks and Merry Christmas.

Hi Paolo, Daniel

Thanks for this. Moreover, when you try to compress the file, can you distinguish name of original file and name of compressed file as different names.

For example: name of compressed file should be "name_c" and name of original file should be "name". Actually, this is more convenient for me to process files after unzipping it(avoid coincidence in name of files). Moreover, after unzipping files, I will delete the compressed file. So it is easier to distinguish between two kinds of files.

I think I solved the problem. So you don't need to change names of files.

As Paolo stated, you should (and actually you should already had) contact the Professors and post-processing group in order to decide:
- up to what difference two file can be considered as both correct
- whether we should keep all the lines contained in both files or whether we should keep only the intersection of the two
Please, let us know your progresses.

I will try to stop only the validator of the project, so that maybe the workunits won't be sent multiple times on validation errors.
I think that when we have a good validator we can start it up again, and hopefully it will assign credits also for old workunits.
Is it a good idea, or is it better to stop the entire project?
I will wait your answer before doing anything, valterc.