DA Photo Contest Forum / Announcements / General Announcements / Building and maintaining integrity / Latest Posts

RE: Building and maintaining integrity

WillPat — Sat, 23 Jun 2007 10:42:20 GMT

"Building and maintaining the integrity of the system" is very important. I am gratified to see our Admin publicly restating his commitment to that end, and thankful for the effort he puts into it.

Thank you Eric, keep up the good work.

RE: Building and maintaining integrity

NiKiMoTo — Thu, 21 Jun 2007 12:20:52 GMT

[quote][b]Admin (6/21/2007)[/b][hr]Each situation is unique. In this case, being that the focus was just on 10s, it is reasonable to first look at how those 10s are spread. If 250 of them were for a single user, that is likely a problem.[/quote]

Yes, of course, it was reasonable to first look at the spread. I didn't mean to suggest that it
wasn't.

RE: Building and maintaining integrity

Admin — Thu, 21 Jun 2007 10:48:40 GMT

Each situation is unique. In this case, being that the focus was just on 10s, it is reasonable to first look at how those 10s are spread. If 250 of them were for a single user, that is likely a problem.

RE: Building and maintaining integrity

NiKiMoTo — Thu, 21 Jun 2007 07:20:57 GMT

Wouldn't you have to look deeper than just how
many X votes member A gave member B at
some point, if you really thought there was a problem?

Maybe it would be a rare case but I can
envision that there could be a scenario where
member A likes to vote in the Macro category
more than others because they are fascinated
by and interested in Macros in particular.

Member B is extraordinarily accomplished and
excels at Macros. Member B also uploads
many more Macros into that category than any
other and also more than any other member.
Couldn't this skew the appearance of how A
is voting on B's work?

Basically I'm suggesting that, beyond how many X votes A gave B and whether C-Z also gave B some X votes, categories and even dates
perhaps should be factored in.

What if the member with the irregular amount
of 10's given to User2888 also gave 10's to
other users in the same category at around the same time?

I don't know...I'm all for investigations, I just
hope that if any action against someone is ever being considered there will be a deeper look
into the data and perhaps a second opinion sought.

Then again, maybe my scenario above is a
silly impossibility. :)

Are there people who tend to enter and people
who tend to vote only in particular categories?

RE: Building and maintaining integrity

Glynn — Wed, 20 Jun 2007 16:49:01 GMT

Its not me who voted all the 10's - I was simply the recipient of 23 of them. I have only given out 56 votes of 10 since I started here. It all makes for something akin to a bell-like structure

Votes you've cast
0.01% 1	0.00% 0	0.02% 2	0.26% 24	2.24% 205	20.11% 1837	48.35% 4418	24.48% 2237	3.91% 357	0.61% 56
1	2	3	4	5	6	7	8	9	10

So I will not be modifying my voting habits - and you will get your usual unbiased and anonymous vote from me.

RE: Building and maintaining integrity

Glynn — Wed, 20 Jun 2007 15:25:33 GMT

It gives me great encouragement and faith in the integrity of the site to see Eric blitzing the stats and turning over anomalous patterns. Way to go Eric.

I am actually the user who recieved 23 of those 400+ 10 votes. I can't say that I noticed a flurry of 10's from the same member - as they also gave me 6's & 7's. 23 votes of 10 from a person doling out 400+ is around 5% of the total. My stats show that 6.5% of my collected votes are 10's - so statistically I was actually short changed.

Votes you've received
0.00% 0	0.00% 0	0.00% 0	0.00% 0	0.52% 35	5.93% 400	28.88% 1948	41.59% 2805	16.52% 1114	6.55% 442
1	2	3	4	5	6	7	8	9	10

I am not at liberty to name the member with the odd voting pattern - but I do know that they are not malicious. This was simply an overzealous voter with more votes than sense it seems.

I'm impressed that the site admin will actively police, and investigate when necessary, these type of situations. >5 Eric.

RE: Building and maintaining integrity

gretchensteele — Wed, 20 Jun 2007 13:45:23 GMT

Eric - many thanks for posting this as it has helped me to understand the system in place.

Building and maintaining integrity

Admin — Wed, 20 Jun 2007 13:24:43 GMT

I would like communicate to everyone here how important the task of building and maintaining the integrity of the system that grants ratings and awards is to the admin of this site. I want to give everyone an understanding of the kind of research that I do with the data that is recorded as you vote by providing a real example of a situation that happened today.

First, to clear up some confusion, it is my belief that every user's voting 'pattern' should develop into a normally distributed bell curve given enough samples. This is not to say that I believe that everyone should vote the same (if that was the case, we'd only need one voter). This is to say that a majority of your votes should be centered around [u]your[/u] median score and as you move more extreme (towards 1 or 10) the votes should become less frequent (this should happen naturally).

With that said, an example arose today of a user who's distribution does not follow this reasoning. This user has approximately 260 votes cast as a score of 9 and approximately 400 votes cast as a score of 10. This raises a red flag for me to see the number of 10s so much higher than the number of 9s.

What I then looked for was whether these votes of 10 were concentrated among a few users (indicating unethical voting behavior) or whether these '10' votes were distributed in a reasonable fashion. In this particular case, the 400 votes of '10' were spread among 178 users with a slight concentration on a few accounts.

Next, I selected out those accounts that showed a higher concentration (one had 23 votes of 10 from this user) and looked at the individual images that received the '10' vote to see how many others they had gotten. Doing so yielded again normal results as most of those images had received scores of 10 from many other individuals (one even had 17 '10' votes).

The point here is to say:

1) The very large majority of votes cast here are well intentioned
2) Your admin works hard to protect the integrity of the system
3) You can have faith in the awards that are granted and the ratings you see on each image

You may view this [url=http://www.dailyawards.com/vex.xls]spreadsheet[/url] to see the true results of this research. If the user wishes to reveal his/her identity then you may validate the findings just as I did.