Yahoo Forum Archive

> Message: 14
> Date: Tue, 3 Jan 2006 00:01:08 -0600
> From: Anthony Holder <aaholder@swbell.net>
> Subject: Re: Re: points and various voting matters
>
>>> I wouldn't mess with the point system, unless it were to drop it in
>> favor of> straight character counts.
>>
>> I have to say that I wouldn't be in favor of using straight character
>> counts because one long glowing review could effectively keep the
>
> Straight character counts, with a cap of 1000, would be very similar to
> what is currently in place, and would be easy to code. The cap would
> eliminate the problem with "one long glowing review" dominating the
> voting.
>
> Changing the number per point would also be easy to code.
>

Thanks for weighing in on programmability. I don't want to do anything
that will create a lot of work for you, and I'd be happy with either of
these options. I think I prefer the straight character count (with
cap), though I could live with the other way too.

> Here are some stats to chew on: I binned the reviews by number of
> characters (0 => 0-99, 100 => 100-199, ..., 3400 => 3400-3499)
>
> Number of reviews | Number of Characters
>
> <snip stats>

Looking at (I think) Inkling's point spread, we would have gotten the
following:

1 point (1-149 chars) = 1537 reviews
2 point (150-299 chars) = 1836 reviews
3 point (300-449 chars) = 1119 reviews
4 points (450-599 chars) = 601 reviews
5 points (600+ chars) = 930 reviews

I know Inkling's points were slightly different, but I was trying to
work with the numbers Anthony gave. I think you get the idea.

Anyway, I think if we do want to go with something similar to this
point spread we might consider making the #4 bracket a bit larger.
Maybe making it 450-700 points? That might make the brackets more
equitable.

> Of the 6083 reviews, about 11% were more than 700 characters
>
> Fewer than half were > 300 characters.
>

I find these statistics fascinating because it shows that I really am
in the minority. Or maybe I'm just mis-remembering my own stats.
Anthony, would it be hard to run similar statistics for my own reviews?
Is this even possible? I'd love to see my own reviews broke down into
the 100-point brackets you did above.

If this isn't too hard and you don't mind, feel free to email them to
me privately; if not, don't worry about it at all. In any case don't
feel the need to reply onlist to this part.

> I don't know, from the goals, etc., of the awards, what the
> benefits/drawbacks of limiting the impact of the longer reviews and
> emphasizing the shorter reviews would be.
>
> It does seem that maxing out the points on a review doesn't completely
> inhibit people from writing longer ones.
>

Here's my thoughts on this:

People who want to write longer reviews are probably going to already
do this,, time permitting. I don't see these people getting embarrassed
or feeling like they should write more; their reviews are worth more
points anyway. But some people aren't naturally as verbose and so *all*
of their reviews will be shorter, while other people will write shorter
reviews for some stories.

If there's a voting system that will make it easier to write these
short reviews as well, I think this will make it easier for those
people who would write primarily short reviews to get involved. It
would also make them feel like their contribution is valued -- both of
which I think will make the awards run better.

> Question: If someone is reworking things to get that extra point, and
> going from 180 to 200 characters, do you really think the extra 3-4
> words is going to add significantly to the quality of the review?
>
> I would suggest a more continuous scale, for that reason. That way,
> people don't artificially inflate their reviews to get that next point,
> they write what they think is a good review and quit when they're done.
>

I know I occasionally added an extra (valid) thought to a review, but
more often than not I think people trying to reach the next point level
do add an extra word here or there and end up artificially inflating
the reivew, like you say. So yes, I like the more continuous scale.
Besides actually working better than the point system, it would keep us
from having to decide what exactly the character counts for each point
should be - we'd sidestep that issue entirely.

> You could come up with some added benefit to those extra long reviews,
> like maybe counting 5% of the characters above 1000. That way, the 3468
> character review would count 1123 points, rather than 1000. I do think
> that one long glowing review shouldn't dominate the scoring, but I do
> think that person should get some added benefit from all that extra
> writing. This is especially true if you want to lower the cap to
> emphasize the reviews with lower character counts.
>
> It could be something a bit more complicated than that, like they do
> with income taxes:
>
> 0 to 500 chars, 100% of chars
>
> 500 to 750 = 500 + 50% of chars > 500
>
> 750 to 1000 = 500 + 50% of chars between 500 and 750 + 25% of chars
>> 750
>
>> 1000 = 500 + 50% of chars between 500 and 750 + 25% of
> chars between 750 and 1000 + 5% of chars > 1000
>
> The current system essentially does this, but uses 2/3 of chars > 500,
> with a hard cap at 1100 chars, and has step discontinuities (where it
> jumps from 1 point to 2 with a change of just one character), where
> this would be a smoother system with no discontinuities.
>
> We could even match the current system, but eliminate the
> discontinuities.
> 0 to 500 = 100% of chars
> 500 to 1100, 500 + 2/3 of additional chars
>> 1100 (either no additional, like present, or 5% of additional chars)
>
> As long as it is just math to determine the score from the valid
> character count, I can do anything you want, and it's easy, so have fun
> thinking of ways to count scores that will achieve the goals of the
> MEFAs (lots of good reviews for your stories).
>

I do want to keep this relatively simple so it's easy to explain. That
doesn't mean we have to go with strict character counts, but let's not
make it *too* complicated. Something like

1 point for each character up to 500 +
1/2 point for each character between 501-1500

I also don't want to not have a cap on the number of points per review.
Part of it is I don't want one reviewer to control too much of the
competition. But more than that, I want to encourage people to move on
to review another story.

But that caveat aside, I really like this idea. It's probably my
favourite choice of all the ideas presented.

Off to bed... I may get to another email tomorrow morning, otherwise it
will be tomorrow night. Feel free ot keep discussing, guys.

Marta

On voting by direct character counts:
I don't like thsi idea, because instead of asking myself whethera story is worth 1-10 points, I would end up trying to work out where on the scale 1-1000 it isworth and itwould be much harder.
Also, suppose there were two stories that had about the same level at the moment I might review by saying,
"Interesting story. I like the way you characterised A and B."
For a story that I thought was worth one point. Now if character counts were being used if A&B were Frodo and Sam, that review would be 53 points, but if they were Aragorn and Legolas would be 59 points.
I would feel that this shows one of the problems, that the same comments for a story would be worth more if names etc were mention that are longer in one story. The fact that this difference could be a result where one story gets an award and that would seem to be unfair.
By voting in stages these sort of differences are negated because it is easy to adjust a review to ensure that it is in the right range without too much worrying about exact character counts.

Also, if we are going by points, I can encourage the scale to be arranged so that the catogories have equal width. I think Marta was talking about the fact that one range had fewer reviews in it last year. However, I would have thought that this would not necessarily show anything because the scales were not the same and at least some people were looking at the scale to vote.

Jenn



---------------------------------
Play Santa's Celebrity Xmas Party, an exclusive game from Yahoo!

[Non-text portions of this message have been removed]

I didn't want to get into this discussion but as someone who writes
notoriously short reviews I think I have to weigh in.

>If there's a voting system that will make it easier to write these
>short reviews as well, I think this will make it easier for those
>people who would write primarily short reviews to get involved. It
>would also make them feel like their contribution is valued -- both of
>which I think will make the awards run better.

I think Marta you hit here on the main point for me. I felt bad because, as
much as I tried, I was not able to write a 10 point review which would fit
my writing style and not feel inflated to me. I tried for one story but
couldn't do it. After that I gave up and didn't think about any point count.
The reviews got as long or as short as they would flow out of my keyboard.
When I looked over them, the majority ranged in the 3-5 point count and
there were some stories in there I would have loved to give more points. But
that would have taken the fun out of the awards for me. It would have felt
more like a dreaded paper had to be delivered in school.

Chris

I'd guess that some people probably tend to range in reviews from 1 to
about six pointers and others range from three to ten, but it doesn't matter
much because each reviewers range still weights their choices fairly among
the stories. So a person who usually writes short reviews might be lower in
the numbers but still be giving an accurate indication of their opinion
overall.

If it wouldn't be such a pain in the arse to figure out how to do it, I'd
say don't tell reviewers how many points they've given out when
they're composing or editing reviews -- just show them the list of stories
from that subcategory ranked in order with "unreviewed" at the bottom of the
list. That way you could adjust your reviews to get the stories in the
preferred order if you really wanted to, and ignore the whole "number of
points" question if you felt like it too. You'd only be comparing your own
efforts to your own efforts as a reviewer.

Obviously I am getting too tired to be allowed at a keyboard. Goodnight
all!

On 1/5/06, Chris Grzonka <grzonka@adelphia.net> wrote:

> I didn't want to get into this discussion but as someone who writes
> notoriously short reviews I think I have to weigh in.
>
> >If there's a voting system that will make it easier to write these
> >short reviews as well, I think this will make it easier for those
> >people who would write primarily short reviews to get involved. It
> >would also make them feel like their contribution is valued -- both of
> >which I think will make the awards run better.
>
> I think Marta you hit here on the main point for me. I felt bad because,
> as
> much as I tried, I was not able to write a 10 point review which would fit
> my writing style and not feel inflated to me. I tried for one story but
> couldn't do it. After that I gave up and didn't think about any point
> count.
> The reviews got as long or as short as they would flow out of my keyboard.
> When I looked over them, the majority ranged in the 3-5 point count and
> there were some stories in there I would have loved to give more points.
> But
> that would have taken the fun out of the awards for me. It would have felt
> more like a dreaded paper had to be delivered in school.
>
> Chris
>
>
>
> ------------------------------
> YAHOO! GROUPS LINKS
>
>
> - Visit your group "MEFAwards<http://groups.yahoo.com/group/MEFAwards>"
> on the web.
>
> - To unsubscribe from this group, send an email to:
> MEFAwards-unsubscribe@yahoogroups.com
> <MEFAwards-unsubscribe@yahoogroups.com?subject=Unsubscribe>
>
> - Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service <http://docs.yahoo.com/info/terms/>.
>
>
> ------------------------------
>


[Non-text portions of this message have been removed]

--- In MEFAwards@yahoogroups.com, rabidsamfan <rabidsamfan@v...> wrote:

>
> I'd guess that some people probably tend to range in reviews from 1

to> about six pointers and others range from three to ten, but it
doesn't matter> much because each reviewers range still weights their
choices fairly among> the stories. So a person who usually writes
short reviews might be lower in> the numbers but still be giving an
accurate indication of their opinion> overall.

>

I do think reviewes tend to have a ranger they're comfortable in, and
that is mainly why I support the 1-5 spread, because I think that
helps to level the field between reviewers *styles* while still
rewarding authors that the authors like enough to write more about.

Sulriel

On the other hand, narrowing the number of points makes it harder for a
single reviewer to give different votes to stories in the same subcategory.


With a ten point spread, it was fairly easy to write a little more and make
it to the next "point" zone, but with only a five point spread over the same
number of characters, a reviewer who really wanted to make one or two
stories stand out over the others has to shell out a lot more characters to
make any difference. Personally, I think that's a disadvantage for the
people who naturally write short reviews, not an advantage. Which is one of
the reasons why I'd rather do straight, or weighted character counts. Then
when I'm looking at a drabble subcategory I can easily write five reviews of

70, 90, 110, 115 and 150 characters or so and still convey how much better I
thought the last one was to the first one. With the point system, I'd be
looking at five reviews which either split into two groups or even all seem
to be equivalent.

On 1/6/06, sulriel <Sulriel@htcomp.net> wrote:
>
> --- In MEFAwards@yahoogroups.com, rabidsamfan <rabidsamfan@v...> wrote:
> >
> > I'd guess that some people probably tend to range in reviews from 1
> to> about six pointers and others range from three to ten, but it
> doesn't matter> much because each reviewers range still weights their
> choices fairly among> the stories. So a person who usually writes
> short reviews might be lower in> the numbers but still be giving an
> accurate indication of their opinion> overall.
> >
>
>
> I do think reviewes tend to have a ranger they're comfortable in, and
> that is mainly why I support the 1-5 spread, because I think that
> helps to level the field between reviewers *styles* while still
> rewarding authors that the authors like enough to write more about.
>
> Sulriel
>
>
>
>
>
> SPONSORED LINKS
> Business writing book<http://groups.yahoo.com/gads?t=ms&k=Business+writing+book&w1=Business+writing+book&w2=Writing+child+book&w3=Writing+a+book+report&w4=Writing+and+publishing+a+book&w5=Book+writing+software&w6=Writing+book&c=6&s=158&.sig=A_1QLmi_KBzc5yJt_LmpIA> Writing
> child book<http://groups.yahoo.com/gads?t=ms&k=Writing+child+book&w1=Business+writing+book&w2=Writing+child+book&w3=Writing+a+book+report&w4=Writing+and+publishing+a+book&w5=Book+writing+software&w6=Writing+book&c=6&s=158&.sig=Y1Wf14eLXiiTKHLU86Iuag> Writing
> a book report<http://groups.yahoo.com/gads?t=ms&k=Writing+a+book+report&w1=Business+writing+book&w2=Writing+child+book&w3=Writing+a+book+report&w4=Writing+and+publishing+a+book&w5=Book+writing+software&w6=Writing+book&c=6&s=158&.sig=kx8FRllMbmaUlxzzNzc8dA> Writing
> and publishing a book<http://groups.yahoo.com/gads?t=ms&k=Writing+and+publishing+a+book&w1=Business+writing+book&w2=Writing+child+book&w3=Writing+a+book+report&w4=Writing+and+publishing+a+book&w5=Book+writing+software&w6=Writing+book&c=6&s=158&.sig=EKZq4da9OsrGfqgIUK_Gtg> Book
> writing software<http://groups.yahoo.com/gads?t=ms&k=Book+writing+software&w1=Business+writing+book&w2=Writing+child+book&w3=Writing+a+book+report&w4=Writing+and+publishing+a+book&w5=Book+writing+software&w6=Writing+book&c=6&s=158&.sig=fOkU4IPRCR5rwvONN-ghNA> Writing
> book<http://groups.yahoo.com/gads?t=ms&k=Writing+book&w1=Business+writing+book&w2=Writing+child+book&w3=Writing+a+book+report&w4=Writing+and+publishing+a+book&w5=Book+writing+software&w6=Writing+book&c=6&s=158&.sig=yI0knfwDHkDFmAD4xV-YoQ>
> ------------------------------
> YAHOO! GROUPS LINKS
>
>
> - Visit your group "MEFAwards<http://groups.yahoo.com/group/MEFAwards>"
> on the web.
>
> - To unsubscribe from this group, send an email to:
> MEFAwards-unsubscribe@yahoogroups.com<MEFAwards-unsubscribe@yahoogroups.com?subject=Unsubscribe>
>
> - Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> Service <http://docs.yahoo.com/info/terms/>.
>
>
> ------------------------------
>


[Non-text portions of this message have been removed]

On Jan 5, 2006, at 8:47 PM, rabidsamfan wrote:

> If it wouldn't be such a pain in the arse to figure out how to do it,
> I'd
> say don't tell reviewers how many points they've given out when
> they're composing or editing reviews -- just show them the list of
> stories
> from that subcategory ranked in order with "unreviewed" at the bottom
> of the
> list. That way you could adjust your reviews to get the stories in the
> preferred order if you really wanted to, and ignore the whole "number
> of
> points" question if you felt like it too. You'd only be comparing
> your own
> efforts to your own efforts as a reviewer.

RSF, this'll be at the bottom of the list, but it might be possible.
Sorting is fairly easy, as is linking the review character count to the
story so it can be sorted.

About the story review showing up in the Author review box, I was
thinking that I could put it on the page, but not in the box. I might
be able to show all your reviews in that category, as well. If you want
(and I have time to) to change the Author Reviews around so they aren't
category-based, then which Author review should you be taken to?
There'd have to be a page with links to all the possible author reviews
for that author.

Anthony

Secret point count: coded so Staff can see it to be sure it's legit and
shown after?

Interesting idea....

--Ainaechoiriel
MEFA Admin and Founder

"This evil cannot be concealed by the power of the Elves," Elrond said, "for
it is Windows-compatible, and freeware at that." --H.F.

http://gabrielle.sytes.net/mefa The Middle-Earth Fanfiction Awards

Blog: http://www.ainaechoiriel.blogspot.com

> -----Original Message-----
> From: MEFAwards@yahoogroups.com
> [mailto:MEFAwards@yahoogroups.com] On Behalf Of rabidsamfan
> Sent: Thursday, January 05, 2006 8:48 PM
> To: MEFAwards@yahoogroups.com
> Subject: Re: [MEFAwards] Re: Re: points and various voting
> matters (Anthony)
>
> I'd guess that some people probably tend to range in reviews
> from 1 to about six pointers and others range from three to
> ten, but it doesn't matter much because each reviewers range
> still weights their choices fairly among the stories. So a
> person who usually writes short reviews might be lower in the
> numbers but still be giving an accurate indication of their
> opinion overall.
>
> If it wouldn't be such a pain in the arse to figure out how
> to do it, I'd say don't tell reviewers how many points
> they've given out when they're composing or editing reviews
> -- just show them the list of stories from that subcategory
> ranked in order with "unreviewed" at the bottom of the list.
> That way you could adjust your reviews to get the stories in
> the preferred order if you really wanted to, and ignore the
> whole "number of points" question if you felt like it too.
> You'd only be comparing your own efforts to your own efforts
> as a reviewer.
>
> Obviously I am getting too tired to be allowed at a keyboard.
> Goodnight all!
>
> On 1/5/06, Chris Grzonka <grzonka@adelphia.net> wrote:
>
> > I didn't want to get into this discussion but as someone who writes
> > notoriously short reviews I think I have to weigh in.
> >
> > >If there's a voting system that will make it easier to write these
> > >short reviews as well, I think this will make it easier for those
> > >people who would write primarily short reviews to get involved. It
> > >would also make them feel like their contribution is
> valued -- both
> > >of which I think will make the awards run better.
> >
> > I think Marta you hit here on the main point for me. I felt bad
> > because, as much as I tried, I was not able to write a 10
> point review
> > which would fit my writing style and not feel inflated to
> me. I tried
> > for one story but couldn't do it. After that I gave up and didn't
> > think about any point count.
> > The reviews got as long or as short as they would flow out
> of my keyboard.
> > When I looked over them, the majority ranged in the 3-5 point count
> > and there were some stories in there I would have loved to
> give more points.
> > But
> > that would have taken the fun out of the awards for me. It
> would have
> > felt more like a dreaded paper had to be delivered in school.
> >
> > Chris
> >
> >
> >
> > ------------------------------
> > YAHOO! GROUPS LINKS
> >
> >
> > - Visit your group
> "MEFAwards<http://groups.yahoo.com/group/MEFAwards>"
> > on the web.
> >
> > - To unsubscribe from this group, send an email to:
> > MEFAwards-unsubscribe@yahoogroups.com
> > <MEFAwards-unsubscribe@yahoogroups.com?subject=Unsubscribe>
> >
> > - Your use of Yahoo! Groups is subject to the Yahoo! Terms of
> > Service <http://docs.yahoo.com/info/terms/>.
> >
> >
> > ------------------------------
> >
>
>
> [Non-text portions of this message have been removed]
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>

Hi Nerwen,

Sorry that you felt like you had to repeat this post. I just hadn't
gotten a chance to reply to it.

> Message: 12
> Date: Thu, 5 Jan 2006 09:46:19 +0000 (GMT)
> From: Nerwen Calaelen <nerwen_calaelen@yahoo.co.uk>
> Subject: Re: Re: Re: points and various voting matters (Anthony)
>
> On voting by direct character counts:
> I don't like thsi idea, because instead of asking myself whethera
> story is worth 1-10 points, I would end up trying to work out where on
> the scale 1-1000 it isworth and itwould be much harder.

I think you'll find that this is actually a lot simpler. If you want to
think your way through reviewing on a ten-point scale, you can still do
that -- just whatever you decide a story is worth, multiply that number
by one hundred and aim for about that many characters. It will work out
the same because the maximum number of points per story will be 1000
instead of 10.

> Also, suppose there were two stories that had about the same level at
> the moment I might review by saying,
> "Interesting story. I like the way you characterised A and B."
> For a story that I thought was worth one point. Now if character
> counts were being used if A&B were Frodo and Sam, that review would be
> 53 points, but if they were Aragorn and Legolas would be 59 points.
> I would feel that this shows one of the problems, that the same
> comments for a story would be worth more if names etc were mention
> that are longer in one story. The fact that this difference could be
> a result where one story gets an award and that would seem to be
> unfair.
> By voting in stages these sort of differences are negated because it
> is easy to adjust a review to ensure that it is in the right range
> without too much worrying about exact character counts.
>

I think you're thinking just a bit too hard about this. Yes, there's a
six-point difference between the two, but think how many points the
average story will be getting. To give you an example -- there's a
drabble this year that got 31 points by 11 people reviewing it, for a
total of 2,457 characters. I haven't looked at the exact reviews for
this drabble, so I can't be sure, but I think with that many reviewers
giving those number of points, I'm almost certain none of those is
longer than 1,000 characters. If the hypothetical review you mention
was cast for this story, the Frodo & Sam one would have resulted in
2.16% of that story's total points, and the Aragorn & Legolas one would
have resulted in 2.40% . That's less than one quarter of one percent of
the total points -- really not that significant. And this wasn't by any
means an especially high scorer. In the stories that will be competing
for places the overall score will probably be even higher.

Oh, and one other things -- I went and saw the number of characters
that separated the stories in a subcategory with eleven stories in it.
The closest stories were separated by 164 characters, and most were
separated by more than 1,000. And for what it's worth, this was a
pretty competitive sub-category.

> Also, if we are going by points, I can encourage the scale to be
> arranged so that the catogories have equal width. I think Marta was
> talking about the fact that one range had fewer reviews in it last
> year. However, I would have thought that this would not necessarily
> show anything because the scales were not the same and at least some
> people were looking at the scale to vote.
>

I don't think we'll ever get exactly equal sub-category sizes, but we
can certainly try to make them more equal. For instance, I'm really in
favor this year of making sure no sub-cat has more than 15 stories in
it.

Marta

On Jan 6, 2006, at 8:08 PM, Ainaechoiriel wrote:

> Secret point count: coded so Staff can see it to be sure it's legit and
> shown after?
>
> Interesting idea....
>
> --Ainaechoiriel
> MEFA Admin and Founder

Unless you're talking about something else, that's how it worked in
2005, but I think it was only admins that could see the points.

A.