Voting and/or Rating Threads
 

Any thoughts about voting and/or rating threads?

Could this be done easily using the polling feature?

What if there was a standard default poll for all threads, and the polling feature otherwise disabled. It doesn't seem to be used much.

Threads that reach a predetermined denigration threshold of some number could be "moved" to a Forum called Trash by the Administrator.

We've seen how the "move" feature works on the current IotD. In the case of a thread that is "moved to trash" it might be marked "Trashed" rather than moved, to discourage Guests from wasting their time following that thread to it's new location. The members who are still interested in the trashed thread because they are involved, or otherwise are already familiar with the content, can still get to that thread from its original location for a short time, while they are on "notice" that it has been trashed.

Such a thread could still be "followed" by those interested, but it would move down the page and off the page of the forum relatively quickly. Admittedly bad threads with high posting rates would not be misrepresented to Guests and Members as the "best of the Cellar" by implication of the multiple pages and high posting and page view numbers.

This would keep stale and worthless threads from "cycling" at the top of the forum longer, the more people rant and complain on the thread. Now, even if someone posts to a thread the opinion that it is the worst piece of trash they've ever seen, it goes right to the top of the pile and takes forever to get off the page, especially in IotD.

I think that this approach is quite democratic.

And, like most democracies, you don't even have to vote, and you can only vote if you are in the thread. Administrator can discount flagrant voting abuse, so there is no incentive to stuff the polling ballot.

Voting could be available only to "citizens" i.e. members, and not Guests, who would not even see the Rating Polls. Maybe it could be programmed so that Guests don't even see the trashed threads, except in Trash. (That's not essential and might take some special programming, though.)

I made a small mistake when I set up this thread, because each member on this thread can only vote once. (Unless UT were to change that.)

In the default poll that I'm advocating for every thread, a member should be able vote in the thread only once in every separate "login session" in which he/she opens that thread. Any voting abuse would be apparent to the Administrator, as I've said.

That way, a members opinion of a thread could change as the posts to that thread change his/her rating, or opinion of the value of the thread to the forum.

I voted "OK", mainly because I don't agree with your implementation. I would rather have a rating scale listed and not have any threads moved. That is if we were going to implement any ranking system whatsoever. If it were implemented like <b>that</b>, I think it would be a cool idea - one could sort threads by rank, etc.

As far as voting multiple times - No. The ability to go back and change your vote, yes. We'd need to come up with some system of determining the "rank" - say it was out of 5, with 5 being the highest. Do we want just an average? Do we want to throw away the extremes, if there are any, and then vote? Should there be some other formula applied to the votes to determine its "rank"?

Anyway. I don't really agree with the way you're wanting to do it, but I definitely think it a) could be done and b) could serve a real value, especially for new people coming in.

The reason for suggesting that a member be able to "rate" the thread multiple times is valid, I think, because the thread changes with each post.

A well conceived initial post can deteriorate into drivel.

A mediocre threat might take on a new vibrant life.

I don't mind that you don't agree with me, at this time.

Perhaps you might change your evaluation of even this thread when others add different ideas that you might like a lot more than mine. :)

Should you then have an opportunity to re-evaluate, or should you be stuck with your orignial assessment of a completely different series of posts?

As far as a rating system, with nothing happening as a result, what is the point of voting if it doesn't change the status quo?

Members will only vote if their votes effect change, one way or the other.

The implementation that I'm suggesting is exactly the reverse of K5 and other community sites, where the threads don't get published up front, until they have the weight of positive votes.

In this implementation, every voice gets a hearing ... but if it is nonsense to start with, or turns into a garbage thread, in the judgment of the members, not me, not the site Administrator alone ... why should it be at the top of the pile?

I say, if the proposed rating system shows that a vast majority of the members thinks a thread is horrible ... by all means get it off the forum.

Why make the members and guests wade through the garbage in search of the nuggets?

I trust the members to create their own environment by natual selection.

It is, perhaps, just semantics whether a thread is moved off the forum to Trash or just moved to the bottom of the forum, by a sorting by ranking idea, as you suggest, dhamsaic.

I think you missed something in my original post -

I say that because I think that it gives a fairer representation as to the quality of the thread. For example - suppose this thread got ranked a "1" because someone thought it sucked. But it turned out to be a great discussion, so they vote it a "5". Now their median is "3", aka - "alright, but don't waste your time on it if there's a 4 or a 5 around". This makes it harder for the ratings to actually change on a thread. However, if the person can change their vote, more weight is given to that one vote. Now their median is a 5, because that's their only vote. Coupled with the other votes, we see what the actual rating is.

I agree that the rating of a thread should change as the thread does, but I don't agree that each person should be able to vote multiple times. It makes it more difficult for the thread to change "positions", if you will - therefore, a thread that got marked initially as a "1" might turn into a great conversation but only be represented by a "3".

Ok, I agree with that. :)

The above system is just a 1-5 rating, with 3 being a "positively weighted median" but I think it adds meaning to put WORDS beside the choices, because that's how people think.

and that's what got me thinking about this ... so I thought we might like to discuss this in a separate thread here.

And I got thinking that's a good idea that might work for all the forums, if handled in a way different from K5.

I think there's way too much Slashdottishness and k5itude here already. I'd hate to think the only way for the Cellar to survive is by doing crap like that. Moderation and karma have become issues of their own at /. , and are blown way out of proportion.

I don't happen to think that the majority opinion about a thread has much to do with its value to me personally, especially as the user population here grows and broadens.

Right. But what I'm saying (and what I think he's saying) is that it could be of some value to new people. I read everything that's posted here, period. So I don't really care about ratings. And I'm great at taking threads off topic, as we all know. But I think that having a "rating of threads" thing could be useful, if implemented properly.

As far as karma and all that shit - We definitely don't need that. I'll be glad to fight that for as long as I'm here.

Guys, I read everything that's posted here, too. I don't really want to, though. I'm just looking for the best stuff.

It's not all good stuff. Problem is ... one has to read through the crap, to find out for oneself that some of it was a waste of time. Then there's the good stuff, that keeps one coming back ... to a point. Eventually, it will get to be too much trouble to sort through the weeds.

If the members who have already read the threads could note, positively or negatively, the value of the thread in the "membership's" opinion ... that would be of value to all members, not just new members and guests. God knows, everyone here has an opinion on everything ... why not an opinion on the thread itself?

Nothing in this proposed rating system takes away Maggie's or dham's or anyone's right to read it all ... even if to judge for themselves what's the value. I'm sure that even some of the drivel here is interesting when you know well the poster. But even you guys will lose interest in this, if strangers start going on and on about poutine or some other exotic foodstuff.

(And then the core group will start to think seriously about starting over with the original founding group, like CitizenX is doing. That wasn't their plan for their community, but CX didn't put in place a system that works for a growing audience and scales for a larger membership.)

I fear that this community will lose members and miss out on attracting the right new members if the existing members don't soon take an active role in sorting through the growing number of posts ... and telling the broader audience out there just what's so good here.

If the core group doesn't want a growing membership, and is happy with the Cellar exactly the way it is ... just put the site behind a password login page and call it a private community that only has to worry about attrition.

I'd weigh in on this with a great deal of thought but I'm real busy right now.

Nic - I agree with you. A lot. Kinda. :)

Don't be offended or anything. All I'm saying is that something like that would need to go through a period where we, as a community, figured out exactly what we wanted and then implemented it. 'Sallgood. :)

I'm not offended or anything. ;)

btw, the question in the "poll" is How would you rate this thread? not How do you like this idea?

I agree with you that we as a community have to think this through and discuss it and figure out what would likely work ... and that's exactly what this thread is about.

I find it hard to believe that this is a Horrible thread that we should not be discussing here.

Of course, if it really is, we should get off this thread soon ... and that's the point.

I think people rated the idea, not the thread. That's okay - the idea can be rated too, right?

dham, I think that is an important distinction.

this proposal isn't for polls ... "hey, what do ya think of this idea?"

we've got that already. although it's seldom used, because members prefer to sound off in a reply post if they agree or disagree ...

what i'm talking about here is a RATING mechanism that will tell the "audience" i.e. members + guests, what the members who have read a thread think of its value as a thread in the forum.

essentially ... is this a good discussion? ... or is this thread a waste of time?

i was proposing to make the implementation of such a proposal easy for UT by utilizing the existing software polls in as a way to RATE the threads ... because, it is really unnecessary to have polls for this type of community ... the ideas and opinions for and against anything are really expressed in the posts of the threads.

that's what i think, at this point, anyway.

Nic - I got it. Seriously. I think everyone else did too. What I'm saying is that I don't think people used this poll to rate the thread - I think they used it to rate the idea of using the poll to rate the thread. I don't think it's indicative of the general feeling toward your idea. That's all I'm saying. I very clearly understand what your idea was. I would think that my previous posts would have established that...

And I don't particularly agree with that either. I most especially disagree with a mechanism to deprecate in some way the threads that ddn't appeal to the hoi polloi...which would presumambly include the "new people". I say it's not broken, and I say don't fix it. Fair minds differ all the time here as to what's drivel and what isn't. I'm not even convinced "attracting new members" is a good goal to have. I'd prefer to see quality not quantity.

I may have to turn in my geek's licence for saying this, but not everything that *can* be done *should* be done.

Maggie - I agree about the deprecation of threads. That's why I said I didn't think they should be moved to "Trash", etc. However, it might be an interesting thing to have, especially if there was an option to "ignore ratings" - i.e., you wouldn't even see them, and they wouldn't affect you. Maybe this would be off by default for current users, but on for new users.

If you think that "you wouldn't even see them" and "they wouldn't affect you" are the same thing, you've never read Slashdot. I think you'd immediately see some people starting to pander to the crowd, and the pandering wouldn't even be to the *entire* crowd, it would be that subset who are dim enough to spend a lot of their time voting; not the most interesting people on the planet. One of those "amongst those expressing a preference" deals that Pepsi made so famous. I just don't think "it might be interesting" is a good enough reason.

All this discussion, and then I read in the FAQ that vBulletin is already set up for rating threads ... what a great idea!

I can't believe the Cellar doesn't have this feature activated, UT.

Which would be the reason why you would have the ratings turned <b>off</b>. People are going to say what they say, and it's not like any one person owns a thread - so therefore, there's really no incentive. We're not talking karma here. We're not talking about rating specific posts. We're talking about rating an entire thread based upon the quality of its posts. This rating would not affect the thread - it would just be used as a criteria to sort threads <b>if the viewer wanted to see the ratings</b>. Don't like 'em? Turn it off. Want to see 'em? Turn it on. With only a single vote happening per person, the ratings couldn't be stuffed.

I really don't see anything negative that could come from this. There would be no incentive to rank a thread, nor would there be any incentive to post there. Why would people pander to some crowd? As we all know, we individually rank people based upon the merits of their comments - whether or not they're participating in a "highly rated thread" would be ignored.

To put it another way: I simply do not understand your stance. Please clarify.

Well, actually it is. *Every* thread has a starter displayed rather prominently, and we're starting to see more individual *forums* too, where most threads are started by the forum owner.

As originally conceived, the indivdual Cellar forums were on a specific topic, like MMM's Gun Locker and Toad's Wine Cellar, as I recall. But lately the new forums are starting to look much more like personal blogs...to the point where Syc even stacks a bunch of apparently unrelated items into a single "stuff I was thinking about at the same time" thread.
You're saying that a high or low rating would not affect who reads a thread...and that how widely read a thread is doesn't affect it? I still think that whoring for ratings--even thread ratings-- will change the nature of the dialog. Anything that changes how stuff is presented--even to a *subset* of the users--ultimately affects the content. I don't see this as having positive effect.

On the original Cellar, threads only happened because people posted as a reply; there was no formal "create a thread" mechanism. Threads did not have titles, posts did...it was very much modeled on USENET, which was an important reason The Cellar existed right up to the point where the USENET feed volume became unsupportable. People would typically log on and be presented with a list of all new posts in all forums they were tracking. That's still how I use it....recent postings keep a thread on the list, and I find out pretty quickly which threads I find interesting. *Forking* a thread is rather difficult now, and it was originally stone simple.

OK!

There are numerous problem with thread ratings, which I'll try to convince you all of, because it's important.

The major problem is that you think they're answering a question, but they're really answering a diffferent question than the one you ask. You ask "What are the threads that I'd like best?" You receive back a list of threads that OTHER PEOPLE thought were best... by definition, a group not including yourself. This is very precisely the wrong answer.

The value of the answer that you get varies according to a set of variables over which we have no control. The rating of any particular thread by anonymous masses can be useful or not useful according to the phases of the moon. Newbies rating threads - threads careening away from the rating - people voting at random - people voting for their friends - parimutuel scoring (I vote for you you vote for me) - everything seems to sway the bottom line.

That's the biggest problem, but by far not the only problem. Notice the shock that people had when we went from ver 1 to ver 2 of the software? Ever watch someone starting out on the Cellar and how confusing it is? These are problems for longer memories. The more "stuff" on the system, the more people are basically confused about how it really runs, to the point where they use it for years and years and never really understand it.

This is not a problem for techie sorts, who are oriented around working out how various features work. (I'm reminded here of a time when I pushed random buttons on an automatic coffee machine, and it hung and had to be reset. To me, it was natural hacker behavior. To everyone else, it was stone-cold stupid.)

But to attract people other than techie sorts, you have to settle for a degree of simplicity and strong usability. This simple fact is one of the unseen factors behind the dot-com crunch -- beyond the obvious fliers on dumb ideas. The biggest ISP in the world is STILL AOL, because they solved the usability problem first.

As it is, the system is creaking on the usability front. There are problems. The message icon, the "C"s, learning vB Code, the stupid "user cp" button. all conspire to create a system that is not very usable. Now we want to add to that another column (I already removed a column -"moderator".)

I believe that vBulletin is really suffering from feature creep, to the point where it is getting difficult to get basic things done. And why IS there feature creep? It's because everyone believes they have a question that must be answered, and everyone's a programmer, and everyone wants their code going towards communities where the code will be highly visible and oft-run.

Would thread rating solve the problem of attracting people? No, the ratings are basically invisible until they fully comprehend the software. Would it encourage quality posting? No, as Mags points out, the /. and K5 experiences tell us it doesn't exactly work as expected. It does encourage group-think, voting to "correct" moderation, voting down ideas you don't like. Hey, one of my last comments on K5 was voted 5 by some people and 1 by others... and no 3s at all. Politically motivated voters? Very obviously.

I think our biggest problem is thread hijacking, but I don't for a moment believe that this would be solved by thread ratings. More likely a highly-rated thread will be posted to more often - since it's highly rated, duh. I still think a greater number of forums will help since there will start to arrive topics that not all of us care for, encouraging diversity.

OK, that there is the tip of an iceberg on the topic; I could go on, but you'll stop reading anyway.

As for owning a thread - I don't see it that way. If I start a thread, it's because I thought of and/or acted upon it first. All that means is that I started it. Where it goes, the group controls. It's <b>everybody's</b>. Or at least, those who post in it. The only control the starter has is the ability to delete the thread within the first 24 hours.

As a "moderator" of a forum, the only control I have is to delete threads, move them, etc. I can't edit comments. I don't delete them either. But that's because it's "dhamsaic's weblog". Image of the Day, etc - those all belong to everyone. Tony is the administrator, and he runs the hardware, but beyond that, it's everyone's. I think that remains most true to the spirit of the Cellar (or at least the spirit that Tony has conveyed to me).

Catering to an audience - I still don't agree. On Slashdot, someone posts to boost their Karma. On the Cellar, we post because we have something to say. If we're not affected by the rating, we have no incentive to cater to it. In other words, the rating would be on the thread as a whole, which is "owned" by everyone. Obviously if someone started a rather troll-like thread, it would be marked down and probably forgotten - this is the only indicator I see of it affecting the poster. Otherwise, the rating would be indicative of the thread as a whole, not of any single post. That is precisely why I don't think we should have ratings of single posts - a thread is a thread, and all posts within it are integral to it. Some may be less insightful than others, but they're all there, affecting the thread. I'm getting off topic here, but basically, I don't believe that a rating system <b>of the specific threads</b> will cause pandering to specific crowds.

Of course having more people reading a thread <b>can</b> affect it, but it doesn't necessarily, just as having less people reading may or may not have an affect. It all depends on whether said persons decide to post or not.

I think Sycamoreland and my "weblog" are supposed to be just that - personal forums. When I approached Tony about it, my idea was to make a private, personal weblog - I like the Cellar software and I thought I could bring some new people to the Cellar (which I have). Tony thought the "private" idea was too much of a hassle, so we decided to go public with it. What is it? A place for me to post whatever I want, and for those that choose to do so to respond to them. Nothing more. It's focus is on my life, just like sycamore's is on his. I don't see a problem with this - they're fitting their intended goal.

Anyway. I guess we'll just have to agree to disagree on the whole rating of threads issue. I'm sure that if you could say something that would convince me it's a bad idea, you would have said it by now :) Just as if I were going to convince you of even its "okayness", it would have happened by now. I guess we'll just wait for others to weigh in...

Close one post and start another? Sure it makes sense.

After reading Mag's post up there, which wasn't here when I started my novel, it occurs to me that there are several options available to us.

In vBulletin there is a concept of forum depth, currently set to 2. "Cellar comments and suggestions" is a sub-forum of "Cellar-related". It would be possible, for example, to have dham and syc's and another other daily journal-esque forums on a THIRD level. So on the front page, you see "User-hosted forums", under which is "weblogs", but you don't see the individual weblogs until you click on "weblogs". This keeps the front page compact and easier to comprehend.

I mention the weblogs because it's simple to understand how they'd fit into the scheme. This idea would be perfect for the "cities" idea; you'd go into that section to find the chatter on the city forum of your choice.

This could combine with the concept of subscribing to forums and threads to make a pretty usable setup for those who want to read almost everything but not everything.

Remember, it was only about four months ago when it was cleanly possible to read everything... and it could well be four months before that's possible again. People's interests and desires shift like the wind when it comes to this sort of thing. If five of us were to suddenly stop posting, the system would quickly return to that status.

-

The other very interesting idea is quality selective moderation. As I noted, I'm not impressed with the quality of moderation seen at other systems, and especially with the moderation at forums that are smaller than K5 and /.

K5, in particular, is the fascinating one to me; Rusty tries his best to give all power to the users. I admire that tremendously, but I'm not as enamored of the result.

But what if you took the K5 idea and changed it from a Democracy, to a Republic? What if, instead of moderating every single post or thread, you elected a small set of moderators, who were directed to moderate on the basis of a small, understandable set of principles?

Let's say you elect five moderators, and tell them that the only basis for moderation should be post quality; no politics, no "correction", no voting for friends.

What if there were specific moderators for every forum here, and their only principle for voting posts down would be that they were off-topic?

I think you get my gist... site owners are reaching to technological architecture to solve all their problems, when they should be looking to innovative sociological solutions.

OK, now I've been high-falutin' here for so long that I'm gonna go pour myself a beer. My brain hurts.

Individually selected moderators may indeed be a good idea, as long as those moderators are trustworthy.

However, the whole notion of rating a comment as "offtopic" doesn't really appeal to me <b>unless the user is not punished</b>. Karma on Slashdot is silly. I wouldn't feel right moderating an offtopic post down only because it was offtopic, even if it were well constructed and fit in with the current discussion in its containing thread.

And where does one draw the line of offtopic? Are we still talking about rating threads here? Or are we down to moderating specific posts? Where is the line drawn?

Maybe it's best to leave the system as it is.

As far as setting a depth to 3 or whatever, that sounds fine to me - I wouldn't at all mind having a "weblogs" section and then myself under that. I also wouldn't mind having something like this:

User-Hosted Forums
|- dhamsaic
| |-weblog
| |-Quote Of The Day
|- sycamore

Where each user can have multiple forums under their control. I was thinking about adding a "Quote of the Day" thread to my forum, but it really works better as its own forum, much like IotD does. I really don't need to have separate forums under "User Hosted Forums" - what should be the best way to accomplish this? Another forum under "Main" that's moderated by dhamsaic himself? Or changing the way the Cellar works?

There's really a lot of thinking that needs to be done before we settle on any change. I'd like to hear ideas on what I've posted here.

Here's another thought that comes to me when listening to Maggie, dham and UT above:

What's the need for a field that names the Thread Starter?

The starter of the thread should just have the right to name the thread and make the first post. Then participate in the discussion, just like everyone else. I think that's what is intended.

But the way it is now leaves room for misinterpretation of the Thread Starter as "owner" or "moderator" of the thread. I hear all of you saying start it and let it go ... it doesn't belong to anyone.

So why does there appear to have the name of the Thread Starter "branded" on it for the entire life of the thread?

(Case in point ...IotD ... Be a person ... started by FreeYourself) (101 replies and he hasn't been posting anywhere since 01-02-2002)

As UT says, simpler is better ...

Even dhamsaic's and sycamores personal sites would look better. :) I mean ... what's that about? I know these two don't want to appear so egotistical. They aren't. Yet look at the presentation of their sites. It's enough to scare off participation.

IotD would appear more democratic. The threads would stand on their own merit, not viewed as UT's or Nic Name's or some others' threads. (For my part, I never wanted my name hung out there. I felt the same as dham and UT and the rest of you. It's not about the Thread Starter.)

For all we say about how the system is supposed to work, it really does look otherwise, from the current labelling of the Thread Starter.

dham, i think your quote of the day idea works best as a single thread in your forum, rather than a sub-forum. In your Quote of the Day thread you'd post your quote, and people might comment, and the next day you'd post another quote and people may read and reply or just read. I think it's unrealistic to imagine a great long thread of replies to each quote as an individual thread.

That's how I'd want to do it, anyway.

Where the additional level is very helpful is with the idea of Cities in the Main level, with the sublevel just having the proper names of the cities (that's where Philadelphia would be) Under each city name would be all the different Threads related to that particular city, each with its own title chosen by the first member to post to the thread, and get it started.

UT could add the real names of a sampling of cities to get it going, and then add other cities upon request in due course.

I think this would represent a natural maturation (god i hope i spelled THAT right) of the Cellar, unless you want to keep it in Cellar-related so that this seems like a local website.

Thread starter isn't useful now, when we wind up having 7 pages of thread that fly off-topic. It's more useful when people aren't chatting much and the original topic is kept through the thread. IMO.

Also right now, Subject is a wasted column.

But even then, it just identifies the thread with the starter, and I thought that really wasn't the point.

BTW, I don't even see the Subject column. Is that one you don't use, UT?

Too bad the thread pages don't all look as simple as the Cellar page, is all I'm saying.