Contri Levels and Rating Weightings
rickdugan
Verified and Certifiable Super-Reviewer
Now that I am starting to see this shake out in practice, I am less than thrilled with the notion that historical contributions to the site have suddenly been completely devalued. Guys who have squeaked out a relative small number of reviews of a small number of clubs are getting contri ratings that are as high or higher than people who have contributed a lot of good review content over the years simply because they have managed to squeeze out a certain number in the last 12 months, which really is not a great feat given the publishing standards around here.
I have over 5 years of VIP membership in the bank, so the only thing really motivating me to continue writing reviews (besides the simple joy of contributing) is the notion that I can have some influence on the overall club ratings for many clubs, which I and certain others have earned through years of contributions covering a large number of clubs. I realize that using weightings based upon a short-term contri level is designed to motivate long timers to keep contributing, but for me it could have the opposite effect. If some new person can write 1-2 dubious reviews per month and be granted the same level of influence over the ratings AND I am sitting on a lot of VIP time anyway, then what's my motivation besides of course my boundless altruism? ;)
And to put a less selfish frame around this, long time members with lots of reviews are more likely better equipped to judge the relative merits of a club, which is yet another reason to continue to grant them added influence over a club's rating.
Is there any way that the club rating algorithm could be tweaked to include member influence based upon some combination of the contri-level and number of clubs reviewed over time? It seems to me that this would be the best of both worlds - giving newer members a chance to earn some influence while continuing to motivate longtime members with a lot of VIP time in the bank to keep contributing.
I have over 5 years of VIP membership in the bank, so the only thing really motivating me to continue writing reviews (besides the simple joy of contributing) is the notion that I can have some influence on the overall club ratings for many clubs, which I and certain others have earned through years of contributions covering a large number of clubs. I realize that using weightings based upon a short-term contri level is designed to motivate long timers to keep contributing, but for me it could have the opposite effect. If some new person can write 1-2 dubious reviews per month and be granted the same level of influence over the ratings AND I am sitting on a lot of VIP time anyway, then what's my motivation besides of course my boundless altruism? ;)
And to put a less selfish frame around this, long time members with lots of reviews are more likely better equipped to judge the relative merits of a club, which is yet another reason to continue to grant them added influence over a club's rating.
Is there any way that the club rating algorithm could be tweaked to include member influence based upon some combination of the contri-level and number of clubs reviewed over time? It seems to me that this would be the best of both worlds - giving newer members a chance to earn some influence while continuing to motivate longtime members with a lot of VIP time in the bank to keep contributing.
Got something to say?
Start your own discussion
47 comments
SJG
So you can continue to bump your off topic threads that nobody reads or responds to?
Is a gun being held to your head forcing you to read them?
SJG
SJG
If the contribution level gained by leaving 3 reviews in a month outweighed long-term legitimate reviewers, that would amplify the level of harm.
But Trust Levels, I think they should not depreciate.
We need our old threads and posts, so that we are not always talking about the same stuff with new people, starting in a vacuum.
SJG
Son: A massive long termer like Papi and Shadow
But anyway, I'll stop having my tantrum in my high chair now, lol, and thank you again for everything that you continue to do here. :)
TUSCL is too-random for there to be solid data extrapolation - I agree that a PL's total body of PL-work is probably a good metric to go by, but this "may not" just mean longevity b/c there are lots of old accounts w/ little valuable data thus that should not be given much weight - again maybe total body of work including longevity (8" or more) perhaps will work well - but I still think there's too-much noise (garbage data) that can easily be given weight that shouldn't and why I don't feel weighting is the way to go - due to the randomness/flakiness of TUSCL, best to K.I.S.S.
Easy for you to say, Mr. Massive Long Termer
There isn't gonna be consensus on here, there hardly ever is except MJ being the GOAT - in the recent past the complaint was w.r.t. old data not being relevant, today it's about old data being relevant - pros/cons to either side
zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
oh, sorry, must have nodded off...
LeBron is the GOAT (in basketball).
SJG
It sounds like what we're really going for is a "writes legit reviews" flag. The Couchsurfing website back in the day had a vouching system where only people who had been vouched for three times could vouch for others. Maybe something like that, seeded by founder vouching for known-legit reviewers would be the ideal way to go. Then whenever a vouched-for reviewer writes a review, give that review the weight of an unvouched reviewer during its 12 months relevance period toward club ratings.
When it comes to CLUB REVIEWS, I personally give more weight to more recent ones (say, within the last 1-3 months) vs reviews over 3 years old. Applying the same metric to a reviewers contribution level is not an apples to apples correlation, as I feel many contributions over a long period of time should count for something.
I'm not sure that factoring in the number of clubs reviewed would be an accurate metric either. Say you have 2 reviewers who have reviewed 20 different clubs. One lives in a large city metro area, another is a travelling salesman from a small metro area but has reviewed the same number of clubs in 10 different metro areas. I would give more credence to the later who has a broader perspective on things.
I like the idea of recent reviews for clubs weighing more heavily in their ratings. I'm not sure I like the idea of recent reviewers weighing more heavily. Perhaps a balance? I'm not sure exactly how the logic would work, but perhaps something like a reviewer losing contri-level after being idle for an extended period but as long as they remain active I'd think I'd like to see their level remain high. The logic being #1 that long time reviewers are less likely to be trolls/shills, and that they have a greater depth/breath of knowledge.
I'll also add that an emphasis on a large number of reviewed clubs can be tricky. Minnow's example that he would give more credence to a reviewer who has reviewed clubs in many areas over a reviewer who has focused on one is a good example. I feel the exact opposite. When I'm reading reviews, I'm more interested a clubs relative merit vs nearby clubs over its relative merit over clubs in other areas. Not to say the latter isn't valuable, just not as valuable to me. A reviewer who has been to all the clubs in the area is going to help me decide which club to go to. I may consider one area vs another if I'm deciding to go to a SC at all in a particular area, but in truth we all know that's more of an inevitability. In short, I may feel a guy who has 50 reviews for 5 clubs has more useful info about those 5 clubs than a guy who has 50 reviews for 50 clubs.
_____________
Best comment in the whole thread.
I suppose I should start complaining now. :D