In another thread, Ozyman was asking how members get on the review board, and I had a thought.  As it currently stands, players have to ask to be on the board.  I think for the time being this is fine, but partly with the idea that we need more members, and partly with the idea that we need qualified members, I propose that anyone who has submitted a board for review that has passed is automatically placed on the board, with the option to decline membership.

i wouldn't be opposed to this if there were a minimum of 2 or 3 boards released. I don't think one and done is enough to autofill you into being a reviewer.

weathertop wrote:

i wouldn't be opposed to this if there were a minimum of 2 or 3 boards released. I don't think one and done is enough to autofill you into being a reviewer.

Two seems reasonable. Right now, technically one doesn't have had created any boards to be eligible.  Part of my concern was that there are a number of potential reviewers out there who have no idea that they could be part of the process.  As far as I'm concerned, making it through the "gauntlet" even once gives one quite a bit of expertise.  You don't have to be an expert map maker to be a good reviewer. 

That said, maybe we could start with a minimum of two boards (which would apply to all existing designers as well), and see how many participate.  If we still need more, drop it to one.   I think tom should also continue to consider requests from non-map makers on a case-by-case basis, with a bias toward strong players who are better equipped to evaluate a board's mechanics and fairness.

I don't think we should do it now but I'll just throw the idea out there.. Additional invitations (or autofills) could be issued to all premium members when they reach a certain achievement, like a 1900 global rating, or 30 championship points, etc.

I think it's a great idea to automatically invite any mapmaker to the board that has passed at least 2-3 boards. I think 3 is better, but 2 is ok also.  Additionally if someone releases a really stellar first board  (like for example Yuma), the board reviewers could always recommend an invitation directly.  I also like the idea of having well ranked players get invitations.  Especially using championship points as that points to a player that is good at several boards.

M: anyone can be eligible, but to automatically put you on the list is where i was going with that.

it would be interesting to allow all non-host reviewers to invite others at their discretion. not sure if that's as easily implemented tho.

To be fair, we need good players on the review board more than we need mapmakers. Sure, fellow mapmakers may (not guaranteed) have a better idea of the abilities and limitations an author might be working with; but the review game is more about proving that the map is playable, fair, and fun.

We want other mapmakers to be involved in the DEV community, before something lands at the feet of the review crew.

Cramchakle wrote:

To be fair, we need good players on the review board more than we need mapmakers. Sure, fellow mapmakers may (not guaranteed) have a better idea of the abilities and limitations an author might be working with; but the review game is more about proving that the map is playable, fair, and fun.

We want other mapmakers to be involved in the DEV community, before something lands at the feet of the review crew.

Cram makes some very good points here. I would only slightly shift the emphasis to say that you need good strong players just as much in Dev.  It almost seems like what we're headed for is a two step review process where the bulk of the work is done in Dev.  When you think about it, reviewers who play a board for the first time often miss a lot of its nuances, bot good and bad.

isn't that the way it currently is supposed to be? (a 2-step DEV, REV)

Why is there a distinction between the Development community and Review Board?  What are the responsibilities of one vs. the other?  If they could be combined in a clear process that would be helpful.

1) My personal opinion (similar to what Cram said) is that you don't have to know the board making tools to be able to review a board.  It certainly helps to have someone that knows board making well enough such that when criticisms are posted, that there are people available who can aid the board maker in ways to fix the issues.  But once that board is in review, if the image/fill/layering issues haven't been fixed, then the board has moved to review too early and should be pushed back to DEV status.  Review should be nothing more than a final stamp of approval, not a "what else do I need to fix" stage, and therefore as long as the person is a well versed in the mechanics of WarGear, they could be qualified to give that board a stamp of approval.

2) If anything, I think that the ball is getting dropped in the education of the new board makers.  The problems I've seen are that they board makers aren't realizing the purpose of DEV games to work out the kinks/images/design flaws/bonuses/cards/description/etc.  I might propose that a board isn't able to be moved from DEV to Review without nomination (of sort) from a member of the review board.  This forces the board to stay in DEV until someone has OK'd it to move to review.  And also puts the onus on the review team to only push boards into review that are worthy of review.

To save any work for Tom, it could be an informal nomination:
DEV games are played and terminated until the map maker think it's ready
In a "final" DEV game, map maker can request a reviewer to nominate
(I'm kind of assuming that most/all DEV games have at least one Review Board member in them)
When the review game is created, map maker can link to the game where the nomination happened.
If someone posts a map to review without the link the review board can send that board back to DEV status until the issues are worked through.

As an aside:   Feel free to take my opinions with a grain of salt, as I think I might be the only, or quite close to the only member of the review team who ISN'T also a map maker, nor am I anywhere near the top 100 players on the site (by either major metric).  So if this makes the map makers uncomfortable, and the decision of the group is that Map Makers/Top Players make the best Board Reviewer, I have zero problem with stepping down from that role.  

BorisTheFrugal wrote:

1) My personal opinion (similar to what Cram said) is that you don't have to know the board making tools to be able to review a board.  It certainly helps to have someone that knows board making well enough such that when criticisms are posted, that there are people available who can aid the board maker in ways to fix the issues.  But once that board is in review, if the image/fill/layering issues haven't been fixed, then the board has moved to review too early and should be pushed back to DEV status.  Review should be nothing more than a final stamp of approval, not a "what else do I need to fix" stage, and therefore as long as the person is a well versed in the mechanics of WarGear, they could be qualified to give that board a stamp of approval.

2) If anything, I think that the ball is getting dropped in the education of the new board makers.  The problems I've seen are that they board makers aren't realizing the purpose of DEV games to work out the kinks/images/design flaws/bonuses/cards/description/etc.  I might propose that a board isn't able to be moved from DEV to Review without nomination (of sort) from a member of the review board.  This forces the board to stay in DEV until someone has OK'd it to move to review.  And also puts the onus on the review team to only push boards into review that are worthy of review.

To save any work for Tom, it could be an informal nomination:
DEV games are played and terminated until the map maker think it's ready
In a "final" DEV game, map maker can request a reviewer to nominate
(I'm kind of assuming that most/all DEV games have at least one Review Board member in them)
When the review game is created, map maker can link to the game where the nomination happened.
If someone posts a map to review without the link the review board can send that board back to DEV status until the issues are worked through.

 

As an aside:   Feel free to take my opinions with a grain of salt, as I think I might be the only, or quite close to the only member of the review team who ISN'T also a map maker, nor am I anywhere near the top 100 players on the site (by either major metric).  So if this makes the map makers uncomfortable, and the decision of the group is that Map Makers/Top Players make the best Board Reviewer, I have zero problem with stepping down from that role.  

I don't think the process is the problem here. (Help: http://www.wargear.net/help/display/Board%20Review)

IMO, with the DEV community working the bulk of the suggestions/bugs out of the board, the Review Board should be the best of the best in either gameplay or cartography or both.  Otherwise, what qualifies them to have final approval?

From the sound of the original thread, it sounded like there were problems getting enough people in Review games.  Is this right?  If so, I like weathertop's suggestions for getting more qualified folks aware and on the Review panel.

I think that a board should have at least 5 games played in DEV before the Submit button is even available.  Or maybe it is red with a tool-tip that says not enough Dev games played.  I think that new Map Makers are seeing Submit as the first step instead of the last. 

I am on the fence with the nominate to review idea.  It is sometimes hard to get a board pushed through the review/pass process. I am not sure that a dev/nominate/review/pass process is going to be better. 

Maybe a helpful automatic email/PM when someone creates a new board.  It could include some encouragement, suggested process and resource links.

I agree with you, Boris, on point 1 completely.

I think point 2 is well intended, but a little off-base. First, I don't know anyone who's first few maps were a solid combination of good looks and good game-play. There are plenty who nailed one or the other, but it can take a long time to get both. My point being that even a new mapmaker who fully leverages the existing Development community, asks a million questions, memorizes the FAQ, and sells his soul to RiskyBack is probably still going to churn out a map or two that don't delight the senses and tickle your fancy. But you're probably right that either by ignorance or negligence there are some who aren't looking for the help that is there.

I take umbrage at the idea of having to have a map nominated out of DEV status, though. My last two published maps each had exactly 1 Dev game played on them, and I only bothered with that because I felt it was due diligence. I think Steel City might even play better than Castles, which the game-play is based on. I guess there comes a point in an author's career where he can spin a map totally from the gut. I also trust that if someone's made that point, they also probably know when they need to stop and ask for help, and I don't want to burden such authors with another approval step.

The easiest way to maintain the 2-step process is that if someone submits a sub-standard board, give them a playful swat on the bottom and a link to the Board Designers forum while swiftly rejecting the board back to DEV status where they can get some help. That would eat up all of 10 cumulative seconds of the entire Review team's time while also pointing the mapper towards help.

tl;dr - status quo is fine

AttilaTheHun wrote:

From the sound of the original thread, it sounded like there were problems getting enough people in Review games.  Is this right?  If so, I like weathertop's suggestions for getting more qualified folks aware and on the Review panel.

Should probably just make a post in the general forum asking for volunteers and then vet them as a community based on what they think their qualifications are. Are they an accomplished author? Do they have a high ranking or lots of tournament wins? Are they active on the forums and prompt on their turn-taking?

This isn't parliament, not everything has to be super formal.

Cramchakle wrote:

This isn't parliament, not everything has to be super formal.

I would like to request that the remarks from the gentleman from Scotland be stricken from the record.  Following bylaw 14, subsection C-2, super formality is encouraged, and ultra formality is allowed, moderate formality is required and at the bare minimum decorum must be upheld and extensive falderal employed.

Ozyman wrote:

Cramchakle wrote:

This isn't parliament, not everything has to be super formal.

I would like to request that the remarks from the gentleman from Scotland be stricken from the record.  Following bylaw 14, subsection C-2, super formality is encouraged, and ultra formality is allowed, moderate formality is required and at the bare minimum decorum must be upheld and extensive falderal employed.

..do I hear a second?

see even Cram agrees with me. what a birfday treat!

i think the current process works 98% of the time. for those 2% we get someone new who submits right away and we handle it well  by offering some critique, telling them they need some DEV games, and failing the review. some take the next step and continue on. i've seen a couple give up and leave. but that's fine with me too. means they weren't going to stick around anyway.

i still don't have a problem increasing the review team, if others think it's needed. tho i'd prolly take a poll of the current members to ask if they feel they're handling it well or need help.

I think it's time we revisited this. there's a lot of current review members on hiatus. I think we need to select another half-dozen to add to the list. there's been a couple boards lately that sit for a couple days waiting for enough review members.

I'm back to a more full time playing status and I'm bored with most of the current maps so if tom sees this I would be interested in being included in the review process.

I'll apologize for having been MIA for the last 2-3 boards that came up, but I knew I was going to be on vacation a few times, and didn't want to impede progress.  I'm back now, and ready to fail boards left and right!!