How do you guys think the board submission / review system should work?

Should it be a group of reviewers who should all vote / comment on each submission and a majority vote or unanimous vote passed the board?

Or should any of the reviewers individually be able to review a board and pass / reject it?

Say we have 10 registered "map reviewers".

Once a map is submitted for approval it gets put into that waiting list for the map reviewers to look at. I don't think it's necessary that ALL of the map reviewers okay the map before it's accepted. That would take a while since not everyone is on at the same time. Likewise, I also do not think just 1 person could have the ability to okay a map.

I think 3 might be the magic number? Once 3 of your map reviewers give their "Approval" for a submission it then gets added to the public map pool.

Thoughts everyone?

... and where's Reich? He has been complaining about the Warfish map system longer than anyone, so he should be here to help make the decisions about the new map system.

You might want to do it the opposite, in the sense that X % of the map reviewer pool has to reject a map within X time or it goes live.

Or a combination, because that would mean your map would always take X time to go live. Hmm.

You know the mere existence of a review system will discourage a LOT of the stupid boards from ever being submitted for approval.

The main question I have is, "What sort of criteria must be met for the board to be approved?"

Do the reviewers need to play (and finish) an entire game on the board?
Do the reviewers merely need to assure that the board is intact and playable?
Do boards get rejected because they are "ugly"?

There is the possibility for merit and disdain for each of those questions.

If someone's design gets rejected, is there an appeal process?

I definitely think that reviewers must play at the very least one game on the board. I also think that development games should count toward that as long as they are on the release version, or whatever the development version is right before release. I don't know how exactly the versions system works, so I hope what I said makes sense.

I think Toaster raises good questions as to what doesn't qualify? You don't want to discourage people from making maps though either, but at the same time you probably won't want the Reviewers doing nearly everything for a map.

In 5 years what do you to be the criteria if the Review process is getting 10+ maps a week?

Early on perhaps just go by decent looks then create a more streamlined process for designers to follow?

How about when a board is submitted for review a thread gets automatically created in a private Board Reviewers Forum? Then the reviewers can discuss it and come to an agreement informally. No need for a formal voting system in that case. What do you think?

I imagine that would work, of course those that are a part of the review process should really probably play the map or at least give a very in depth look at it before commenting.
Although it still could be tough to Decline boards, a Decline would definitely need to be justified IMO. Which means you would probably still want some sort of guidelines qualify an acceptable board.

Toaster wrote: Do the reviewers need to play (and finish) an entire game on the board?
Do the reviewers merely need to assure that the board is intact and playable?
Do boards get rejected because they are "ugly"

No, takes too long.

Yes, as a minimum requirement.

Yes, if they are that bad.

I like 11's idea of including both an "accept" and a "deny" procedure in the process. Also builds in predictability in timeframe.

Yertle wrote: I think Toaster raises good questions as to what doesn't qualify? You don't want to discourage people from making maps though either, but at the same time you probably won't want the Reviewers doing nearly everything for a map.

In 5 years what do you to be the criteria if the Review process is getting 10+ maps a week?

Early on perhaps just go by decent looks then create a more streamlined process for designers to follow?

 I don't necessarily think a LITTLE discouragement is a bad thing. You don't want to stagnate your creative community, but you don't want the constant headache of neverending unacceptably bad maps.

Re "in 5 years..." if the community is that large, presumably you also have grown your pool of in-house designers and accepted reviewers, spreading out the burden of approving maps.

I agree that firm, public guidelines as to what's "good" and what isn't is a must. Communication between the map submitter and the review committee will also be important.

Oh, and I'd like to be included in the review team, please.

By board review I'm taking it you mean the submission process, and not a public rating system.

I'm a little concerned about too small of a group having all the say over what gets through and what doesn't. As we've seen from the core over on Warfish, there's a tendency to get a little too inbred and a little to artsy. The general public isn't necessarily as demanding that every board be cutting edge in graphics or gameplay. Even though I strive the other way in what I design, basic isn't bad.

What I'd like to see is a submission Purgatory. Once an author submits a map, it becomes publicly viewable in a separate map category: Purgatory, Limbo, Pending, Public Beta, Test, whatever you want to call it. While in purgatory, people are free to play open, unranked "test" games on the map. After a complete game, players are prompted to leave either a thumbs up or thumbs down and a textual review with comments for the author. A board will be held in purgatory until it receives at least 10 thumbs and the number of thumbs up is at least 10 more than the number of thumbs down. (as in, 9 up and 1 down is stuck in purgatory, another up and it goes live / another down and it needs 2 more ups to go live).

If you want to maintain a group of people with some administrative clout, make your review committee votes count x3. That way, 3 ups from the inside plus 1 outsider would be enough to pass the board. Or, a down from the inside counts as 3, and you'll need at least three public ups to offset that opinion.

If a map in purgatory gets 10 downs, then it is removed from purgatory and cannot be resubmitted until it is re-versioned. Sure, you could submit an unchanged new version of the map and spend eternity in purgatory, but whatever. If someone does that, use your administrative clout to remove it and warn them. Don't be afraid to refund and remove people who are abusing your system.

I like this system because it gives the public a way to override their patriarchs. It also gives those in charge enough power to quickly put down any maps that really suck before everyone else is subjected to it. Plus, it creates a period where lots of eyes can go over a board and check it for broken borders and little things that even the most experienced authors screw up. There's no reason a new board shouldn't have at least 1 full game played on it before it goes live.

Lol Reich, through me way off there for a bit.
Not a bad idea, although 10 may be a bit high (at least hitting 10 reviews on Warfish can be quite an accomplishment).

Cramchakle wrote: ...there's a tendency to get a little too inbred and a little to artsy. 

Reich, your outlook on beautifully enhanced map images hurts me deep... Graphics are a vital role in a map's design, and are the initial hook to catch a players attention. And if you couple great graphics with good gameplay, then you ensure that map's continued survival in the active games list.

"Oooooh, shiny pretty map... I want it..." is the general public opinion.

Also, I really like Reich's idea for map submissions. It strikes a nice balance between the power of Tom's Map Approvers and the power of the WarGear public.

Also, do we have to call you Cramchakle now? I'd prefer not to while at least everyone is on BOTH WarGear and Warfish... confusing.

By too artsy, I don't mean graphically. I more mean as in art-house appeal in the gameplay. My last several maps have been very complicated, BAO, border-modifier extravaganzas that only appeal to a few people. Whether they're good or not aside, 9/10 people on Warfish wouldn't even play them. 9/10 of the people who do won't play them enough times to really understand the depth of gameplay options. Some blame falls on me as a designer for ignoring the vast majority of gamers, but they weren't really my target audience. However, my target audience was the group of people who in Warfishland consider themselves the arbiters of what is good or not.

You can call me most whatever you want. Just dont call me Shirley.

Yertle wrote: Lol Reich, through me way off there for a bit.
Not a bad idea, although 10 may be a bit high (at least hitting 10 reviews on Warfish can be quite an accomplishment).

10 isn't very high when you consider that its only 3 if you have the support of the 'Map-approval Council'. If all of them play a game on it together, you could be through Purgatory in one game played.

Plus, if you can't muster 10 people or 3 insiders to simply click "Thumbs Up", maybe the map shouldn't be moved into the Live part of the site anyway. We're not asking for even as much thought as 1 through 5 stars. Just a 'this map is acceptable and not broken' or 'this map needs more time in the oven.'

i definitely think Cramchakle is on the right track. you should have a core group (who you feel will give a unbiased look at each board) who will have a higher value for there opinion.

maybe a option is that some one can apply to be on the review panel. they will be put in to the pool and when a board is submitted a random selection of people on the review panel will automatically be placed in a test game. once the game is finished every person must give a thumbs up or down. down must have a written comment attached with it.

unanimous thumbs up goes strait through and into the board pool available for normal play. any split decision then the board will be sent to tom for further review. a unanimous thumbs down then the board will be locked from play and the review comments will be sent to the author and then the author will have to resubmit a revised version of there board.

I like Cramchackle's idea. In fact, I'll one-up it and suggest that the "map-approval council" should be composed of the top 10 players on Wargear. This way, if people become inactive, they'll drop down, and it also gives a stronger incentive to play and stay up there.

Oatworm wrote: In fact, I'll one-up it and suggest that the "map-approval council" should be composed of the top 10 players on Wargear.

I would oppose that idea I think.  While the top players normally know a good map from a bad one, there are people that I think are better map designers and idea generators that I would say should be involved that would more than likely not be in the top 10.

It also takes a person willing to look at and/or play these possibly hideous designs and I'd venture a guess that the top 10 ranked players would not be happy with this responsibility. Plus, what happens if you're ranked #10 when you begin reviewing a board and then once you've given it enough of a look over to decide it worthy or not, you lose a game and are now #11?

Now, if the list is, "Top 10 Players - as determined by Toaster," then I wholeheartedly agree that is one of the greatest ideas known to man.

I like Reich's idea but I'm a bit concerned as to how this sort of system might be abused - e.g. with designer's having multiple accounts, creating a single 10 player game and then voting it through (as we've seen on WF before). Or even just  inviting all their mates to play and getting it voted to Live without a single other player or board reviewer getting a say in it.

KrocK wrote: maybe a option is that some one can apply to be on the review panel. they will be put in to the pool and when a board is submitted a random selection of people on the review panel will automatically be placed in a test game. once the game is finished every person must give a thumbs up or down. down must have a written comment attached with it.

unanimous thumbs up goes strait through and into the board pool available for normal play. any split decision then the board will be sent to tom for further review. a unanimous thumbs down then the board will be locked from play and the review comments will be sent to the author and then the author will have to resubmit a revised version of there board.

I quite like this option - the designer sets the review game size to x players and all reviewers get automatically invited to the game. Reviewers join the game on a first come, first served basis so not every reviewer has to review each game.

tom wrote:

I quite like this option - the designer sets the review game size to x players and all reviewers get automatically invited to the game. Reviewers join the game on a first come, first served basis so not every reviewer has to review each game.

Yes, this is good.