Post-mortem/meeting transcripts/2008-08-09/minutes

2008-08-09 | 2008-08-13

In which Angela wins a prize, Sue likes charts, even phoebe gets up in hours of hell, the meaning of meaning of mania is considered, delphine shoots sj, and Cary noted that some attendees are distracted by off-topic people. This meeting was determined to be public. General survey results will be public. Attendee surveys going out this week; specific subwriteups being claimed.

Attending:

achal        
austin
barcex       (quiet)
bastique     (moderating)
brassratgirl  
britty
damianfinol   
delphine
effe         
james_f      
mido         (quiet)
michaelsnow
patricio     (quiet)
_sj_         (scribing)
suegardner

Present but silent:

dungodung    (later)
hexacoder     as  abdel rahman
kibble        as  casey brown
WhiteCat
zuirdj

summary

Here is a shortlist of suggested things to do: areas that need some work / decision / clarification / public discussion. They are roughly in order of how much work has to be done by people at the meeting, rather than the larger community.

set post-mortem timelines (for 8-12 reports and 2-5 surveys)
- find people responsible for unclaimed reports, figure out who is to interpret survey results
- what role a retrospective, and proper archiving of older reports?
clarify scholarship process and timelines
- how/where will criteria for scholarships be set / how can selection and announcements be more open (considering privacy needs)?
- discuss coordination b/t foundation, sponsor, chapter, host-city and independent scholarships
resolve jury role and transparency
- can we clearly limit the role of a jury so that the community can take up the rest of the process?
- how can jury selection and deliberation be more open (considering privacy needs)?
- discuss w/larger audience options for bid selection (mentioned : a. jury --> open community vote; b. open community assessments --> jury; c. self-selected reviewers --> jury)
expand community involvement
- open discussion of these interesting questions to public audience
- discuss: what parts of criteria review & clerking can an open community group own? what to call such a group?
- discuss: /how/ to focus on community, how anyone can submit broader suggestions
clarify bid criteria, requirements, expectations
- publish templates and detailed requirements/questionnaires
- discuss also accessibility & rotation: how important is accessibility and cost? is rotation important beyond this?
- set expectations for bid teams (not just win/lose), jury members (work!), mania clerks/planners

meeting goals

what is the meaning of the meaning of mania?

The purpose of the meeting

Achal: is the purpose to come up with (a) a set of broad things around future wikimanias that we agree upon and (b) a plan to get them implemented?

some agreement, some passing those questions to community members (Getting a. from community surveys, and opening the discussion for b.)

brg: broad goals but I would also like to focus on 2010 if we can

<SueGardner> I tried to order the list, roughly, by how much discussion a topic seemed to provoke. So topic areas that seemed important to people should cluster at the top.

[NB -- discussion provoked is related to the shed-painting rule more than to importance. --Ed.]

Overall: agreement to focus on community; need to get to / talk about 2010. / more discussion of purpose and goals over time, a la MSnow's post http://lists.wikimedia.org/pipermail/foundation-l/2008-March/040636.html

Achal - decide whether we want a checklist to do its work pre and post meeting, that kind of thing. What actually goes on the checklist will be a much longer discussion (I am thinking)

Submitting broader suggestions

Angela :is there a process by which we/others might submit broade3r suggestions?

brg: we need to figure out a way to get the larger community feel like they can give input, also

<_sj_> process: can we turn this reflection on wikimania into a brainstorming exercise to help all attendees to reflect on what is most valuable to them and how to celebrate it with other wikimedians

<brassratgirl> I have been told that the wikimania judging process seems like "a black box"

<bastique> See the EuroWikimania link I put up, for instance. If there was a braoder process to do this, this event might get more input from others

<brassratgirl> by, um, someone who is very well connected in the foundation

<_sj_> finding out how to expand feedback beyond the scope of these meetings is important. make ongoing community discussoin enjoyable and with focus

there are lots of valid, useful goals for wikimania that conflit with one another

the meaning of mania

britty: we unanimously want a community event / eia: that means many things

bastique: central gathering space, for posters and booths and hanging out

sue/britty/brg: these should be in bidding criteria.

brg: with pictures and examples / mido: have 1.5 hrs or so to hang out set in the schedule

Achal - For how to rework Wikimania. Let's assume we know what we want to have done - my question is, should be we begin by thinking about how we intend to have that done vis-a-vis community, program and local teams - and then decide the details of what we want to have done?

Sue - Frank's new public outreach role could change the landscape re Wikimaia. Maybe we relaunch this conversation on foundation-l or wikimania-l | bastique: With the caveat that it will be a community based event | MichaelSnow : more than one list would be okay

sj - ...whether the goal of wikimania should be decided by a broad community and varied by bid teams each year, with focus left out of any bid judging criteria (or settled by community vote among the most competent bids) or whether it should be decided by a small group of people associated with past events...

Postmortem techniques and reports

Postmortem techniques: http://wikimania2008.wikimedia.org/wiki/Post-mortem

survey and reports from

a. attendees (questions compiled by del)
 http://meta.wikimedia.org/wiki/Wikimania_2008/Postmortem_Survey
 Delphine is working on it (via LimeSurvey), geting question feedback, to share Tuesday
b. speakers, (moderation / organization, questions by James and del)
c. sponsors, (who is making questions?)
   suggested : how they were treated / what went well / what they would like to see in the future.
   <patricio> may be the sponsors survey should be a personal interview with [each]... not so many 
d. organizers (is anyone making questions?)
x. casual survey of other community members (is anyone doing this?  
   suggested: why not attend?  would they have come if they could?  what would make them likely to?

stories : things learned / projects joined/discovered / animosities/edit wars resolved in person

question: microsurvey? no site notice. how much attention?

brg: cares most about those who attended. bastique: also prev attendes who did not come,

eia: and those who may come next time. also a good signal to non-attendees (thatit's not a closed group)

[nb: focus on those who had time and funds and general interes, but still didn't come]

reports from teams, for the future:

 0. reports and stats drawn from the surveys above
   0.1 - all attendees - Del?
   0.2 - just speakers - same as program committee?
 1. sponsors report   - Kul?  postponed to Wed discussion
 2. program committee - Lodewijk.  eia & jakob working on it, "will take some time"
    : brg - "Inviting Tim Berners-Lee: a five-part retrospective"
 3. local team / bid team - ?
 4. press team - Jay (int'l) / Abdel & Mido (local/nat)   
 5. foundation staff  - Cary to coordinate, touching on many other areas?
 6. scholarships team - ?
 7. local venue/host team (BA)  - anyone?  no consensus it's needed
 8. technical & backend process - reg, program tools, sending out mail 
 9. online team - ?
10. l10n team   - ? (britty mentioned kibble)

<SueGardner> Mido has been very polite and recessive as we all yak about 2008; it would be lovely to get his views :-)

<Angela> for team reports, having a meeting on irc followed by a write-up of that, would probably be more useful than trying to survey the teams. I expect more will come out of a discussion than people would think of separately

phoebe: work on a history of mania with feedback from other years
what info do we need? Achal: subjective and qualitative
- registration information : demographics - projects, nationality.
- scholarship information: some privacy issues, we can study at least this year if not more.
- noted: complaints that we have the 'same people' on scholarship many years. focus on WMF-facilitated schols, even though there are many others through chapters and companies.
- outside perception of the conf. PR success nationally and int'lly.

do we have a process for annual contributor/reader/participant survey [for Wikimedia projs, like UNU]?

Survey timeline, results, and questions

bastique: timeline and questions?

sue: everyone should fill out the survey, but additionally we should ask a few groups to provide an essay-like document with analysis & recommendations from their perspective. Yes?

<delphine> if we can flesh out the speakers questions by the end of the week-end I can send it Tuesday

Achal - people on this list should express their strong views formally; people with experience with Wikimanias right through

- organized, if not formalized

other postmortem fodder

a. sue: including the doc I sent last night and last week's conversation as postmortem fodder :-)

b. sj : a spot survey of editors in different projects and languages about wikimania, including whether or not they attended, assuming it is for all active members, not just those attracted today

britty - note which lang and project is their primary

c. eia: include report on software used for prog and reg | james - and other processes/supporting infra

<brassratgirl> - there's two parts to [most] reports, how this year went and recommendations for next year

Achal - we should have a checklist. This would serve TWO purposes, showing it upfront to the bidding teams (to flesh out the details of social space, etc.) - and evaluating the event. That would be fair?
- general support for this.

Speakers can be surveyed along with everyone else, with a link from the general survey; are already in the attendee db. (everyone agrees)

sj - what about feedback in various languages from people who wouldn't feel comfortable filling out an english survey? britty - add primary language as a question.

Report timeline, structure

No timeline specified yet; see above for the list of reports.

brg: For all things, get feedback from previous years as well via a retrospective

send a copy of the current survey(s) to previous years' org teams?

Angela suggested irc meetings to organize ideas for each team

<delphine> - can't we just skip the irc part? Have someone write a preliminary report, leave it on the wiki for a week. Add a personal comments section

Processing the results of surveys and reports

Who will take the survey results and do something with them? Can we assign responsibility for pieces?

Books of results
sj - a little blue book of people invited to wikimanias, with past correspondence (see TBL, TBL's secretary, Putin, parrot-owners)

ditto for sponsors and their motivations
Processing existing feedback from 2006/07
Practical notes (dos and donts), separate from essay like reports.
Achal - give sponsors (and others) feedback as well; sponsors might like a report on what attendees felt/learned/did | Sue: ask Kul

<eia> lets just leave it up for two weeks or so?

eia : this is also *the* opportunity to show to the outside world what our mission is

<delphine> Through PR, or through inviting the right emissaries?

<eia> through picking the right speakers and keynotes, through making noise in press releases, through having promotional activities around wikimania?

Scholarships

<Austin> to write up something on scholarships.

<SueGardner> I would like someone to take on the scholarships question - consult, write up recommendations :-)

<bastique> SueGardner: I have a lot to say about scholarships

<SueGardner> I think it would be reasonable for the Foundation to fund scholarships.

Cary: I think the community should manage the process, but the Foundation can fund it.

<brassratgirl> running scholarships -- need to involve more people (who are trustworthy, know what's going on); need to know how much money we have up front; and need to set clearer criteria

<delphine> I would keep chapters out of the overall scholarship process.

<James_F> Chapters should be strongly encouraged to do their own scholarships, but they're seperate from the WMF ones.

<brassratgirl> and we have 3 iterations of scholarship software, thanks to austin

<Austin> I am the only person to have participated in every scholarship process to date, anyway. :)

Sending people to regional events

<_sj_> by thetime you're talking $20k in scholarships to one region you could just throw a small local event with a very good connection to the global one

<Angela> as an alternative to scholarships that only allow a couple of people from Australia (or wherever) to attend, what about more Foundation support for local conferences? Why not send the Board and pay for other important speakers to attend not only Wikimania but also a second conference in a "less accessible" location each year?

<SueGardner> The Foundation committed this year to supporting the annual chapters conference. If people organize stuff and invite us, we would go :-)

<bastique> Angela: And I'll put my full support in helping an Australian conference

<brassratgirl> just so everyone knows: the Chinese are organizing the Chinese Wiki conference again for April/May next year, in either Hong Kong or Macau

<Austin> Angela's idea is a great one.

<MichaelSnow> brassratgirl: Thanks, I may try to be there

On bids and bid creation

Question: do we want lots of bids, or do we want to somehow encourage excellent bids?

brg: I think more than 2 or 3 bids is important

sj: the bid process is useful when expectations are set properly

SueGardner: How can we help people develop their bids? we can provide better templates, perhaps a support group for early bids

+general support

how to organie thoughts / consider timelines
who you should think about contacting
how to frame your requests so that you can still use goodwill for alocal conf if you don't win the wikmania bid, esp when pitching to sponsors

again, expectations

maybe earlier host groups could volunteer to mentor bidders during the bidding process?
examples of good bids : clear roles, clear goals, serious assessment of network, accessibility (including cost), potential for local collabs around the event, good venue prep
a bidder's "questionnaire" that guides them to anse3wr things that show up in the table

<SueGardner> As a juror last time, I found it difficult to compare bids because the information wasn't presented in an easily-comparable manner.

<DamianFinol> Like a table

<brassratgirl> ok, templates for the bids;

<brassratgirl> well, for instance, I think we were all impressed by Argentina's budget charts; but probably other bids could have come up with those

this checklist can overlap with achal's for criteria and post-conf review

<SueGardner> Yes, a questionnaire would help with comparability too.

<eia> I believe that we should allow people freedom too if they want to do something unexpectedly, sure

Bid criteria

<bastique> http://meta.wikimedia.org/wiki/Wikimania_2009/Judging_criteria <Angela> would adjusting the criteria lead to less controversy than we had the last couple of years. And does that matter?

<_sj_> there should be something explicit about being bold : a great novel idea for a bid should always trump tradition

: brg: but if we use these criteria and this process, we are putting some limits on the whole thing

britty: ... but we publish criteria and points for 2007, still a blackbox? <brassratgirl> yeah...

<eia> handle the timeline better (ie, dont extend once fixed etc, and dont cross deadlines as a jury)

<britty_> bruntly saying: wikimania is for me an opportunity to exchange real hugs: that's all

<brassratgirl> It seems like we are moving away from money/sponsorship being important?

<bastique> I think we want to have a conference be put on by the best people.

<brassratgirl> space-wise, I think we can focus on the community gathering part and sleeping and the airport as the biggest factors

<barcex> Let me say that in my opinion the most important criteria for selecting a bid is the bidding team.

<patricio> i want to reinforce barcex's point: I think one of the most important criteria is the bidding team

some agreement (sj)

<Austin> Actually, historically we've put up a page on meta and said "please edit the criteria for us" nobody wants to edit the criteria ahead of time. :)

<_sj_> lack of interest in the past may have been due ot how it was presented. I never quite felt it had the right tone

agreement (bastique, brg)

on rotation v. accessibility v. other

There was no agreement on rotation, but a long debate; with people suggesting many conflicting ways to implement rotation, and a minority who felt accessibility mattered but not rotation itself.

<delphine> the first and most important criteria should be Rotation. [at equal venues] rotation always wins.

[debate ensues]

<bastique> Rotation has played a BIG role

<Austin> We've never seen equal venues, though.

<delphine> don't agree :)

<brassratgirl> look, people are going to ask all of us, if they should even bother bidding if they are in Asia or Africa or South America. maybe a definite rule -- not the same continent twice?

<_sj_> I would say that accessibility and community balance is more important than rotation

<Austin> I know for my part, I would have no problem selecting a venue in, say, Rio, if was the best choice.

<DamianFinol> I'd say not the same continent as the previous two Wikimanias as a rule

<SueGardner> If the main purpose of Wikimania is to enable the core community to get together once a year, then surely accessibility & cheapness are the core crtieria, no?

some agreement (bastique, sj, austin)

<bastique> James_F: I think Chicago two years after Toronto is perfectly fine provided the venues are gerat

<_sj_> afaic, I think if someone comes up with a great cheap venue that is easy for people all around the world to come to -- the impossible mecca of all conference sites -- and makes it accessible to all cultures and languages, wikimania could be held there forever

<brassratgirl> so, back to frankfurt then? ;)

<Angela> I don't think you can define it in terms of km or continents. The 2 sides of Australia are more than 2000 km apart. And I don't see why having it in England one year should rule out anywhere in Europe the next year

<_sj_> in practice, location tends to discriminae so to balance that, there is some motivation for rotation

) location discriminates on visas, on language barriers, on cost

<bastique> Argentina, as it happens, tends to be better on visas than most

<eia> ok, can i summarize that geolocation plays an important role compared with the last two years, but not an absolute?

<bastique> Let me state that I think we need to work harder at bringing more people to Wikimania via scholarships

great agreement

<brassratgirl> 90% of the attendees will need scholarships if we have it somewhere very difficult, so that's something to consider.

<MichaelSnow> So if we keep turning down Australia, we should have a special Australians to Wikimania scholarship fund?

<SueGardner> Michael: yes, exactly. Seriously.

<_sj_> for some version of "we" .

define accessibility in a way that can apply to any wiki[m]edian Not necessarily a special fund, but a heavier weighting for .au / etc. people.

Jury process

[no consensus on what to do]

<brassratgirl_> we should write up a spectrum of possibilities; it's a little hard to keep track of

jury process connects with the question of "broader feedback" more than one conference, who decides the mission, etc.

Three ideas:

jury of community members, with mix of experienced and fresh ideas (a. larger than before / b. better defined than before)
community voting (a. to support a jury / b. supported/fed by a jury)
two-stage process, one for assessing/reviewing, another for selecting final bids

small jury of community members: jury selection

<bastique> obviously we want the community to input more on this

<Austin> You can't pin the past lack of community input entirely on the selection committee.

<_sj_> the first and most important criteria [for jury] should be Rotation. [confusion ensues]

I mean for jury members... shoot me for trying to make a joke before morning coffee. the selection committee personal opinions shouldn't have much to doi with identifying competent bids but public eprception is that it does.

jury membrership should be a bit time consuming, an important task, but no big deal

<bastique> I'm also believe that Mido or some other person from the 2008 team should be on the jury

<Austin> I agree that at least *attending* a conference and knowing what you're selecting a venue for is important.

<MichaelSnow> delphine: Sure, there are reasons to exclude individuals, but not having attended Wikimania before is a form of diversity, not a lack of it

brg, eia: In the past we've had the jury made up of:

previous organizers, because they knew something about wikimania
sue, to represent the foundation's interests
board members, also for the foundation's interests
these all seem like good things to me
and long time visitors

community voting, helped by / helping a jury?

<DamianFinol> A jury should weed out bids that are lacking, leave two or three strong bids. This gives bidders a chance to sort of present their bid at that WM

Also, a lot of WM attendees can ask questions to the bidders

<_sj_> maybe whatever we choose on voting, we can encourage people to show off bids 2 yrs in advance at wikimania

<bastique> and require that bids meet the criteria

<Mido> I don't think a community voting is a good way to select the venue at all

barcex - I agree with Mido.

<eia> nor do I think the community is eager to vote on it I prefer to select some community people

<DamianFinol> This is how I saw it: Bidding starts January 2009, bidders put up their bids, work on the, find sponsors, already have venues etc. Say 8 bids pop up. A jury picks the two or three strongest ones

Those two or three go to WM09, show their bids, receive questions from atendees

<Austin> You're asking someone to judge "is this venue the best fit for the conference?" without actually knowing what the conference involves.

<DamianFinol> You already had a jury to weed out bad, or lacking bids

<MichaelSnow> No one individual judges by themselves, the jury does it as a group I'm confident that someone who has never attended can learn from others on the jury what is generally needed

I'm not saying it's a dealbreaker, I'm just arguing that it's a factor.

the problem with jury, is it's definately not totally representative brassratgirl

damian: right. does it need to be representative?

As long as enough of the jury has experience, a fresh face or two would be a good thing

a bit nervous about community voting... With the community, it's more about "whether I can go" over "how do we make the best conference"

wikimania's attendees are not representative either: there's many local people

general agreement that putting it to a vote of one year's attendees isn't balanced.

sponsor someone from leading bid teams to attend wikimania? it's a lot of work to organize a bid, you learn a lot, the local community grows a lot. this would help people not feel tey've been totally left out

<Angela> perhaps just widening the jury would make the process appear fairer to those who were complaining it was unfair?

agreement (bastique)

<DamianFinol> I'll drop off bastique I'll certainly help if you guys ned

<eia> dror will drop off too iirc :)

[losing focus... long meeting...]

<brassratgirl_> I also do like the idea of community voting between the finalists

<James_F> DamianFinol> I think [voting at wikimania] that would (a) weight towards the populist over the best, (b) weight towards richer Wikimedians, and (c) assumes two (and no more) really good bits.

<brassratgirl_> and what's wrong with populist, precisely, if all the bids are reasonable?

some agreement (damian, bastique)

<eia> it favors the locals from last few years

<_sj_> I think most variations on voting would always lean towards picking a place that is cheaper to get to / closer / run by people from the largest editing communities unless we radically change how we tell / inspire peoplea bout wikimania

<brassratgirl_> and as I said, what is so terrible about that?

[whether we want this or not is another question]

separating groups for assessment and final bids

Have a large group, perhaps with an open call for participation, for general assessment; and to discuss/update criteria with community input. Then a small group to neutrally assess developed bids according to existing criteria.

<MichaelSnow> Two-stage jury? One to set the criteria, an expanded one to decide between bids?

<brassratgirl_> I want to be able to say to anyone: the people who made the decision are in that position because of ____"

<Austin> Phoebe: we had rationale for the members for the last jury.

<brassratgirl_> but *I* am also unhappy with the process.

<MichaelSnow> brassratgirl: I think an open call for volunteers might help with that

<SueGardner> What if the jury was split into two: a group that did the work of prepping the bids & criteria, and a group that made the decision according to the criteria?

<_sj_> separate groups is an interesting thought. the bid-prep and criteria-massaging group could be open to al. an open call for that group could just be amatter of people wanting to self-identify with the 'mania process

it /feels/ as though it's closed even if it is not

<Angela> is there some other way that people could be involved and useful without being on the jury?

Sue: yes, [if we open up the pre-process]

<MichaelSnow> It could also bring in people who are trying to open it up

<_sj_> the group that hopefully comes to consensus should be people trusted to assess neutrally a given set of criteria and not be swayed by nationalism, anger, friendship, &c.

<James_F> The "hopefully" led to a month's delay last time.

On community voting/assessment as /support/ of other deliberations: <DamianFinol> Instead of a 'vote' you can say what you think of certain criteria of a bid, Make it a survey

that would be great. non-binding, but specific focused commentary

Moderator work will be time consuming

let people who want to help do the legwork of researching prices and geting cites and local details do that

<brassratgirl_> what about asking people to help with evaluating bids? do the same work the jury does

Criteria setting / jury as neutral assessors

<SueGardner> (Incidentally, if there is a jury, I think it needs to agree to adhere to the criteria. Its job is not to reopen the criteria/secondguess them.)

<Mido> we should have clear criteria and a selected jury and give very clear explanation for the selected venue

maybe the last N years for continuity

<SueGardner> I think it needs to be understood that being on the jury is _work_.

<brassratgirl> well, re: attendees -- I have often thought that if some people who were dissatisfied with the jury realized just how boring and hard it is to sort through the <SueGardner> Like, ensuring bids are complete and understandable, measuring against criteria, seeking out additional info as required, etc.

<DamianFinol> Then maybe leave a few places so people can vote for people to get IN the jury?

<SueGardner> See, I think if the criteria are correct, the makeup of the jury does not matter much. Why should everyone have to be an expert in every bid? (Like, if we were to put it to a community vote.) Maybe the community should just vote on the criteria :-)

<Angela> they should definitely help to evaluate them, but that doesn't mean they need to be given a vote or position on the jury

<eia> you need to design (a broad survey along the lines of criteria) it in a way that the jury can still make an independent decision...

<James_F> eia> Informative, not binding.

jury process transparency

sj - do jury deliberations / assessments need to be private? are there important aspects of assesment that would be painful for bidders to see, requiring secrecy?

<bastique> no reason at all. I got this complaint from a number of the previous jurors

<SueGardner> it could theoretically get painful. The jury needs to make assumptions based on partial information, about the local team's competency.

<brassratgirl_> some things should be private, probably

<_sj_> [yes] a transparent review process imposes extra burdens on the jurors [to avoid being rude] but you could say the same thing about having private discussions on public talk pages.

<Austin> Another thing about openness is that confidential information could (and probably should) come into play. Lined-up sponsorships, budget information.

<bastique> the process can be transparent. Discussions can still be private

wrapping up

<SueGardner> I think we need to start thinking about wrapping up. Should we try a mailing list conversation for this, between now and Wednesday, maybe?

<Austin> Yes, I'd like to take this to the mailing list before the next meeting.