A couple of days ago I posted how I was disappointed not to have either of my sessions accepted for presenting at the PASS Summit 2011. While I was saddened, I understood how not every session could be picked and understood completely how difficult it is in choosing what would be the winning abstracts. A large part of this understanding was because I worked on one of the PASS Abstract Review/Selection committees this year.
I was one of three committee members who had it in our hands to help choose 29 Business Intelligence sessions. These were broken down to just a single half day session, 7 spotlight sessions and 21 regular sessions.
That’s just 29 sessions to be chosen (plus 6 regular and one ½ day alternate) out of over 180 submitted. Only 16% of the BI sessions would be accepted. Wow! That made things tough. Tacked on top of that there’s no speaker who could have more than 2 sessions, oh and anyone that got a pre-con is required to present regular or spotlight session. That’s a lot to tally and manage.
Tim Ford (blog|twitter) already talked about how this year the speakers were ranked by a team of volunteers. With these numbers in a nice new tool our committee went about scoring the abstracts (and abstracts alone).
Here are a few of the key factors that I looked at when scoring:
- A title and abstract that matched
- Well outlined pre-requisites and takeaways from the session
- Appeal to a good sized audience
- Well written and clear abstract
- Not a sales pitch
- Scores from the speaker preference tool
Each committee member individually scored a 0-5 for every session on three criteria and those scores were averaged and saved in the tool. As an example:
- Topic – 4
- Abstract – 5
- Subjective – 4
- Abstract Score: 13/3 = 4.333
The abstract score was added to the speaker score to give a final score:
- Speaker score: 4.600
- Abstract score: 4.333
- Total Score: 8.933
Once all of the committee had entered scores, we got together on the phone and via email to try and figure out which sessions would get picked. Even with the scoring this was not an easy task. Ensuring there was a broad range of topics as well as the other requirements on speaker limits made for some good discussion around things that should or should not be considered. In the end we completed our deliberations and finalized our decisions. All the sessions were chosen and everything returned to PASS for final approval. To their credit we only had a single change to make and that was because a speaker already had a couple of sessions in another track. That forced us to move up an alternate into a slot and promote another contender into the alternate position.
Overall it was a time consuming process and the general quality of the submitted abstracts was very high. The difference between a session being chosen and one not was very small. Be assured though every single session was given due consideration and thought. I made comments around each and every abstract I reviewed so that when it came to decision time I could quickly see why I gave the scores that I did. Believe me when I say that it was not an easy task and I know that there were sessions left out that I would have loved to have seen included but there were just not enough available slots.
Why I volunteered
This process was difficult and time consuming. So why did I do it? Last year I worked on the PowerPoint review committee and wanted to volunteer in some fashion again this time around. If you’ve not volunteered at something like a SQLSaturday event, PASS Summit or helped someone on Twitter or a forum then you’re missing out. The SQL community is one of the largest connected vibrant tech communities around. There are people from all walks of life from all over the world who work on different aspects of SQL and who have different skills and weaknesses.
When you first started out working with SQL Server was there someone that helped you? Guided you? Gave you advice, taught you the basics? Was patient with you while you worked your way through things? Have you gotten anything from reading blogs, books or attending events? Why do you think folks write those books? Trust me, it’s not for the money. Why would someone blog? I’m not getting a dime for this and rarely get feedback comments, yet I keep doing it. Why would someone speak at a SQLSaturday? Why would they travel to do so given that they don’t get paid? I’ve already travelled to Chicago to present this year and plan on hitting Atlanta in September. Why do these things? Because it is all about community.
Community: it’s why you read blogs (and why you are reading this post). It’s the reason you attend a SQLSaturday event or try to go to the PASS Summit. It’s the reason that local User Groups exist and that there’s a #SQLHelp tag on Twitter. It’s a large, passionate group of people that come together so that they can improve the collective knowledge. There are some great people with great ideas who’ve become great friends through the power of SQL.
Given that there’s this community rather than ask why I would volunteer to help out I’d like you to ask yourself why you wouldn’t volunteer.
Think about it and the next time you get a chance to help out at a local event put yourself out there. Trust me, it’s worth it.
8 thoughts on “Working on a PASS abstract review/selection team”
Nic, Thanks for sharing this – I hope that it encourages others to volunteer. You've been a great help the last couple of years and I'm glad that it was a good experience for you.Lori
I was on the PPT/Abstract review team this time around. It was very interesting. So far I have just done abstract reviews. It was very depressing to read all those abstracts and realize that I would probably not get to go the Summit to see any of them…:(
Thanks for writing this. It is a very difficult decision to choose from so many good abstracts. It is good to know that you understand that difficulty. Sorry you didn't make it this time around.
Nic, thanks for writing this and sharing more about the process. I'm sorry to hear you didn't get chosen to present. Has the committee considered publishing the more information, such as the scores, possible reasons for an abstract's rejection, or the order in which they were ranked? As a candidate, there's very little feedback to indicate what made your abstract get chosen or rejected, or how close you came. From a committee standpoint, I can imagine more transparency would lead to more complaints about why an abstract was given a certain score, ranked low, etc., such that the tradeoff isn't worth the added work and flak. Anyway, I appreciate how long and hard you all worked to choose high-quality sessions for this year's summit. Thanks.
Doug, we were asked to provide a some information wherever possible on sessions that were not chosen. Detailed feedback can be challenging to provide with such a large number of abstracts being reviewed by multiple people. I think requiring some improved feedback mechanism would be one of the small improvements for next year.
Shawn, the PPT review is a lot of fun, but exhausting work. I did that last year and it was great to see what all the presenters had in their slides before I went to the event.
Doug, we were asked to provide a some information wherever possible on sessions that were not chosen. Detailed feedback can be challenging to provide with such a large number of abstracts being reviewed by multiple people. I think requiring some improved feedback mechanism would be one of the small improvements for next year.-Karen Millen fashion shoes
It does not follow that because we do not subsidize smoking, we should not regulate unhealthy activities. Costs and savings are not the only variable. The fact that obesity creates costs is merely an additional reason to regulate it, not the only one. The main reason is its danger to an individual. You are dismissive of subsidizing smoking precisely because of this moral intuition. igxgwe igxgwe – Manolo Blahnik bottines.