Ravelry is still officially in “beta”. This word has been drained of most of its meaning in today’s web but here is what it means to me (and hopefully our users): the site is being very actively developed and things will change, unfinished “trial” features will be introduced, there will be bugs. In the last 365 days, there have been 241 releases where a new version of the code is pushed out to the production site.
As an aside – I’m looking into doing away with the beta label and having beta testing being an opt-in thing for users who want to be guinea pigs. We’re approaching 150,000 registered users and not everyone realizes that they often part of an experiment – this ends up creating more work for us in the form of questions (and sometimes, complaints)
Our users have been incredible – they report bugs of all kinds, come up with tons of great ideas and suggestions, enter things into our issue tracking system, and comment on existing issue. Here is how we do it:
Collecting feedback
Some sites collect feedback from beta testers with an email form. Don’t do this. All of your users need to be able to see and read the bug reports and ideas that everyone else is sending in. Working as a group is more efficient (less duplication), more productive (because small what-ifs can turn into great ideas through conversation) and more fun.
We set up one board in our forums that is specifically for talking about Ravelry itself. The board isn’t limited to feature requests and bug reports – it is a place for any kind of Ravelry talk like “Who has the biggest stash on Ravelry?”. In the last 30 days, 2300 people have posted on this board and 18000 people have read this board. hm… I guess I have to come up with some ideas to drum up participation – people who aren’t reading are missing out on their chance to shape the site.
Keeping Track
Our Ravelry discussion board contains 97,000 posts. We can’t use the board itself to help us track, categorize, and update all of the bug reports and suggestions. Even if we tried to do it by keeping the threads themselves excessively organized and moderated, we’d be sure to lose things.
So… we have an issue tracking system (The List) built in to Ravelry. Jess, Mary-Heather, myself, and 30ish other Ravelry users called “trackers” have the ability to turn posts in the forums into new issues in our system. When people post bug reports or suggestions we all try to either add them to The List or connect them up to existing entries. You can see that the post below has 1 agree vote and 1 disagree – sometimes the agree/disagree votes that are part of our forums can be helpful when we are looking at suggestions.
Now what?
When I am not actively working on them, the list items serve several purposes:
- Users can search The List to see if their suggestion or bug report has already been brought up
- Trackers can connect future forum posts to list items in case people have more or different information to share. This happens a lot and it is handy to have slightly different bug reports attached to bugs and further thoughts attached to suggestions.
- All users can comment on items and I read these comments when I set out to work on something.
Although I do have some basic prioritization type features to The List itself, I do all of my work-gathering and organizing outside of Ravelry. I tend to go into the list, gather a set of items that fit with what I am working on, and take those back to my virtual work area so that I can sort through them, read comments, and organize.
I’m really happy with this. I enjoy talking with Ravelers about the site and getting ideas and help from them. I’m also very glad that I don’t have to worry as much about losing good ideas and bug reports in a sea of posts. Ravelry Users: You can find the For the Love of Ravelry (FTLoR) board and a link to The List on the forums tab. As always – if you have ideas on how we can improve tracking and The List, we’d love to hear them. Just make sure to post them in FTLoR so that we can track them 
Comments (21)
This was extremely helpful — and inspiring.
On Ravelry, including FTLoR, I tend to read and often agree, occasionally disagree, but I virtually never chime in (unless something comes up that I vehemently disagree with! which is rare), because I feel like you and your partners in crime have so much on your plate already. So I feel like asking for more or better is — greedy! Since there are already so many planned enhancements as well as things we don’t even know about happening constantly.
But if you are actively soliciting comments on The List, feedback, other ideas — you got it! I really like the way you have the feedback loop, if you will, set up, and I agree that many minds are more powerful and creative than any one alone, as far as brainstorming. Plus one person’s strong opinion won’t drown others out in this exchange of ideas. (Well, unless it’s yours, I suppose, wink. Just kidding!)
Thanks for all you do; again, most interesting behind-the-scenes look.
OMG! Mah avatar is on Casey’s blag! I is famous!
This is a very cool and educational post.
thanks for this. I know you work hard and this just shows how much you care. rav user sandyknits
Ravelsourcing – is there any need for codemonkey interns/assistants? I am an Oracle DBA by trade – but am dipping my fingers into mysql. I’d be happy to volunteer. (However, I know Ravelry is your intellectual property, so I understand if you decline.)
It’s all google’s fault! Gmail is still beta after over four years. People tend to forgot what beta really means. I’ll be opting for the beta option (hopefully I’ll have the choice). You’ve done a great job! Thanks!
Thanks for the extended info Casey… I often wonder how the wheels are greased and turned on Ravelry… You have a great network here and I love all the tools you’ve added to the site… For certain, I’d opt-in for Beta! Thanks again!
I read separately that there might be a guinea-pig opt-in option on our profile pages, but if you’re tracking volunteers some other way, I officially volunteer to be a guinea pig. I would be able to give feedback on Windows, Mac, and iPhone OS’s.
Thank you, this was a very interesting read. I will, too, be a beta tester if I have that option, being part of the growth of Ravelry is such a fun ride, I wouldn’t want to miss it. Also to be given the chance to help out, even just a tiny little bit, is great.
thank you and congratulations, once again, for all your hard work Vero
Thanks Casey,
I will definitely be opting in for the beta, if i get the choice. I am sure that it is alot of work to keep things going, and i would be glad to help in any way i can.
Very cool—I appreciate knowing how you pull all the info together to frame the “problem” and make sure your solution hits the target. Looks like very very cool stuff.
This was very helpful… thanks. Much like the first commenter I rarely bring up (minor) issues because I know you have plenty to do and you’re still working on things (I do understand what being part of beta means). But if you want us to speak up I will. And I never used the agree/disagree button but if you’re also tracking that then I surely will. Ravelry is such a big place… sometimes it’s had to know where to go to post the questions and to find the answers. Thanks for the directional pointers.
Thanks for this Casey. It’s really neat to get an idea about how you work on the various updates for the site.
I like the idea of having opt-in for beta testers. That way it’s more productive/serious; as it is now, most of my compatriots wouldn’t care to report a bug/problem unless it was major. Sign me up.
I thought it was interesting to see how you track issues. There’s so much depth to Ravelry’s code and everyone does enjoy your little details (like the little icons for cake and wine).
I’ve been trying to behave and not cause you headaches. It’s so hard. -sigh-
I love the beta. I’ve been on plenty of development teams as monkey, tech lead, or systems analyst and know that as long as you are in beta you are still creating.
Folks love it, you are getting all sorts of testing and feedback, features are still flowing, long standing bugs are rarer and rarer… we would say Ravelry is now in its Master Bata period.
Let me know if you could use my opting-in
–Dancing Gardener
Casey – I love everything you do for Ravelry and this was definitely informative!
I read through some of your back posts and admire how much you know and care about being a “code monkey.”
Wow Casey Thank you for the very educational look inside of this wonderful community you and jess have created. Thank you for keeping us on track and up to date on the latest happenings.
Dear Mr Codemonkey,
You do a great job by keeping me from working (at work that is!). Have a banana.
Dare I ask: where are we on adding spinning? How do we (users) keep track of progress (if any) on a specific idea?
241 releases in 365 days? That’s crazy. How in the world do you do that?
…wait a minute. Superhuman abilities? You’re the final Cylon, aren’t you?
(Er. Sorry. The BSG geek just slipped out there. But still. I am vastly impressed!)
is rav down? Can’t get on today and had probs yesterday with both firefox and IE
Trackback/Pingback (1)
[...] Beta testing and beyond < Code Monkey Island [...]