We have been hard at work now for a few months on the new features that will be coming in WordPress 2.9, and we are near the time when the first beta version will be available. We’ll need your help with beta testing the new features and ironing out any bugs.
There are a number of different ways in which you can get involved in the testing process, and each way is suited for each persons skill set and comfort level. First of all, you can join the wp-testers mailing list to keep up to date with the testing progress and to discuss things with the other testers. Secondly, you can head over to the Trac ticketing system and either create tickets for bugs you find or use some of the useful searches to look for patches that need testing or that need someone to try and reproduce the issue.
During the beta phase we are going to focus on stabilizing the new features and removing existing bugs which are well-understood and have easily testable solutions. During this process we will not be adding any new enhancements so as to ensure that the focus is on making the 2.9 release as bug-free as possible. We will also try and have a few special bug hunt days where one or more experienced WordPress developers will be available to help people track down issues and get patches committed to fix bugs.
To make is as easy as possible for you to get a beta testing install up and running we have put together a small WordPress plugin which makes it really easy to convert a test install of the latest release version of WordPress into a beta test install of the next up and coming release. The plugin is called WordPress Beta Tester and is available to download from WordPress Extend or can be installed using the built-in plugin installer. Please make sure you to only install this plugin on a test site. We do not recommend running beta versions on your normal, live sites in case anything goes wrong. You can read more about the plugin in “Making it easy to be a WordPress Tester”
We are aiming to release the first beta version of 2.9 around the end of October, after we have put the finishing touches on the new features. Then we switch to full on beta testing mode and your help and feedback will be very much appreciated. During the beta test program will push out new builds for automated upgrades regularly. Once we feel that a suitable level of stability has been achieved we will move into the release candidate phase. We hope to be able to make the final release 2.9 build available in either late November or early December.
One of the reasons WordPress 2.7 was such a success is the amount of usability testing that took place during the development cycle. Starting with testing 2.5 and the Crazyhorse prototype and following with the 2.7 beta, the testing program looked at almost every feature and function in the application. That kind of thing? Takes a lot of time.
For readers who aren’t familiar with the process behind usability testing, here’s an overview. First, determine the scope of your test and create a test protocol/script. Determine the audience segments to be included in the test group(s), and begin recruiting. Recruiting may mean hiring an agency to find participants, but for testing WordPress, it makes more sense to recruit from within this community, so that means making a screening survey, reading all the responses, segmenting the respondents into categories and contacting people until you’ve filled your desired quotas (for whatever segments you’re seeking, such as newbie, experienced user, developer, CMS user, photoblogger, mobile user, etc. ). Then come the test sessions.
Depending on what is being tested, these last anywhere from half an hour to an hour and half apiece. Sessions are generally recorded using screen capture and web cam, with a video camera for backup. The moderator(s) generally take notes during sessions and/or (depending on what software is being used for the session capture) set markers in the video to indicate task completion, comments of interest, etc. In some cases, auxiliary test methods such as eye-tracking may be included. When the sessions are complete, the results are analyzed. All the notes and videos are reviewed, patterns are identified, and ultimately a report is written and the feedback informs the next round of revisions.
Some people think it shouldn’t take much time to do all this. I’ve lost count of the number of people who cite an old article by Jakob Nielsen that says you don’t need to test with more than 5 users because usability issues become clear right away. While I’ve found that to be generally true, when your user base is as diverse in experience level, usage, platform configuration, language (right to left languages have a pretty different experience) and demography as the WordPress community is, 5 users really isn’t enough to get a clear picture. We try to test with at least a dozen people each round, but then we are limited to a geographic region (test in NY, test in SF, or test wherever we can schedule enough people back to back to make it worthwhile), while WordPress users are located all over the globe.
To address this, we’re introducing a set of new contribution opportunities to expand our usability testing program. As with development and graphic design, we’re going to create an infrastructure to allow community participation in usability testing on a regular basis and in a much broader capacity than existed before, when it was limited to announcements that we needed participants in x city on y date. We’ll be looking for volunteer moderators as well as participants, hopefully from all over the world.
Moderators. Observational usability testing isn’t rocket science, but neither is it a simple task to reduce bias. Because of this, at first we’ll choose only moderators who have professional experience conducting usability tests. People who conduct testing for design agencies, software companies, usability consulting firms and the like will be our first round draft picks. In the future, when we have a group of regular volunteers and have ironed out any kinks in the process, we’ll ideally match up experienced testers with aspiring ones, using a mentorship model to increase the number of people who can contribute in this fashion.
Participants. If you use WordPress, chances are you could participant in a usability test at some point. In some cases we’re looking for particular behaviors (people who upload large video files, people who blog from their iPhone, people who manage more than 5 blogs, etc.), while other times the behaviors we’re looking for are much more common (do you have widgets in your sidebar, have you changed themes in the last 6 months, is there more than one author on your blog, etc.).
So how will these opportunities come into play, and how will it make WordPress better?
We’ll start with the moderators, and try to get volunteers with a decent geographic spread. Then, we’ll start signing up potential test participants in those areas (though we’ll also allow at-large registrations, since traveling testing will still be happening). We’re working on a registration process for potential participants. You’ll enter your basic info (location, contact info) and answer some questions about your WordPress usage to be entered in the database, and when there’s a testing opportunity coming up that’s appropriate for you, a local moderator will get in touch to see if you’re interested. Further down the road we may experiment with remote testing and other methods, but for now, this approach will broaden the geographic scope of our testing.
All moderators will follow the same test protocols and script, and their results/reports/video will be reviewed and collated, with a composite report (including the protocol/script that was used) published to the community. This will provide designers and developers with broader feedback during the dev cycle, and will allow the wider community to both understand and participate in the testing program.
If you’re interested in being a moderator during this initial phase (meaning you do it professionally), send me an email and introduce yourself. If you’re interested in signing up as a potential test participant, watch this space. We’ll post a link to the registration survey once it’s ready.
A question I hear pretty frequently is, “Why a redesign of the admin panel so soon after 2.5?” Those who have attended WordCamps in the past few months have already heard the answer, but for the people who haven’t had that opportunity, this post is for you.
When the community response to the 2.5 admin redesign was mixed, it seemed like a good idea to do usability testing to find out which issues were based on actual interface problems vs. which complaints were just a result of not liking change. To prevent bias, a third party was contracted to conduct usability testing, Ball State University’s Center for Media Design, Insight and Research division. Try saying that three times fast with a mouth full of peanut butter. Or fitting it on a business card. To save time, we’ll just call that third party CMD, since that’s what they call themselves.
The plan that was developed involved multiple rounds of testing, as well as the creation of two prototypes, hardcore! The first phase involved a usability review of 2.5 by CMD, the results of which were discussed with lead developers. A quick prototype was created that addressed some of the lightweight issues, so that the test participants could use both 2.5 on their own blogs and the prototype on a test site. Results would be analyzed and compared, leading to a second round of suggestions. A second prototype would be developed over a week or two, which would then be tested with the same participants, and a final report delivered. But you know what they say… the best laid plans of designers and developers often go awry.
After the first round of testing, it was clear that a prototype delivering the kind of fixes that could be coded in a week or two wouldn’t make much of a difference overall. We all decided a more ambitious prototype was in order, one that would experiment with a new approach to screen real estate and attempt to address as many of the issues from 2.5 as was possible with a few extra weeks of time. A rapid design process was followed by an even more rapid development cycle. The second prototype is what you know as Crazyhorse.
The second round of testing blew everyone away. The research team had never seen such consistent results. Tasks were completed faster, participant opinions rated it higher, understanding of how interface elements worked was greater, and it wasn’t even a fully functional application. Of the test participants, every single one said they would choose the prototype over their current administrative interface, and it wasn’t even pretty (those of you who remember the original Crazyhorse will vouch for this).
A presentation on the process from start to finish was part of the schedule at WordCamp 2008 in San Francisco, and the slides are available online, but as always the slides only tell you so much without the videos, live demo and verbal narration that went with it. (Use Google and you can see audience videos of the presentation.)
Here, then, is a PDF of reasonable size that you can download and peruse at your leisure that outlines the usability testing project in some detail. I wanted to include some eye tracking videos, but the file was so huge it would have been ridiculous for anyone to download it, so I stuck with eye tracking outputs called gaze trails to illustrate the findings. I also tried to pare down the text to the more salient points, since more than 50 hours of test video really does reveal an insane amount of data. I also cut out the section about designing Crazyhorse in the interest of staying under 25 pages. Hopefully you’ll think it’s a good balance. I’ll try to put together a separate document on the design process of 2.7 in a couple of weeks that will include the early Crazyhorse material.
So, if you want to know what we learned from the usability testing this summer that caused us to create what is now 2.7, go ahead and read the report.
WordPress 2.5/Crazyhorse Usability Testing Report (PDF)
There are two contests going on in the WordPress community right now. If you’d like a chance to flex your WordPress skillz and perhaps win a prize and lifelong fame, you should consider dropping your code in the hat.
The first is the Sandbox Designs competition, which is like a theme competition, but working only with CSS and the highly semantic Sandbox theme. They already have a thousand dollars in prizes, so check it out.
Second our friends at Weblog Tools Collection are running a WordPress Plugin competition. They have a blog to track all the entries, but if you’re participating don’t forget to submit your code to the Plugin directory.
Both competitions require entries to be under the same GPL license that WordPress is, so regardless of who wins they’ll make the entire community much richer. (Remember, WordPress itself was written on the base of existing GPL code!)
Over at my site I’m running a small contest for people to create alternative color schemes for the administration interface. When it’s done I’ll pick a few winners and package everything up in a plugin you can use to spice up your administration a bit. If you have a few free minutes try your hand at it. There will be a few prizes for the winners, including a copy of Color Schemer Studio.
Just a summary of some of the things I’ve been working on today.
- The RSS 2.0 feed now supports multiple categories. Not sure how to do it for 1.0 yet, looking into it.
- The includes stuff now uses constants instead of variables. This means if you’re on the bleeding edge of the CVS you’ll have to update your wp-config.php again. Sorry! I anticipate this being the last change, ever.
- The renaming is just about complete, with outdated variable references being removed.
- Quicktags now enter whitespace with list markup to format it nicely.
- Error messages are a lot smarter, espescially when first starting out. It now tells you the problem exactly if there’s not a wp-config.php file, or if you have any problems with your database connection info. It offers a few common solutions and links to the support forums. It also doesn’t generate a thousand errors like it used to.
- The default index.php has been updated to clarify some concepts and have the current version number in the generator meta tag.
- readme.html has been updated to remove some wrong and old things. Needs a lot more work, but hopefully the wp-docs group will get this to be a really great document.
- The wp-links directory has been eliminated and the files in it have been moved to wp-images or wp-includes.
- Cleaned a final few include problems. The including system is so much cleaner now.
- The “Blog this” bookmarklet link has been moved to the posting screen and renamed “Press it”.
And I’m done for the night. More tomorrow…
Before we do a release to the world we really need some testing on the .7 beta, which is now available as .zip and .tar.gz in the beta directory.
If you have any problems or comments you may contact me directly or post to the new Beta forum.
Thanks for your help.
WordPress 4.0 Beta 2 is now available for download and testing. This is software still in development, so we don’t recommend that you run it on a production site. To get the beta, try the WordPress Beta Tester plugin (you’ll want “bleeding edge nightlies”). Or you can download the beta here (zip).
For more of what’s new in version 4.0, check out the Beta 1 blog post. Some of the changes in Beta 2 include:
- Further refinements for the the plugin installation and media library experiences.
- Updated TinyMCE, which now includes better indentation for lists and the restoration of the color picker.
- Cookies are now tied to a session internally, so if you have trouble logging in, #20276 may be the culprit.
- Various bug fixes (there were nearly 170 changes since last week).
If you think you’ve found a bug, you can post to the Alpha/Beta area in the support forums. Or, if you’re comfortable writing a bug report, file one on the WordPress Trac. There, you can also find a list of known bugs and everything we’ve fixed.
WordPress 4.0 Beta 1 is now available!
This software is still in development, so we don’t recommend you run it on a production site. Consider setting up a test site just to play with the new version. To test WordPress 4.0, try the WordPress Beta Tester plugin (you’ll want “bleeding edge nightlies”). Or you can download the beta here (zip).
4.0 is due out next month, but to get there, we need your help testing what we’ve been working on:
- Previews of embedding via URLs in the visual editor and the “Insert from URL” tab in the media modal. Try pasting a URL (such as a WordPress.tv or YouTube video) onto its own line in the visual editor. (#28195, #15490)
- The Media Library now has a “grid” view in addition to the existing list view. Clicking on an item takes you into a modal where you can see a larger preview and edit information about that attachment, and you can navigate between items right from the modal without closing it. (#24716)
- We’re freshening up the plugin install experience. You’ll see some early visual changes as well as more information when searching for plugins and viewing details. (#28785, #27440)
- Selecting a language when you run the installation process. (#28577)
- The editor intelligently resizes and its top and bottom bars pin when needed. Browsers don’t like to agree on where to put things like cursors, so if you find a bug here, please also let us know your browser and operating system. (#28328)
- We’ve made some improvements to how your keyboard and cursor interact with TinyMCE views such as the gallery preview. Much like the editor resizing and scrolling improvements, knowing about your setup is particularly important for bug reports here. (#28595)
- Widgets in the Customizer are now loaded in a separate panel. (#27406)
- We’ve also made some changes to some formatting functions, so if you see quotes curling in the wrong direction, please file a bug report.
If you think you’ve found a bug, you can post to the Alpha/Beta area in the support forums. We’d love to hear from you! If you’re comfortable writing a reproducible bug report, file one on the WordPress Trac. There, you can also find a list of known bugs and everything we’ve fixed so far.
Developers: Never fear, we haven’t forgotten you. There’s plenty for you, too – more on that in upcoming posts. In the meantime, check out the API for panels in the Customizer.
Media, things in between
Please help look for bugs
Older Posts »