We have been hard at work now for a few months on the new features that will be coming in WordPress 2.9, and we are near the time when the first beta version will be available. We’ll need your help with beta testing the new features and ironing out any bugs.
There are a number of different ways in which you can get involved in the testing process, and each way is suited for each persons skill set and comfort level. First of all, you can join the wp-testers mailing list to keep up to date with the testing progress and to discuss things with the other testers. Secondly, you can head over to the Trac ticketing system and either create tickets for bugs you find or use some of the useful searches to look for patches that need testing or that need someone to try and reproduce the issue.
During the beta phase we are going to focus on stabilizing the new features and removing existing bugs which are well-understood and have easily testable solutions. During this process we will not be adding any new enhancements so as to ensure that the focus is on making the 2.9 release as bug-free as possible. We will also try and have a few special bug hunt days where one or more experienced WordPress developers will be available to help people track down issues and get patches committed to fix bugs.
To make is as easy as possible for you to get a beta testing install up and running we have put together a small WordPress plugin which makes it really easy to convert a test install of the latest release version of WordPress into a beta test install of the next up and coming release. The plugin is called WordPress Beta Tester and is available to download from WordPress Extend or can be installed using the built-in plugin installer. Please make sure you to only install this plugin on a test site. We do not recommend running beta versions on your normal, live sites in case anything goes wrong. You can read more about the plugin in “Making it easy to be a WordPress Tester”
We are aiming to release the first beta version of 2.9 around the end of October, after we have put the finishing touches on the new features. Then we switch to full on beta testing mode and your help and feedback will be very much appreciated. During the beta test program will push out new builds for automated upgrades regularly. Once we feel that a suitable level of stability has been achieved we will move into the release candidate phase. We hope to be able to make the final release 2.9 build available in either late November or early December.
One of the reasons WordPress 2.7 was such a success is the amount of usability testing that took place during the development cycle. Starting with testing 2.5 and the Crazyhorse prototype and following with the 2.7 beta, the testing program looked at almost every feature and function in the application. That kind of thing? Takes a lot of time.
For readers who aren’t familiar with the process behind usability testing, here’s an overview. First, determine the scope of your test and create a test protocol/script. Determine the audience segments to be included in the test group(s), and begin recruiting. Recruiting may mean hiring an agency to find participants, but for testing WordPress, it makes more sense to recruit from within this community, so that means making a screening survey, reading all the responses, segmenting the respondents into categories and contacting people until you’ve filled your desired quotas (for whatever segments you’re seeking, such as newbie, experienced user, developer, CMS user, photoblogger, mobile user, etc. ). Then come the test sessions.
Depending on what is being tested, these last anywhere from half an hour to an hour and half apiece. Sessions are generally recorded using screen capture and web cam, with a video camera for backup. The moderator(s) generally take notes during sessions and/or (depending on what software is being used for the session capture) set markers in the video to indicate task completion, comments of interest, etc. In some cases, auxiliary test methods such as eye-tracking may be included. When the sessions are complete, the results are analyzed. All the notes and videos are reviewed, patterns are identified, and ultimately a report is written and the feedback informs the next round of revisions.
Some people think it shouldn’t take much time to do all this. I’ve lost count of the number of people who cite an old article by Jakob Nielsen that says you don’t need to test with more than 5 users because usability issues become clear right away. While I’ve found that to be generally true, when your user base is as diverse in experience level, usage, platform configuration, language (right to left languages have a pretty different experience) and demography as the WordPress community is, 5 users really isn’t enough to get a clear picture. We try to test with at least a dozen people each round, but then we are limited to a geographic region (test in NY, test in SF, or test wherever we can schedule enough people back to back to make it worthwhile), while WordPress users are located all over the globe.
To address this, we’re introducing a set of new contribution opportunities to expand our usability testing program. As with development and graphic design, we’re going to create an infrastructure to allow community participation in usability testing on a regular basis and in a much broader capacity than existed before, when it was limited to announcements that we needed participants in x city on y date. We’ll be looking for volunteer moderators as well as participants, hopefully from all over the world.
Moderators. Observational usability testing isn’t rocket science, but neither is it a simple task to reduce bias. Because of this, at first we’ll choose only moderators who have professional experience conducting usability tests. People who conduct testing for design agencies, software companies, usability consulting firms and the like will be our first round draft picks. In the future, when we have a group of regular volunteers and have ironed out any kinks in the process, we’ll ideally match up experienced testers with aspiring ones, using a mentorship model to increase the number of people who can contribute in this fashion.
Participants. If you use WordPress, chances are you could participant in a usability test at some point. In some cases we’re looking for particular behaviors (people who upload large video files, people who blog from their iPhone, people who manage more than 5 blogs, etc.), while other times the behaviors we’re looking for are much more common (do you have widgets in your sidebar, have you changed themes in the last 6 months, is there more than one author on your blog, etc.).
So how will these opportunities come into play, and how will it make WordPress better?
We’ll start with the moderators, and try to get volunteers with a decent geographic spread. Then, we’ll start signing up potential test participants in those areas (though we’ll also allow at-large registrations, since traveling testing will still be happening). We’re working on a registration process for potential participants. You’ll enter your basic info (location, contact info) and answer some questions about your WordPress usage to be entered in the database, and when there’s a testing opportunity coming up that’s appropriate for you, a local moderator will get in touch to see if you’re interested. Further down the road we may experiment with remote testing and other methods, but for now, this approach will broaden the geographic scope of our testing.
All moderators will follow the same test protocols and script, and their results/reports/video will be reviewed and collated, with a composite report (including the protocol/script that was used) published to the community. This will provide designers and developers with broader feedback during the dev cycle, and will allow the wider community to both understand and participate in the testing program.
If you’re interested in being a moderator during this initial phase (meaning you do it professionally), send me an email and introduce yourself. If you’re interested in signing up as a potential test participant, watch this space. We’ll post a link to the registration survey once it’s ready.
A question I hear pretty frequently is, “Why a redesign of the admin panel so soon after 2.5?” Those who have attended WordCamps in the past few months have already heard the answer, but for the people who haven’t had that opportunity, this post is for you.
When the community response to the 2.5 admin redesign was mixed, it seemed like a good idea to do usability testing to find out which issues were based on actual interface problems vs. which complaints were just a result of not liking change. To prevent bias, a third party was contracted to conduct usability testing, Ball State University’s Center for Media Design, Insight and Research division. Try saying that three times fast with a mouth full of peanut butter. Or fitting it on a business card. To save time, we’ll just call that third party CMD, since that’s what they call themselves.
The plan that was developed involved multiple rounds of testing, as well as the creation of two prototypes, hardcore! The first phase involved a usability review of 2.5 by CMD, the results of which were discussed with lead developers. A quick prototype was created that addressed some of the lightweight issues, so that the test participants could use both 2.5 on their own blogs and the prototype on a test site. Results would be analyzed and compared, leading to a second round of suggestions. A second prototype would be developed over a week or two, which would then be tested with the same participants, and a final report delivered. But you know what they say… the best laid plans of designers and developers often go awry.
After the first round of testing, it was clear that a prototype delivering the kind of fixes that could be coded in a week or two wouldn’t make much of a difference overall. We all decided a more ambitious prototype was in order, one that would experiment with a new approach to screen real estate and attempt to address as many of the issues from 2.5 as was possible with a few extra weeks of time. A rapid design process was followed by an even more rapid development cycle. The second prototype is what you know as Crazyhorse.
The second round of testing blew everyone away. The research team had never seen such consistent results. Tasks were completed faster, participant opinions rated it higher, understanding of how interface elements worked was greater, and it wasn’t even a fully functional application. Of the test participants, every single one said they would choose the prototype over their current administrative interface, and it wasn’t even pretty (those of you who remember the original Crazyhorse will vouch for this).
A presentation on the process from start to finish was part of the schedule at WordCamp 2008 in San Francisco, and the slides are available online, but as always the slides only tell you so much without the videos, live demo and verbal narration that went with it. (Use Google and you can see audience videos of the presentation.)
Here, then, is a PDF of reasonable size that you can download and peruse at your leisure that outlines the usability testing project in some detail. I wanted to include some eye tracking videos, but the file was so huge it would have been ridiculous for anyone to download it, so I stuck with eye tracking outputs called gaze trails to illustrate the findings. I also tried to pare down the text to the more salient points, since more than 50 hours of test video really does reveal an insane amount of data. I also cut out the section about designing Crazyhorse in the interest of staying under 25 pages. Hopefully you’ll think it’s a good balance. I’ll try to put together a separate document on the design process of 2.7 in a couple of weeks that will include the early Crazyhorse material.
So, if you want to know what we learned from the usability testing this summer that caused us to create what is now 2.7, go ahead and read the report.
WordPress 2.5/Crazyhorse Usability Testing Report (PDF)
Comments Off on Usability Testing Report: 2.5 and Crazyhorse
Comments Off on Usability Testing in New York
There are two contests going on in the WordPress community right now. If you’d like a chance to flex your WordPress skillz and perhaps win a prize and lifelong fame, you should consider dropping your code in the hat.
The first is the Sandbox Designs competition, which is like a theme competition, but working only with CSS and the highly semantic Sandbox theme. They already have a thousand dollars in prizes, so check it out.
Second our friends at Weblog Tools Collection are running a WordPress Plugin competition. They have a blog to track all the entries, but if you’re participating don’t forget to submit your code to the Plugin directory.
Both competitions require entries to be under the same GPL license that WordPress is, so regardless of who wins they’ll make the entire community much richer. (Remember, WordPress itself was written on the base of existing GPL code!)
Comments Off on Two Contests
Over at my site I’m running a small contest for people to create alternative color schemes for the administration interface. When it’s done I’ll pick a few winners and package everything up in a plugin you can use to spice up your administration a bit. If you have a few free minutes try your hand at it. There will be a few prizes for the winners, including a copy of Color Schemer Studio.
Comments Off on Color Contest
Just a summary of some of the things I’ve been working on today.
- The RSS 2.0 feed now supports multiple categories. Not sure how to do it for 1.0 yet, looking into it.
- The includes stuff now uses constants instead of variables. This means if you’re on the bleeding edge of the CVS you’ll have to update your wp-config.php again. Sorry! I anticipate this being the last change, ever.
- The renaming is just about complete, with outdated variable references being removed.
- Quicktags now enter whitespace with list markup to format it nicely.
- Error messages are a lot smarter, espescially when first starting out. It now tells you the problem exactly if there’s not a wp-config.php file, or if you have any problems with your database connection info. It offers a few common solutions and links to the support forums. It also doesn’t generate a thousand errors like it used to.
- The default index.php has been updated to clarify some concepts and have the current version number in the generator meta tag.
- readme.html has been updated to remove some wrong and old things. Needs a lot more work, but hopefully the wp-docs group will get this to be a really great document.
- The wp-links directory has been eliminated and the files in it have been moved to wp-images or wp-includes.
- Cleaned a final few include problems. The including system is so much cleaner now.
- The “Blog this” bookmarklet link has been moved to the posting screen and renamed “Press it”.
And I’m done for the night. More tomorrow…
Comments Off on Latest Updates
Before we do a release to the world we really need some testing on the .7 beta, which is now available as .zip and .tar.gz in the beta directory.
If you have any problems or comments you may contact me directly or post to the new Beta forum.
Thanks for your help.
Comments Off on Point Sever Beta Test
WordPress 4.1.2 is now available. This is a critical security release for all previous versions and we strongly encourage you to update your sites immediately.
WordPress versions 4.1.1 and earlier are affected by a critical cross-site scripting vulnerability, which could enable anonymous users to compromise a site. This was reported by Cedric Van Bockhaven and fixed by Gary Pendergast, Mike Adams, and Andrew Nacin of the WordPress security team.
We also fixed three other security issues:
- In WordPress 4.1 and higher, files with invalid or unsafe names could be uploaded. Discovered by Michael Kapfer and Sebastian Kraemer of HSASec.
- In WordPress 3.9 and higher, a very limited cross-site scripting vulnerability could be used as part of a social engineering attack. Discovered by Jakub Zoczek.
- Some plugins were vulnerable to an SQL injection vulnerability. Discovered by Ben Bidner of the WordPress security team.
We also made four hardening changes, discovered by J.D. Grimes, Divyesh Prajapati, Allan Collins, Marc-Alexandre Montpas and Jeff Bowen.
We appreciated the responsible disclosure of these issues directly to our security team. For more information, see the release notes or consult the list of changes.
Download WordPress 4.1.2 or venture over to Dashboard → Updates and simply click “Update Now.” Sites that support automatic background updates are already beginning to update to WordPress 4.1.2.
Thanks to everyone who contributed to 4.1.2: Allan Collins, Alex Concha, Andrew Nacin, Andrew Ozz, Ben Bidner, Boone Gorges, Dion Hulse, Dominik Schilling, Drew Jaynes, Gary Pendergast, Helen Hou-Sandí, John Blackbourn, and Mike Adams.
A number of plugins also released security fixes yesterday. Keep everything updated to stay secure. If you’re a plugin author, please read this post to confirm that your plugin is not affected by the same issue. Thank you to all of the plugin authors who worked closely with our security team to ensure a coordinated response.
Already testing WordPress 4.2? The third release candidate is now available (zip) and it contains these fixes. For more on 4.2, see the RC 1 announcement post.
Comments Off on WordPress 4.1.2 Security Release
The release candidate for WordPress 4.2 is now available.
We’ve made more than 140 changes since releasing Beta 4 a week and a half ago. RC means we think we’re done, but with millions of users and thousands of plugins and themes, it’s possible we’ve missed something. We hope to ship WordPress 4.2 on Wednesday, April 22, but we need your help to get there.
If you haven’t tested 4.2 yet, now is the time! (Please though, not on your live site unless you’re adventurous.)
Think you’ve found a bug? Please post to the Alpha/Beta support forum. If any known issues come up, you’ll be able to find them here.
To test WordPress 4.2 RC1, you can use the WordPress Beta Tester plugin or you can download the release candidate here (zip).
For more information about what’s new in version 4.2, check out the Beta 1, Beta 2, Beta 3, and Beta 4 blog posts.
Developers, please test your plugins and themes against WordPress 4.2 and update your plugin’s Tested up to version in the readme to 4.2 before next week. If you find compatibility problems, we never want to break things, so please be sure to post to the support forums so we can figure those out before the final release.
Be sure to follow along the core development blog, where we’ll continue to post notes for developers for 4.2.
Achievement unlocked: RC
Release here we come
Comments Off on WordPress 4.2 Release Candidate
Older Posts »