bugs and fixes in LibreOffices issue tracker — beyond the 3.6 series

Hey ho, let’s go
Hey ho, let’s go
They’re forming in a straight line
They’re going through a tight wind

— Blitzkrieg Bop, Ramones

Italo wrote this nice overview on the history of LibreOffice giving a somewhat nostalgic view back on the early days and some good statistics on what we achieved since then, so when I posted the postmortem on LibreOffice 3.6 yesterday there were some questions on the numbers beyond the LibreOffice 3.6 series. Well, without further ado here they are:^(*)

There is some healthy growth in fixes going in each release although it somewhat slowed down^(**) around the 3.6 series as the amount of bugfixes grew in a way that made it quite some extra work to keep up with their administration purely on a mailing list. Luckily, this is were gerrit came into play: In the 4.0 series most commits (77%) are reviewed on gerrit, which steamlined the work in a way that made the rate of fixes climbing again^(***), so that the current LibreOffice 4.0.4 has more bugfixes in a minor release than any previous version that early in the cycle.

Note though that these bug fix counts can not be simply added for a multitude of reasons:

• some bugfixes go into multiple releases (because there is more than one active branch at any given time)
• some bugfixes do not get accounted for in the bug tracker
• many (in fact most of the exciting and interesting ones) go into the major series updates and not into a minor bugfix update

So how many bugs did LibreOffice resolve since it started? Its hard to tell, because these issues are not always tracked in one issue tracker^(****). However, this table from bugzilla gives a lower bound. As of 2013-07-23, the LibreOffice project resolved 12.596 bug reports on its own issue tracker, half of those fixed by developers, half of those hunted down and triaged by the QA team:

• 4.389 bug reports were intentionally fixed by developers (resolution: FIXED)
• 2.123 bug reports were unintentionally fixed by developers and then found fixed by the QA team (resolution: WORKSFORME — sometimes one bug causes multiple symptoms, so a fix for one bug report might also solve another that the developer is not aware of)
• 3.003 bug reports were identified as a duplicate of an existing report by the QA team (resolution: DUPLICATE)
• 3.081 bug reports were found to be invalid of some kind (resolution: INVALID, NOTABUG, NOTOURBUG, …)

So in summary: Since it started and as of 2013-07-23, the LibreOffice project in total at least fixed 6.512 issues and resolved 12.596 bug reports from its own issue tracker.

^(*) A note on the minor release bug fix counts: They are just scraped off from the ChangeLog pages like https://wiki.documentfoundation.org/Releases/4.0.4/RC1 — esp. for older releases these might still be a bit off.
^(**) note that the 3.6 series was alive 3 month longer than 3.5 (~35% more time), without receiving the same amount of additional fixes.
^(***) To the tune of the Ramones quoted above.
^(****) For example, at the time of writing, there are 256 resolved issues in launchpad tracking bugs at LibreOffice of 1969 resolved issues filed against LibreOffice on Ubuntu in total as only a subset of well-triaged and hard to fix issues is upstreamed. So the numbers given above are not conflicting at all with e.g. the estimate of 3.000 bug fixes in LibreOffice 4.1 alone. See the development FAQ for an overview of common issue trackers referenced in commit messages.

LibreOffice 3.6.7 on Ubuntu: 547 bug fixes and zero known, well-triaged regressions against version 3.6.0 on release

Dear Prudence, won’t you come out to play
Dear Prudence, greet the brand new day

–– Dear Prudence, White Album, Beatles

Yesterday, I put a build of LibreOffice 3.6.7 in the according Ubuntu PPA. As LibreOffice 3.6.7 is the last minor release update for the 3.6 series this is a good time to have a look back and see how our choice of a train release model pays off. For this I had a look at the number of bugfixes and the regressions over the series. First lets look if there were any regressions in the minor series: that is, if there is something which worked in 3.6.0, but which stopped working in 3.6.7. For LibreOffice 3.6.7 we look at:

• known issues (that is: they have to be reported e.g. at our Bug Submission Agent)
• they have to be well-triaged to be a real problem (that is: They cant be in bug status NEEDINFO or UNCONFIRMED — which would mean that they are incomplete)
• they have to be marked as a regression
• they have not been there in 3.6.0, otherwise they would not be a minor release regression
• they are not fixed in one of the 3.6.1 — 3.6.7 minor release updates.
• they have to affect Linux

At the time of publishing 3.6.7 this query results in bugzilla replying with its infamous  “Zarro Boogs found” — so there are no known, well-triaged regressions in LibreOffice 3.6.7 vs. LibreOffice 3.6.0 at the time of release on Ubuntu.

Does this mean that LibreOffice will give you a hard guarantee on this? No, the release train model of LibreOffice is more important: It keeps everyone accountable and gives users a reliable date and time for a new release to test and use. So LibreOffice will not “stop the train”, if such a bug candidate is found. If you for example extend the above query to all platforms you will see it return 5 bugs from other platforms. It is unlikely that there are “real” minor release regressions though: those issues are just not confirmed to have been there at 3.6.0 already.

Enough talk about regressions in the 3.6 series — lets have a look at the positive things in minor releases: bug fixes. Despite LibreOffice 3.6.7 on Linux having no known, well-triaged release regressions, it has 547 bugs fixed against 3.6.0. So for Ubuntu users installing 3.6.7 this is the picture:

 

So much for the quality of minor releases, but what about major releases? Well, for major releases there is one additional important factor: timely reporting. So lets have a look at LibreOffice 4 regressions that have been filed in a timely manner: That is, between the tagging of the first alpha on November 20, 2012 and the release of LibreOffice 4.0.0 on February 7, 2013. Bugzilla shows that currently more than 95% of these are already resolved, even though LibreOffice 4 will still see a few more minor updates.

For regressions reported in the alpha and beta phase before the tagging of the first release candidate on January 9, 2013 its even more impressive: All of those regressions are resolved by now.

I assume the statistics for LibreOffice 4.1 to become even more impressive as the LibreOffice bug triage contest was hugely successful and sure will help fast tracking the triage of bugs, getting them ready to be fixed by a developer quickly. Thanks to Joel Madero, Joren De Cuyper and Robinson Tyrone for organizing this awesome event and big “Thank you!” too to all the volunteers taking part in this.

Which finally brings us to the little tune quoted at the top-right of the post: The earlier you test LibreOffice and report an issue, the better for you and for the product. With LibreOffice 4.1 approching fast, I invite you to test what will become LibreOffice 4.2 by downloading a daily build, playing with its exciting new features and get involved on #libreoffice-qa while listening to Sir Paul doing his magic on the bass over … Sir Paul doing his magic on the drums.