LibreOffice 5.2.3 as snap from Day One

Meist scheint manches auf den ersten Blick unmöglich.
Manches ist es auch, doch es wäre tödlich, das selbst zu glauben solange noch nichts feststeht
und die Party zu verlassen, bevor sie losgeht.

— Die Sterne, Stell die Verbindung her

Yesterday, two nice things happened: For one, LibreOffice 5.2.3 has been released and secondly Ubuntu Core 16 has been released. But beyond that, something in the middle between these two has happened: LibreOffice 5.2.3 has been released to the stable channel of the snap store at the same day. Now LibreOffice has been in the snap store for some time — and has also been on the stable channel since the Ubuntu 16.10 release. But this is the first time the LibreOffice snap is released in sync with The Document Foundation announcing the general availability of the final downloads. This was possible even though I was on vacation yesterday: LibreOffice snap packages are now being build on launchpad, which simplifies a lot, and launchpad can be asked to populate the edge channel of the store. This is making life very easy. Having smoketested the amd64 build from that channel before, to release LibreOffice 5.2.3 to the beta/candidate/stable channels too all I had to do was push three buttons on a web interface and it was available to all.

Building on launchpad, I also had the opportunity to create builds for armhf and i386 along with the usual amd64 builds with little extra effort. If you are adventurous you are encouraged to test these builds too: Be aware though that these so far aren’t even smoketested, I havent looked at them at all yet, so use them at your own risk.

All in all, this is great progress: LibreOffice 5.2.3 is available to users of Ubuntu 16.10 and Ubuntu 16.04 LTS as a snap on the day of the upstream release. And beyond that on all other distributions where snap is available — quite a few these days.

Update: ICYMI here is how to get the LibreOffice snap: http://www.libreoffice.org/download/snap/ — although strictly speaking you dont need the --channel=beta option anymore now. I will fix that soon.

Merging Communities

Come together, right now
— Beatles, Abbey Road, Come together

So September, 28th 2016 is the 6th birthday of LibreOffice and at the recent conference, we took a picture of those who were there on day zero:

brno32

As you might notice, I am not on that picture — on day zero I was working at Oracle, and were surprised by the news — like many others in this picture:

brno30

This is everyone at this years LibreOffice conference who used to work on the OpenOffice.org codebase at StarDivision, Sun or Oracle. A few people are in both pictures: Caolán McNamara and Thorsten Behrens were with LibreOffice from the start, but also worked at the OpenOffice.org team in Hamburg at some point in time. Of those working on OpenOffice.org still when LibreOffice started, I was the first to join LibreOffice — I had quit my job for that. It was an exciting time.

Looking back, both of these groups appear small — mostly because merging them, the LibreOffice community became so much more that the sum of its parts:

brno23

And of course, while a lot of people were at the conference, not everyone could
join, so there are more contributors from each of these groups than are in the
pictures. This years “state of the project” presentation showed again that the members of The Document Foundation are a truly worldwide community:

tdf-members

So, like the branches of the different descendants of OpenOffice.org, the
contributors and communities did come together in Brno to push
LibreOffice forward as one!

LibreOffice 5.2.0.2 available in the snap store

‘Cause the players gonna play, play, play, play, play
And the haters gonna hate, hate, hate, hate, hate
— Taylor Swift, 1989, Shake It Off

The latest release candidate of the upcoming LibreOffice 5.2.0 feature release is available for installation from the snap store. This makes it very easy to install this prerelease of LibreOffice for testing out new features (an incomplete glimpse on what to look forward for can be found on the LibreOffice 5.2 release notes page, which is still under construction, go on #libreoffice-qa if you want to help with testing).

To install this build of LibreOffice on any snap supported platform just open a terminal and run:

sudo snap install --channel=beta libreoffice

To start this version of LibreOffice, you run:

/snap/bin/libreoffice

The full path should only be needed, if you have another version of LibreOffice installed. If that is not the case a plain “libreoffice” should do.

Note that this version is still a prerelease and not for production use yet. That said, it is mostly a full-featured package including everything that would be packaged for end users of LibreOffice. While this package also includes a set of localizations to show that they work, their number has been restricted to English, French, German, Italian, Portuguese (Portugal/Brazil), Spanish for size considerations for now. This set is mostly the one Ubuntu provides on its installer images (removing those that might have issues as they need special fonts).

Another difference to prior downloads is that while LibreOffice still uses X11, now runs in confinement provided by snaps. Unlike previous releases on Ubuntu, this package defaults now to do so via the newer GTK3 backend: This has a lot of advantages, see details on Caolans Blog, but it is also a younger backend, that hasnt has that much time to be polished yet.

A third of a LibreOffice snap

Take your time, hurry up
The choice is yours, don’t be late
Take a rest as a friend
— Nirvana, Come As You Are

I have just updated the LibreOffice snap package. The size of the package available for download created some confusion. As LibreOffice 5.2 is still in beta, I built and packed it with full debug symbols to allow analysis of possible problems. Comparing this to the size of e.g. the default install from Ubuntu *.deb packages is misleading:

  • The Ubuntu default install misses LibreOffice Base and Java unless you explicitly install them
  • The Ubuntu default install misses debug symbols unless you install the package libreoffice-dbg too

As many people are just curious about running LibreOffice 5.2 without wanting to debug it right now, I replaced the snap package. The download and install instructions are still the same as noted here — but it is now 287MB instead of 1015MB (and it still contains Base, but no debug symbols).

The package file including full debug symbols — in case you are interested in that — has been renamed to libreoffice-debug.

(Note that if you downloaded the file while I moved files around, you might need to redo your download.)

LibreOffice 5.2.0 beta2 as a snap package

What’s been happening in your world?
What have you been up to?
— Arctic Monkeys, Snap out of it

So — here is what I have been up to:

LibreOffice 5.2.0 beta2 installed as a snap on Ubuntu 16.04
LibreOffice 5.2.0 beta2 installed as a snap on Ubuntu 16.04

The upcoming LibreOffice 5.2 packaged as a nice new snap package. This:

  • is pretty much a vanilla build of LibreOffice 5.2 beta2, using snapcraft, which is making packaging quite easy
  • contains all the applications: Writer, Calc, Impress, Draw, Math, Base
  • installs easily on the released current LTS version of Ubuntu: 16.04
  • allows you to test and play with the upcoming LibreOffice version to your hearts delight without having to switch to a development version of Ubuntu

So — how can you “test and play with the upcoming LibreOffice version to your hearts delight” with this on Ubuntu 16.04? Like this:

wget http://people.canonical.com/~bjoern/snappy/libreoffice_5.2.0.0.beta2_amd64.snap{,.sha512sum}
sha512sum -c libreoffice_5.2.0.0.beta2_amd64.snap.sha512sum && sudo snap install --devmode libreoffice_5.2.0.0.beta2_amd64.snap
/snap/bin/libreoffice

and there you have a version of LibreOffice 5.2 running — for example, you can prepare yourself for the upcoming LibreOffice Bug Hunting Session. And its even quite easy to remove again:

sudo snap remove libreoffice

This is one of the things that snap packages will make a lot easier: upgrading or  downgrading versions of an application, having multiple installed in parallel and much more. Watch out as there are more exciting news about this coming up!

Update: As this has been asked a few times: Yes, snap packages are available on Ubuntu. No, snap packages are not only available on Ubuntu. This text has more details.

Update 2: The original download included debug symbols and thus was quite big. The download now has 287MB. This post has all the details.

Zu Spät: Hackfest Hamburg

Warum hast Du mir das angetan?
Ich hab’s von einem Bekannten erfahren.

— Die Ärtze, Debil, Zu Spät

Its been more than two years since the last Hackfest in Hamburg! So we are indeed much too late (german: Zu Spät) with repeating this wonderful Event. Right a day after everyone updated his or her Desktop to Wily Werewolf we will meet for a weekend of happy hacking again in Hamburg!

Hamburg Hackfest 2013 - carelessly stolen from Eikes Retrospective
Hamburg Hackfest 2013 – carelessly stolen from Eikes Retrospective

So now, we will meet again. You are invited to drop by this weekend, we will celebrate a bit on Friday evening (ignoring the german culinary advise in the song linked above about “Currywurst and Pommes Fritz” — I imagine we prefer Club Mate and Pizza) and hack on LibreOffice on Saturday and Sunday. Curious new faces are more then welcome!

Blast from the Past: the Pimpl?

They sentenced me to twenty years of boredom
For trying to change the system from within
— Leonard Cohen, I’m your man, First we take Manhattan

Advance warning: This blog post talks about C++ coding style, and given the “expressiveness” (aka a severe infection with TimTowTdi) this is bound to contain significant amounts of bikeshedding, personal opinion/preference. As such, be invited to ignore all this as the ramblings of a raging lunatic.

Anyone who observed me spotting a Pimpl in code will know that I am not a fan of this idom. Its intend is to reduce build times by using a design pattern to move implementation details out of headers — a workaround for C++s misfeature of by default needing a recompile even for changing implementation details only without changing the public interface. Now I personally always thought a pure abstract base class to be a more “native” and less ugly way to tell this to the compiler. However, without real testing, such gut feelings are rarely good advisors in a complex language like C++.

So I did some testing on the real life performance of a pure abstract base class vs. a pimpl (each of course in a different compilation unit to prevent the compiler to optimize away what we want to measure) — and for reference, a class with functions that can be completely inlined. These are the three test implementations, inline:

-- header (hxx) --
class InlineClass final
{
	public:
		InlineClass(int nFirst, int nSecond)
			: m_nFirst(nFirst), m_nSecond(nSecond), m_nResult(0)
		{};
		void Add()
			{ m_nResult = m_nFirst + m_nSecond; };
		int GetResult() const
			{ return m_nResult; };
	private:
		const int m_nFirst;
		const int m_nSecond;
		int m_nResult;
};

Pimpl, as suggested by Effective Modern C++ when using C++11, but not C++14:

-- header (hxx) --
#include <memory>
class PimplClass final
{
	public:
		PimplClass(int nFirst, int nSecond);
		~PimplClass();
		void Add();
		int GetResult() const;
	private:
		struct Impl;
		std::unique_ptr<Impl> m_pImpl;
};
-- implementation (cxx) --
#include "pimpl.hxx"
struct PimplClass::Impl
{
	Impl(int nFirst, int nSecond)
		: m_nFirst(nFirst), m_nSecond(nSecond), m_nResult(0)
	{};
	const int m_nFirst;
	const int m_nSecond;
	int m_nResult;
};
PimplClass::PimplClass(int nFirst, int nSecond)
	: m_pImpl(std::unique_ptr<Impl>(new Impl(nFirst, nSecond)))
{}
PimplClass::~PimplClass()
	{}
void PimplClass::Add()
	{ m_pImpl->m_nResult = m_pImpl->m_nFirst + m_pImpl->m_nSecond; }
int PimplClass::GetResult() const
	{ return m_pImpl->m_nResult; }

Pure abstract base class:

-- header (hxx) --
#include <memory>
struct AbcClass
{
	static std::shared_ptr<AbcClass> Create(int nFirst, int nSecond);
	virtual ~AbcClass() {};
	virtual void Add() =0;
	virtual int GetResult() const =0;
};
-- implementation (cxx) --
#include "abc.hxx"
#include <memory>
struct AbcClassImpl final : public AbcClass
{
	AbcClassImpl(int nFirst, int nSecond)
		: m_nFirst(nFirst), m_nSecond(nSecond)
	{};
	virtual void Add() override
		{ m_nResult = m_nFirst + m_nSecond; };
	virtual int GetResult() const override
		{ return m_nResult; };
	const int m_nFirst;
	const int m_nSecond;
	int m_nResult;
};
std::shared_ptr<AbcClass> AbcClass::Create(int nFirst, int nSecond)
	{ return std::shared_ptr<AbcClass>(new AbcClassImpl(nFirst, nSecond)); }

Comparing these we find:

implementation lines added for GetResult() source entropy added source entropy for GetResult() runtime
inline 2 187 17 100%
Pimpl 3 316 26 168% (174%)
pure ABC 3 295 (273) 19 (16) 158%

So the abstract base class has less complex source code (entropy)1, needs less additional entropy to expand and is still faster in the end on common hardware (Intel i5-4200U) with common compiler optimization switches (-O2)2.

Additionally, in a non-trivial code base you might actually need to use virtual functions for your implementation anyway as you are deriving from or implementing an existing interface. In the Pimpl case, this means using two indirections (resolving the virtual function and then resolving the m_pImpl pointer in that function on top of that). In the abstract base class case thats not happening and in addition, it means that you can spare yourself the pure virtual declarations in the *.hxx (the virtual ... =0 ones), as those are already declared in the class derived from. In LibreOffice, this is true for any class implementing UNO interfaces. So the first numbers are actually biased against an abstract base class for real world code bases — the numbers in parathesis show the results when an interface is already defined elsewhere.

So unless the synthetic example used here is some kind of weird cornercase, this suggests abstract base classes being the better alternative over a Pimpl once the class goes beyond being a plain value type with completely inlineable accessor member functions.

Thanks for bearing with me on this rant about one of my personal pet peeves here!

1 entropy is measured as cat abc.[hc]xx|gzip|wc -c or cat pimpl.[hc]xx|sed -e 's/Pimpl/Abc/g'|gzip|wc -c.
2 Here is the code run for that comparision:

constexpr int repeats = 100000;

int pimplrun(long count)
//int abcrun(long count)
{
        std::vector< std::shared_ptr<PimplClass /* AbcClass */ > > vInstances;
        vInstances.reserve(count);
        while(--count)
                vInstances.emplace_back(std::make_shared<PimplClass>(4711, 4711));
                //vInstances.emplace_back(AbcClass::Create(4711, 4711));
        int result(0);
        count = vInstances.size();
        while(--count)
                for(auto pInstance : vInstances)
                {
                        pInstance->Add();
                        result += pInstance->GetResult();
                }
        return result;
}

Instances are stored in shared pointers as anything that a Pimpl is considered for would be “heavy” enough to be handled by reference instead of by value.

Update 1: Out of curiosity, I looked a bit deeper at this with callgrind. This is what I found for running the above (with 1000 repeats) and --cache-sim=yes:

I1 cache: 32768 B, 64 B, 8-way
D1 cache: 32768 B, 64 B, 8-way
LL cache: 3145728 B, 64 B, 12-way

event inline ABC Pimpl
Ir 23,356,163 38,652,092 38,620,878
Dr 5,066,041 14,109,098 12,107,992
Dw 3,060,033 5,094,790 5,099,991
I1ir 34 127 29
D1mr 499,952 253,006 999,013
D1mw 501,636 998,312 500,097
ILmr 28 126 24
DLmr 2 845 0
DLmw 0 1,285 250

I dont know exactly what to derive from that, but what is clear is that purely by instruction counts Ir this can not be explained. So you need --cache-sim=yes which gives the additional event counts. Actually Pimpl looks slightly better on most stats, so as it is slower in real life, the cache misses on the first level data cache D1mr might have quite an impact?

Update 2: This post made it to reddit, so I looked into some of the feedback from there. A common suggestion was to use for(auto& pInstance : vInstances) instead of for(auto pInstance : vInstances) in the benchmarking function. This had no significant impact on walltime measurements nor made it callgrind event counts show some clearer picture. I also played around with the order of linked objects to see if it has any impact (via cache locality etc.). While runtime measurements fluctuated quite a bit (even when using the same binary), the order was always the same: inlining quickest, then abstract base class and pimpl slowest.