Moved my blog to github.io

I replicated the content of my blog to https://bjoernmichaelsen.github.io/blog/ where it can be accessed ad-free even without ad blockers. Also the old content from livejournal has been replicated there.

New content will be added only there from now on.

For Planet Document Foundation, the feed should be updated to the feed for LibreOffice relevant topics.

The great filter of open source projects

This is how you make yourself vanish into nothing
— 24 frames, Something More Than Free, Jason Isbell and the 400 unit

So, with the recent layoffs at Mozilla — among other things — a bit of discussion on the sustainability of open source projects has been reignited. There was a wide range of takes: from “FOSS is dead” (no) to “we need to re-decentralize the internet” (yes). I could not quite help putting forth opinions on the matter myself and did so on a short twitter thread. Fundamentally though, the opinions expressed on this matter seem to almost talk past each other — and I think the reasons for this might be found in history of open source(1).

users are contributors

In the beginning, pretty much all open source projects had an almost complete overlap of users and contributors. Or at least potential contributors. E.g. the original GNU projects pretty much solved the problems of their own contributors and even most of those that where not contributors (yet) where non the less in the majority able to contribute to all parts of them. You can also see it in the original announcement of Linux as

“just a hobby, won’t be big and professional like gnu”

and while Linux grew way beyond that, there are still indications of this being true today when the main (and only) collaboration tool of the project, the LKML, was down for an extended time in 2018, because a mail server in a cabinet at someones home was not booting through after an power outage and the person to kick it was on vacation. The Linux kernel project was still self-hosting its infrastructure in 2018.

Another — later — project, that I am assuming to have been quite resilient and which I am assuming will continue to be quite resilient is gentoo linux: By requiring users to compile all software themselves, this distribution makes their users either give up on their installs or gets them at least halfway to be packagers (and for a distribution, packagers are contributors) themselves. Also, by not having to deal with binaries, gentoo reduces its infrastructure needs to a minimum. And even while there are some signs of downsizing at gentoo, I am hopeful that the flexibility mentioned above makes gentoo more sustainable and self-reliant than others for quite some time to come.

users are not all contributors anymore

In the 2000-2010 decade, especially in the second half, a lot of open source projects joined the choir were only a minority of users were contributors too. For many, the majority of users were not even possible contributors across the project. For a few, contribution was even implicitly limited to a closed circle. This gave rise to the perception that something like an “open source product” exists — especially by users. Here are some examples:

projectcomplementcontributors goal
Mozillawebprevent Microsoft from monopolizing the web, later generalized
OpenOffice.orgenterprise productivity
Androidweb/appsprevent Apple from monopolizing the web, app and smartphone market
Chromewebprotect a cloud, an advertising business and a search engine
Ubuntu desktopmanychanged over time, my guesses: initially, make launchpad what github is today, then complement the Ubuntu phone, then cloud offerings, but now something new maybe up

Of course, no such thing as an “open source product” exists and all of the above are variations of Strategy Letter V, which — being from 2002 — was already old by the time most of those were started:

Smart companies try to commoditize their products’ complements.

All of the above projects, commoditized their complements and this allowed users, who were not contributors to still benefit from the work of those who were as these contributors were interested in protecting the complement. With technology moving to the web and the cloud you can see this pattern repeating there in a few examples:

projectcomplementcontributors goal
reactfacebookensure all browser stay compatible with own use by making the framework widely used
istiocloud hostingraising the barrier to entry for microservice hosting (embrace and extend style)

the great filter of open source

So, what does this have to do with the layoffs at Mozilla and the current struggles at other open source projects? Well, the complements from strategy letter V that might motivate contributors to work on projects beyond their own needs exist at a given point in time, but … panta rhei, and this gives different outlooks for projects depending on how broad the complements are that it is serving:

Ultimately, open source projects provide a commodity. If their infrastructure needs are limited and their users also contributing they should be quite resilient (see e.g. gentoo). If they have many non-contributing users but have multiple complementing products, they will likely do well too.

However, having non-contributing users and only one complement might just be the Great Filter for open source projects: Once this complement is vanishing, so will the project — at least for non-contributing users. The remaining contributors — those that work without the need for a product, a complement, a business or users — will not feel that too much. But the non-contributing users will, as they wont be relevant at all anymore when the project downsizes itself to serving its contributing hobbyists alone.

conclusions and unrequested advice

So, at least for Mozilla and LibreOffice, I have some opinions and unrequested advice. In general, I see an urgent need for open source projects to establish a shared understanding among contributors what currently is a commodity — and thus is governed by the project and its institutions — and what are the complements of the commodity. For Mozilla and LibreOffice this might mean:

  • Felix von Leitner recently suggested at heise.de that the German government could make earmarked donations to Mozilla so protect digital sovereignty: “Die Bundesregierung könnte der Mozilla-Stiftung zweckgebundene Spenden zukommen lassen.”
    While I agree with both the goal and concerns over spending the money on a “cyber agency” might dilute results, earmarked donations are a also a very painful device to use: The administrative friction between a donor and the foundation will imply a huge overhead.
    Mozillas projects might be better off opening themselves to outside contributions and diversifying its contributions(2). And if there is indeed money in the German or European government to spend on strategic goals like digital sovereignty, tendering those to capable local providers with a crystal clear purpose will have much better outcomes. Both FSFE and OSBA might contribute starting points and help find suitable partners.
    All of this might seem very local, but likely it is not: Mozilla should look globally into diversifying the contributions to their projects, even if it might reduce their control as sole maintainer of them.
  • LibreOffice needs to decide where it really wants to provide a commodity and what its complements are: Both in the dimension of online collaboration vs. desktop and enterprise users vs. home users. The space claimed for commodity should be broad enough to motivate non-commercial contributors and to allow to grow into new complements when they appear, but overextending will leave too little room for complements, products and ultimately users. When the space allocated to be a commodity is overextended, the project will downsize to be a project solely serving its most active contributors(3) leaving aside non-contributing users.
    I see this responsibility at the board of the Document Foundation and there alone. It is elected for this as representatives of the community of contributors for exactly decisions like this.

Maybe not all of the dream is lost.

(1) Note that I am no expert on the history of open source nor do I know the internal workings and politics of all the open source projects, so there is unfortunately a lot of conjecture over the limited body of my own experience.

(2) This is where I might disagree with Michael Meeks’ take on that.

(3) The Document Foundation receives significant donations from individuals compared to other open source projects. Michael Meeks has some numbers on the value of developer contributions in kind to compare them to and put them in perspective.

Yak Shaving Progress Report

Take out the papers and the trash
Or you don’t get no spendin’ cash

— Yakety Yak, The Coasters

 

At last years LibreOffice conference in Tirana I gave a talk on how SwClient is considered harmful. At this years LibreOffice conference in Almeria, I presented a lightning talk, giving some updates on the progress.

Additionally, with some recent idle changes not only the unocore directory in Writer is free of the error prone old SwClient/SwModify combo, but also the directories:

  • sw/source/core/view
  • sw/source/filter/html
  • sw/source/filter/basflt
  • sw/source/filter/ww8
  • sw/source/filter/xml
  • sw/source/ui
  • sw/source/uibase

So far, it seems the hope I expressed at the conference that getting rid of SwClient and SwModify (leaving aside the core layout for now) seems to be quite doable and well worth it for the errors and fragility this will prevent in those areas of code.

The saga begins …

Oh, the Council was impressed, of course
Could he bring balance to the Force?

— The Saga Begins, Weird Al Yankovic

As you might have seen, we have now run four C++ sessions to get started with C++ and LibreOffice development. The origin of this actually happened already at the last LibreOffice Hackfest in Munich where Izabela, Mike, Anxhelo and me conspired on the idea. We also started to recruit LibreOffice developers as mentors right there and Xisco joined us soon.

As the lectures discuss the basics of data structures and C++ I started to create some patches against LibreOffice to show how to start with simple things in the LibreOffice build based on the examples from the lecture, but in the environment of LibreOffice and with some of its framework and conventions:

Some background and discussion about these can be found in the videos linked in the commit messages of these and on the C++ lectures wiki page. I hope these small examples are helpful to someone starting with C++ and LibreOffice even if you might have missed the first C++ lecture sesssions.

Why am I posting this? Now? Well, tomorrow the LibreOffice FOSDEM 2019 Hackfest starts in Bruessls. As you can see from the above, awesome things often take off there! You dont have to take my word for it: Izabela and Anxhelo wrote about their experiences too! If reading their report (or the story about how the lectures started) make you curious, please feel invited to join us in Bruessels at FOSDEM — or if you cant make that, the next Hackfest!

trusty old horses

Will you still need me, will you still feed me
When I’m sixty-four?
— When I’m sixty-four,  Sgt. Pepper’s Lonely Hearts Club Band, The Beatles

So almost six years ago I started using my LibreOffice development rig “Big Bertha”. Back then I thought: “This likely will be my last big machine, the next time I wont buy anything for under my desk, I will just build in the cloud.”. Six years on, that is partially true: I did not buy a new “desktop” machine. On the other hand, I am not building on some cloud machine either: I still use good old Bertha. So, why that?

For one, the speed of processors stopped improving at the insane rate it had before, so indeed machines that one buys today are not that much faster than those six years ago. However, cloud computing prices also stopped dropping like they used to. Which leaves me with not much reasons to buy new hardware — but also with little reason to consider to accept the additional inconveniences that come with building on remote hardware.

In 2012, the fastest build of a then-master checkout of LibreOffice from scratch and without caches I got out of Big Bertha was in ~18 minutes. I have not tried it again with present day LibreOffice — I assume it to be quite a bit slower, if only because e.g. we added tests left and right. But still: the old machine under the desk is still competing. A look at the numbers: A c4.8xlarge “compute-optizimed” instance on EC2 promises 36 vCPUs and 60GiB of memory. Now those 36 cores might be faster than those on the old Opteron 6272s I am running on. But I assume not much: CPUs really did not get much faster in the last six years — especially for workloads like compiling LibreOffice. For the most part they got better in other ways though: they use less power and push out less heat. So in the end the two Opteron 6272 with 32GB are likely still competing quite well with the stuff available from cloud providers.

So, why am I posting this? Buying a modern Server CPU still costs a fortune: a 16-core AMD Epyc 7281 costs 672 EUR at the time of posting. A new full machine with two of those comparable to Big Bertha will cost 2000-3000 EUR. Everybody loves AMDs Epycs apparently. But this challenge is also an opportunity: If one does not buy new hardware. Looking for “Opteron 6272” and “Opteron 6276” I found full systems available for 410 USD or even a 64-core, 256MB RAM system for 999 USD. These systems might be making too much heat and eating too much electricity: When I measured Bertha back in the day, compiling used ~400 Watt. At local electricity prices of 0.30 EUR/KWh that is 0.12 EUR per hour. Even if the c4.8xlarge on EC2 might be a bit faster, it still costs 10x as much. One should not make decisions without measuring, but its unlikely to be that much faster.

So I guess Im saying: If your LibreOffice build is too slow, have a look at the results on ebay for “opteron 6272” or “opteron 6276”. Those beasts might have served their time in the datacenter, but they may still be a steal for LibreOffice development. Or at least they should be worth a consideration …

Getting Started With LibreOffice Development: Object-oriented Programming and C++ Introductions

Turned around and found the right line

— No Leaf Clover, Metallica

So, for getting started with LibreOffice development e.g. with an EasyHack four things are needed:

  • Understanding of Object Oriented Programming
  • C++ Language Fundamentals
  • Completing a Build from Scratch of LibreOffice master
  • Understanding git and gerrit to submit your changes

We traditionally have covered the latter two quite well in our Wiki:

However, we did not have hints on where to find good documentation on object oriented programming in general and modern C++ programming. At most universities these days programming is taught with Java, JavaScript and Python, which leaves a missing piece on getting started with C++ programming. So Eike and me started looking for good resources on these topics too. Here is what we added to the General Programming Guidelines:

The OOP books are using Java as reference language, but they do not get lost in language details and intentionally allow using the concepts on other languages like C++. The university lecture starts off expecting basic knowledge of programming (e.g. in Java/JavaScript) so both together should yield a reasonable coverage of what is needed for your first EasyHacks.

Finding a good and modern starting point for C++ development was by far the hardest topic to cover of those named above. However, looking for them prompted Eike to dig out his personal “developer bookmarks” — a treasure trove that I will keep exploring further for other good content (this is where the “C++ Annotations” link came from).

 

P.S. As a sidenote and additional motivation: All those books — but especially “Head First: Design Patterns” should provide the context to understand many of the in-jokes found on the oldest wiki ever, the C2 WikiWikiWeb which can be an quite entertaining read once in a while.

LibreOffice Hamburg Hackfest 2018 After Action Report and: Where’s next?

Freedom’s just another word for nothin’ left to lose
Nothin’, it ain’t nothin’ honey, if it ain’t free

— Janis Joplin singing “Me and Bobby McGee” by Kris Kristofferson

Years ago, I opened one of the first LibreOffice Hackfests in Hamburg with the motto inscribed in the Hamburg city hall: “Libertatem⁠ quam peperere maiores digne studeat servare posteritas”. This year, the LibreOffice Hackfest came back to Hamburg: to the office of freiheit.com.

freiheit
Carolin, Jessica and Lena from freiheit.com helping me opening the Hackfest
(c) 2018 Thorsten Behrens. All rights resevered.

If your German is as good as your Latin, you will find some common ground between those facts about the 2013 and 2018 Hackfests: freedom.

On the other hand, a lot has changed: For one, the 2018 one more explicitly invited the local LibreOffice community too to support a better exchange of ideas between non-code and code contributors. This was natural in the earlier days, when the project was smaller, but now is something we need to have actively focus on.

Even more importantly, this was the first “Hackfest: the next Generation” and we had two mentors sponsored by the Document Foundation to help contributors to improve LibreOffice in ways they deem important: Armin Le Grand from CIB and Miklos Vajna from Collabora. Armin worked with Regina on layers in Draw, a topic Regina requested as a topic for the Hackfest. Miklos helped a lot of contributors with a wide set of issues, his own report has the details.

DSC02463.JPG
Armin and Regina churning away on layers in LibreOffice Draw.
(c) 2018 Bjoern Michaelsen. All rights resevered.

IMG_2234
Miklos on one of his many mentoring missions in Hamburg.
(c) 2018 Bjoern Michaelsen. All rights resevered.

Of course, there was a lot more going on, and Mike has written an excellent overview of all the good things that happened.

Personally, I was extremely pleased with these results and think the “Hackfest: the next generation” format should be repeated. The question is: Where? The key to answer that question is finding a location that will be attended by people who profit from mentoring the way those that worked with Miklos and Armin in Hamburg did.

So if you think: “A Hackfest is an awesome thing and I want one in my local community”, collect ideas that people — existing and future contributors — who would attend are interested in working on. Reginas reply (linked above) is an excellent example of how to do that.

If there are topics to work on, a local Hackfest should be easy to support. I will likely call for such proposals again soon, but should you already have concrete topics and locations you want to share, dont hesitate to send them to Board Discuss on your own initiative. Rest assured we will welcome them!

DSC02479.JPG
Hackfest or Barcamp? Your Call!
(c) 2018 Bjoern Michaelsen. All rights resevered.

Finally: Thanks again to freiheit.com for hosting us!

Invitation to the LibreOffice Hackfest Hamburg 2018 on April 7th and 8th

… for a holiday of working class
She’s a runaway of the establishment incorporated
She won’t cooperate
— Last of the American Girls, 21 Guns, Green Day

Its once again time to meet again in Hamburg for a LibreOffice Hackfest! We will meet with contributors to LibreOffice – old and new – and also welcome those who have not yet contributed to the project, but are interested in doing so. This will also be the first Hackfest TNG.

We will have EasyHacks prepared for anyone wanting to do their first code contribution to the project – and we will have mentors around who may provide guidance to every interested newcomer. An Hackfest is the easiest way to get involved for a curious newcomers to the project: we certainly can offer an interesting C++ codebase!

Not interested in coding? As the local German LibreOffice community joins this Hackfest, non-coding topics like documentation, QA and marketing especially in the German context will be discussed too. This is an opportunity to meet those active in the German community and plan and coordinate how to push the project, the product and the brand of LibreOffice in the German context. Mixing and mingling of the German community with the international developer community is a bonus for both: otherwise this only happens at the frantic and busy meetings at FOSDEM and the LibreOffice conference, where usually time is a very restricted resource for everyone.

Note that as usual support for travel and accommodation for the Hackfest is available from The Document Foundation as per its travel policy, so please feel invited to make use of that offer, especially if you have always been curious about the LibreOffice project and you want to get involved, but are not quite sure how: a Hackfest is a great way to learn that.

So – where will we meet? Here:

6fwd

This is 6Fwd – a beautiful meeting room at freiheit.com technologies, who are generously hosting us for this event. I am very excited about us being guests at this location and would like to thank freiheit.com for their sponsorship!

Details about the LibreOffice Hackfest Hamburg 2018 are kept on the TDF wiki page and will be continuously updated. If you are interested in joining the event please start adding your name there (especially if you are interested in TDF sponsoring travel and accomodation for you).

(If you are still unsure about joining the event, you can have a look at the wiki pages of previous events to get an idea. But really: the best way to learn about the project is going to a Hackfest, so dont miss this opportunity!)

Hackfests: The Next Generation

Reports of my assimilation are greatly exaggerated.

— Jean Luc Picard, Star Trek: First Contact

(This is a repost from the discussion on the board-discuss@documentfoundation.org discussion — including all typos and misspellings — for more visibility.)

Hi all,

I recently had a look at a variety of challenges the LibreOffice community is facing wrt Hackfests and especially also tenders:

Status Quo
==========

– due to a set of reasons based in the size of the project, the scope and selection of tender topics, the rules of properly running an NGO and the distibution of skilsets and available time in the BoD it is a lot harder to oversee the tenders for TDF than it would be for a for-profit organization.

– also, for historic reasons mostly, these tasks have been limited to the BoD mostly, while as an open disttributed and tranmsparent community we should not needlessly concentrate this work: rather the challenges and solutions should be shared as widely as possible in the community (and beyond).

– the waterfall modelled tenders have no iterative approach. Because of this they also tend to be mostly quite small, leading to significant overhead at both TDF and for the business implementing the tender.

– contracting out tender in bulk in a blackbox fashion naturally limits the ressources spend on documentation of discovered challenges. It thereby also needlessly limits the educational output to the community. There is such output, but it clearly could be better.

– beyond that we reduced doing Hackfests ~2 years ago, limiting the exchange of knowledge they provide.

– Unlike in the good old days, we dont have urgent infrastructual problems and the like to solve that would rally developers around topics. So motivation for certified devs to attend has shrunk. And those that do attend usually use the facetime for syncing on various issues, while the actual work on the code is somewhat limited. Even more when there is an emergency at the (professional) developers employer (Unfortunately, there often is.)

Suggested new format
====================

So everyone hip in the last decade would see the words “waterfall”, “non-iterative”, “controlling overhead” would scream “get agile” from the top of their lungs. In general that might solve the practical problems, but as agile is essentially a way to move the customer in close enough to create the trust and bond allowing the overhead to go away, that is exactly what we need to watch out for and avoid: the foundation should not bind itself too close to any single commercial provider of services in its core operations.

But maybe revitalizing Hackfests are an opportunity here. Here is a suggested new format:

– TDF selects a small of the tenderable topics (6 man days)

– TDF selects a “product owner” for the topic (could be an TDF employee or a qualified and motivated TDF member)

– TDF hires 2 certified developers from LibreOffices companies for 3 days each

– TDF offers 2 days “development training” to its members, but also to the general community: ideally we select four people for this.

– All are invited to a two day Hackfest.

– Hired consultants are expect to pair program with one of the volunteers on each day, with the hired person not distracted by other business and doing the main effort with the paired volunteer focusing on learning.

– On each day, one of the hired developers works on the projector of the room, allowing other partcipants to the Hackfest to observe and learn.

– Closing the day, each paired team will give a 5-15 minutes lightning talk on their progress and challenges over the day. This presentation should be done by the paired volunteer to the best of their ability and recorded e.g. by TDF staff for publication.

– Rest of the extra booked day should be used for a 1-hour prep Hangouts, follow ups and overtime.

– Selected paired volunteers should ideally be 50% certified or uncertified developers and at least 25% volunteers active in non-development areas (e.g. documentation, l10n …).

– Beyond this, it will be a “normal” Hackfest allowing others to mix an mingle.

What this might solve (hopefully)
=================================
– Controlling and the need to proof due diligence going away as it is performed right in the open and self-documented, reducing the vast redtape needed to set up and run tenders in the first place

– the involving the broader community is much more involved in this major aspect of the foundations work

– the community gets a much better transparency on the real cost and challenges
of development

– we approach a more iterative/agile approach without being hit by the challenges this usually implies for an NGO

– we provide clear and visible progress and effort on education for the community and the general public

– we might help onboarding of new developers and make LibreOffice more interesting for contribution

– if this works on this 4-6 man day scale, we might consider extending it (e.g. a three developer week[1] with prep is already a 21 man day project)

There has already been some internal feedback from a smaller circle I shared this with first: There was some concern (one pointing out this new format isnt free of challenges[1], one asking for bigger steps as this proposal was considered to small/iterative[2]), but beyond that the feedback was generally quite positive.

As such, Im looking for people who would like to join in and help giving this new format a try: The starting point would be organizing a broad, welcoming and well-organized Hackfest at a location easily reachable for many in the community: Thus at well-connected place in europe. It should also have local people on the ground, who are enthusiastic to make this a success.

If you are interested in helping with this, either as:

– someone on the ground helping to organize the Hackfest

– someone who wants to pair program with a hired certified developer at a Hackfest

– someone who is just interested in joining the Hackfest in general

– someone who helps fleshing out the details of this idea

feel free to contact me. I will try to set up a team of people interested in getting this off the ground. As noted above, if this proves to be successful, this might be the start of something excited and big bringing this community and project to a new level!

Another (final) note: This list is currently unfortunately less used than it should be to provide information on the proceedings of the foundation. The Board is mindful of that and tries to change this. This is a start. Feel free to share this message to those in the community who might have missed it as they are not (yet) subscribed to board-discuss@.

Best,

Bjoern

[1] But neither is the status quo — in many more ways.
[2] note that the two kinds of criticism pointed in exactly opposite directions

If you are interested in this effort, feel free to:

UPDATE: Time and date for the call are set: Sunday 2017-09-03 14:30 UTC, Talkyoo room 21 24 86 #. Please join!

LibreOffice 5.2.3 as snap from Day One

Meist scheint manches auf den ersten Blick unmöglich.
Manches ist es auch, doch es wäre tödlich, das selbst zu glauben solange noch nichts feststeht
und die Party zu verlassen, bevor sie losgeht.

— Die Sterne, Stell die Verbindung her

Yesterday, two nice things happened: For one, LibreOffice 5.2.3 has been released and secondly Ubuntu Core 16 has been released. But beyond that, something in the middle between these two has happened: LibreOffice 5.2.3 has been released to the stable channel of the snap store at the same day. Now LibreOffice has been in the snap store for some time — and has also been on the stable channel since the Ubuntu 16.10 release. But this is the first time the LibreOffice snap is released in sync with The Document Foundation announcing the general availability of the final downloads. This was possible even though I was on vacation yesterday: LibreOffice snap packages are now being build on launchpad, which simplifies a lot, and launchpad can be asked to populate the edge channel of the store. This is making life very easy. Having smoketested the amd64 build from that channel before, to release LibreOffice 5.2.3 to the beta/candidate/stable channels too all I had to do was push three buttons on a web interface and it was available to all.

Building on launchpad, I also had the opportunity to create builds for armhf and i386 along with the usual amd64 builds with little extra effort. If you are adventurous you are encouraged to test these builds too: Be aware though that these so far aren’t even smoketested, I havent looked at them at all yet, so use them at your own risk.

All in all, this is great progress: LibreOffice 5.2.3 is available to users of Ubuntu 16.10 and Ubuntu 16.04 LTS as a snap on the day of the upstream release. And beyond that on all other distributions where snap is available — quite a few these days.

Update: ICYMI here is how to get the LibreOffice snap: http://www.libreoffice.org/download/snap/ — although strictly speaking you dont need the --channel=beta option anymore now. I will fix that soon.