Archive for October, 2008

This week in OurDelta – Vol 2

This week we’ve been working on…

  • The re-jigged build system, so we have a single patched baseline source tarball that’s then used for all the different builds, as well as being available for you to download and build yourself.
  • Noarch distro rpm for all CentOS to more easily drop repo files into YUM.
  • The repo/download mirror system, it’ll be ready to go from the next build (domain/paths will change slightly). Add your name to https://bugs.launchpad.net/ourdelta/+bug/285200 if you’d like to be a mirror.
  • More documentation for existing (and new) patches. Currently WordPress mucks up the tabs when inside the docs pages, if you’re a WP guru who can fix this, please get in touch!
  • Fixing up existing and updated patches. We really don’t want new reserved words introduced, and if it’s absolutely necessary it should be documented. Likewise, any enhanced feature should be backwards compatible if it relates to (for instance) existing configuration options. This is nearly done, and then we’ll do a new 5.0 build. Other “new” patches are set for the next build after that.
  • Porting patches for 5.1. Since we don’t want you to lose any of the new features when trying 5.1, we’re porting the 5.0 patches that don’t yet have a 5.1 version. This does require some work, more volunteers are most welcome! In some cases a port will be waiting (or needs to be synced) for the work in 5.0.
  • Bugs. Arjen found a bug in MySQL’s mysqld_safe behaviour (any version since 4.0). One active community member submitted a patch very quickly. This is what happens when there’s prospect of getting a contribution included in a build reasonably quick: people feel enticed to participate. And that’s great! We now reckon the fix needs to be in my_print_defaults not mysqld_safe, but the example is still valid.
  • Grabbing the new Ubuntu 8.10 (Intrepid) images to set up build environments for it. Lots of people will stay with 8.04 LTS (Hardy) since that is a Long Term Support edition, but Intrepid is bound to be popular anyway.
  • Pondering the InnoDB recovery code. Surely this can be made faster without a major rewrite. Sure there can be rollbacks also and that’s another matter, but dealing more efficiently with the redo would make a serious difference already. Do feel free to experiment and send in your patch!

OurDelta looking for a logo

Ideas welcome!

General idea… base: Delta symbol (with thicker line on right hand side – a delta it’s not a regular triangle), plus one or more of the following:

  • something depicting deltas: incremental small changes;
  • something depicting a river delta: where streams come together before flowing into ocean;
  • something depicting community: people working together, participating, communicating.

If you can draw even a little bit, rough scribbles are most welcome! We have a good artist who can turn that into magic. And, you don’t have to go with the above… come up with something else suitable!

See also https://bugs.launchpad.net/ourdelta/+bug/284161 where we’re tracking this; you will find other suggestions from people, including ideas that have been dismissed for visual or other reasons.

This week in OurDelta – Vol 1

It’s been about a week since the initial launch of the first 5.0 packages and the OurDelta website, so it’s time for some first impressions and an update on what’s going on right now…

  • Immediately lots of hits on the site (from 67 distinct countries), most downloads and repo retrieves started happening a few days later.
  • Interest spiked even more when we put up the Debian 4.0 (Etch) packages, apparently there was serious pent-up demand there!
  • Lots of positive responses from clients, user groups, global MySQL community, and in fact also from within Sun/MySQL. That’s very nice, as it provides confirmation we’re doing something worthwhile!
  • No negative responses, blog posts or articles that I know of (and yes I did search around a bit through Google, Technorati, etc) which is of course nice too, but rather unusual and thus surprising. Isn’t there always someone who doesn’t agree? ;-) and anyway, a bit of dissent and discussion is healthy.
  • We submitted sessions for OpenSQLCamp, the linux.conf.au Open Source Databases Miniconf, and the MySQL Conference and Expo 2009 call for papers. Not just about the project, but also about using the toolchain and delving into MySQL Server code for non-gurus.
  • Noteworthy: MySQL Performance Blog is a popular site and so quite a few referrals came from Baron‘s post there.
  • As expected some bug reports were received, mainly to do with packaging details. Steve Walsh, Peter Lieverdink and others have been very active sorting out these issues and they’ll be included in the next builds.
  • No OurDelta-related external bug reports about server operation itself, while people have reported running the builds in production. Excellent!
  • The ourdelta-developers mailing list got a fair number of new participants. Do join too!
  • #ourdelta IRC channel (on Freenode) not massively busy but alive and well, today Peter Lieverdink added an IRC bot to notify of OurDelta bug updates from Launchpad there, which is very useful too.
  • We received kind offers for mirroring the download/repo files, James Iseppi is working on the rsync and other details for that.
  • While organising our infrastructure for better patch management (Quilt) and the build workflow, we did find some bugs in existing patches. Most have been fixed, some are still being discussed. It’s nothing major but, for instance, we don’t want patches to create any new reserved words as that can break existing applications.
  • The above has slowed down our upcoming release of 5.1 somewhat. The intent was to make sure that all relevant 5.0 patches would also be available for the 5.1 builds, so that an upgrade would not lose you any features you’ve now come to like! Some porting work is involved there because not all are available for 5.1. With the other bugs getting fixed first, that’s eating up some time. We believe that’s ok though, as the resulting quality is higher.
  • While working on one such issue (getting rid of startup warnings from  one patch on 32-bit builds) we encountered an existing MySQL bug which had been hiding somewhat mislabeled. While the visible symptoms were benign, Antony Curtis was able to add insight to the root cause of things, which indicated that on some platform more serious problems could occur in the server. The story is essentially about old code and old platforms, but just in case, we made a patch and it’ll be in the next OurDelta build – to make sure it definitely can’t cause any trouble. All info and the patch have also been submitted to the MySQL bug, and the MySQL QA and bugs people have been very kind and helpful.
  • Eric Bergen of Proven Scaling submitted a patch, and David Stainton from Spinn3r supplied two more that he’d extracted from the big Google patches.
  • Other excited individuals have been delving into the repositories to play with various bits, or are doing some benchmarks. All good stuff!
Thus, you may agree, plenty going on so far and not at all bad for a first public week of a new project – and as is usually the case, a release doesn’t mean the work is done for a while, instead it immediately creates more! ;-)

Would you prefer InnoDB to be the default storage engine?

This poll was posted last week on Arjen’s blog and is still open for a few more days to receive your vote. The comments on the original post contain some interesting considerations either way, and many in fact non-technical. Anyway, please add your vote, and optionally your thoughts to the comment thread!

5.0.67.d6 for Debian Etch

Build for Debian Etch done and available, simply click to the new Debian page for the info to set up your repo. Thanks to Peter Lieverdink (cafuego on the #ourdelta IRC channel) for the fast work!

Your interest (from 53 different countries according to Google Analytics!) and response has been great! Among the feedback, saw a few bug reports on minor issues in the packaging, most of which have already been addressed for the next build (bug handling progress is all visible on Launchpad). And discussion about various patches that are “around”; some mails came to people directly but we try to encourage using the oursql-developers team mailing list.

Launchpad, IRC, new patches

The setup on Launchpad is better organised (Launchpad rocks!), and we already have the first feedback in the form of bugreports – mainly on the packaging and repo information. That’s excellent! Thanks Vladimir Cherednichenko, Steve Walsh, and Peter Lieverdink! Peter actually deserves a special mention as he has put in a lot of work on the debian packaging.

Also, because sometimes instant banter is useful, there’s now a #ourdelta channel on Freenode. Naturally, real discussion should just happen on the ourdelta-developers team and list. It’s really easy to join.

Antony Curtis did just that (join) and already put one of his many patches in: this one fixes issues which occur while a thread is running ::open() or ::create() on any storage engine. Typically the LOCK_open mutex is held and it could block other threads from doing work. While it needs some work, it might be a worthwhile modification. Mutex contention is an important issue for scaling, and this one is at a fairly high level.

Release 5.0.67.d6

With the build system, repos, patches and launchpad sorted, we can release! First up is a build of 5.0.67, using patchrev d6, for i386/x86_64. This is basically Percona patchset 5 with a few fixes plus the Sphinx storage engine. It even passes the testsuite ;-)

Other useful links:

Now, this being a first release into the wild, there are bound to be some glitches. Most likely in the packaging (and the repo instructions) but possibly also in the builds themselves. Trust comes over time, right? That’s ok, you can help! Simply report any issue using the above bugs link! And please do report success also, although that’s done best through the mailing list or a blog.

In case you’re wondering, 5.1 and beyond are also coming, with more interesting goodies! And you can help! Your first step is to join the mailing list.

A big thanks to everybody involved so far, this was a great team effort! (see the about page for a non-exhaustive list)