Archive for the ‘Weekly’ Category
While hard at work, it’s important to be visible so people don’t wonder if there’s anything happening… the open source development model is extremely good for this, things are just public all the time. Some may find this a source of stress or potential embarassment, but I think it’s great and overall makes for better quality code, products, and a nicer work environment.
To see what’s going on with OurDelta right now, we need to take a peek at https://code.launchpad.net/percona-patches where the Percona developers have been busy putting in lots of changes in both the 5.0.67 tree (mainly little improvements and fixes to existing patches) as well as working on the 5.1 ports of the 5.0 patches. Launchpad lets you subscribe to a particular branch, so you get notified when there’s a new commit, and see the changeset comment. That’s a very handy way to keep up to date.
For OurDelta, the first 5.1 release will very simply be when we have all the 5.0 patches working in 5.1; it’s taking a bit of time and of course it would be great if it took less time, but I think we can all agree that it’s the right thing to do. Or if you feel different, please do speak up!
For 5.0, there’s always little fixes as well as some new patches… so how often to do a build? Right now I think once every two months is decent. Again, if you feel different, please let us know! Simply join the https://launchpad.net/~ourdelta-developers list and put in your thoughts. Community participation (which I believe is the multilateral form of ‘contribution’) is really that simple. Thanks!
OpenSQL Camp 2008 has come and gone, and hooray again for Baron who came up with the idea and made most of it happen (but let’s not forget Sheeri!) Events such as these are always educational, but the most interesting stuff happens outside of the organised sessions (and this being an un-conf, they weren’t that strictly planned anyway 😉
For me (Arjen) a major chunk of the exercise was acquiring jetlag there and back with no days to spare either side, but I feel it was well worth it. I spent most of the time listening and talking with people rather than coding. It was a great opportunity to catch up with Monty, the Percona crew (Baron, Peter, Vadim, Tom, and more – there’s so many of them now!), Brian, Stewart, Jay, Pat, Eric and other Drizzlers, Sheeri, active OurDelta people like Nick and Rob, ex-Brisbanite Ronald, and of course Jim who thought he might learn something in the MVCC session (no I’m not going to explain that joke!)
The Percona patches have moved to Launchpad, so from now on they’re being developed in plain sight, available earlier for peer review, and more easily integrated into OurDelta builds. Thanks Vadim for making that happen! An excellent example of the Open Source development model – the resulting quality will be even higher! The Percona crew is also working on their end of the 5.1 porting of the patches, with priority given to the performance-related ones. OurDelta contains additional patches and features, so we have extra work anyway – all help is great and much appreciated!
OurDelta is, as stated previously, committed to doing 5.1 builds also, but we’re going to continue doing 5.0 builds for the foreseeable future as well. Most deployments are currently on 5.0, and the various enhancements in OurDelta builds provide breathing space (in terms of performance, and monitoring/tuning instrumentation) while people check out 5.1 and plan for a possible upgrade. And we’ll make sure that anything we put in 5.0 will also be available in 5.1, so that you don’t lose anything when you do upgrade. That’s our promise.
We’re currently looking at a few more platforms to build for, such as Solaris and Windows. The latter is mainly aimed at developers, who will certainly appreciate the extra info they can get out of an OurDelta build and thus make better performing applications!
Next to performance-related patches, instrumentation is and will remain a key focus for OurDelta. We want to get even more information from the server (without increasing disk I/O, contention or CPU load and yes that is possible), as it offers more breadth and depth than any external solution. And we reckon -and that’s us who deal directly with real-world deployments on a daily basis- that is well worth the extra effort!
This week saw the release of OurDelta patchset d7 build of MySQL 5.0.67, basically a cleaned-up update of the earlier (and first) OurDelta d6 build. The number of downloads/fetches within the first few hours surpassed the total number from the previous weeks.
Downloads and yum/apt-get repository fetches now always go via one of our mirrors, as obviously the main server can’t possibly handle all that attention! By default you just get sent to “somewhere on the planet”, although you can tweak your repo setup to only use specific mirrors. If you want to become a mirror for OurDelta, drop us a line and we’ll be happy to add you in; the more the merrier!
Ubuntu 8.10 Intrepid is now also supported. We welcome input on which additional platforms are desirable.
There was a podcast, where interviewer James Purser came up with an good description of what OurDelta is: “a new distro for MySQL”.
OurDelta development in the coming weeks will focus on 5.1. If you would like to get involved with this particular effort, join the ourdelta-developers group on Launchpad, and check out the recent mailing list archive. There’s more to it than just code; but getting started there is not as hard as it seems, and there’s plenty of helpful hands about!
This week we’ve been working on…
- The re-jigged build system, so we have a single patched baseline source tarball that’s then used for all the different builds, as well as being available for you to download and build yourself.
- Noarch distro rpm for all CentOS to more easily drop repo files into YUM.
- The repo/download mirror system, it’ll be ready to go from the next build (domain/paths will change slightly). Add your name to https://bugs.launchpad.net/ourdelta/+bug/285200 if you’d like to be a mirror.
- More documentation for existing (and new) patches. Currently WordPress mucks up the tabs when inside the docs pages, if you’re a WP guru who can fix this, please get in touch!
- Fixing up existing and updated patches. We really don’t want new reserved words introduced, and if it’s absolutely necessary it should be documented. Likewise, any enhanced feature should be backwards compatible if it relates to (for instance) existing configuration options. This is nearly done, and then we’ll do a new 5.0 build. Other “new” patches are set for the next build after that.
- Porting patches for 5.1. Since we don’t want you to lose any of the new features when trying 5.1, we’re porting the 5.0 patches that don’t yet have a 5.1 version. This does require some work, more volunteers are most welcome! In some cases a port will be waiting (or needs to be synced) for the work in 5.0.
- Bugs. Arjen found a bug in MySQL’s mysqld_safe behaviour (any version since 4.0). One active community member submitted a patch very quickly. This is what happens when there’s prospect of getting a contribution included in a build reasonably quick: people feel enticed to participate. And that’s great! We now reckon the fix needs to be in my_print_defaults not mysqld_safe, but the example is still valid.
- Grabbing the new Ubuntu 8.10 (Intrepid) images to set up build environments for it. Lots of people will stay with 8.04 LTS (Hardy) since that is a Long Term Support edition, but Intrepid is bound to be popular anyway.
- Pondering the InnoDB recovery code. Surely this can be made faster without a major rewrite. Sure there can be rollbacks also and that’s another matter, but dealing more efficiently with the redo would make a serious difference already. Do feel free to experiment and send in your patch!
It’s been about a week since the initial launch of the first 5.0 packages and the OurDelta website, so it’s time for some first impressions and an update on what’s going on right now…
- Immediately lots of hits on the site (from 67 distinct countries), most downloads and repo retrieves started happening a few days later.
- Interest spiked even more when we put up the Debian 4.0 (Etch) packages, apparently there was serious pent-up demand there!
- Lots of positive responses from clients, user groups, global MySQL community, and in fact also from within Sun/MySQL. That’s very nice, as it provides confirmation we’re doing something worthwhile!
- No negative responses, blog posts or articles that I know of (and yes I did search around a bit through Google, Technorati, etc) which is of course nice too, but rather unusual and thus surprising. Isn’t there always someone who doesn’t agree? 😉 and anyway, a bit of dissent and discussion is healthy.
- We submitted sessions for OpenSQLCamp, the linux.conf.au Open Source Databases Miniconf, and the MySQL Conference and Expo 2009 call for papers. Not just about the project, but also about using the toolchain and delving into MySQL Server code for non-gurus.
- Noteworthy: MySQL Performance Blog is a popular site and so quite a few referrals came from Baron‘s post there.
- As expected some bug reports were received, mainly to do with packaging details. Steve Walsh, Peter Lieverdink and others have been very active sorting out these issues and they’ll be included in the next builds.
- No OurDelta-related external bug reports about server operation itself, while people have reported running the builds in production. Excellent!
- The ourdelta-developers mailing list got a fair number of new participants. Do join too!
- #ourdelta IRC channel (on Freenode) not massively busy but alive and well, today Peter Lieverdink added an IRC bot to notify of OurDelta bug updates from Launchpad there, which is very useful too.
- We received kind offers for mirroring the download/repo files, James Iseppi is working on the rsync and other details for that.
- While organising our infrastructure for better patch management (Quilt) and the build workflow, we did find some bugs in existing patches. Most have been fixed, some are still being discussed. It’s nothing major but, for instance, we don’t want patches to create any new reserved words as that can break existing applications.
- The above has slowed down our upcoming release of 5.1 somewhat. The intent was to make sure that all relevant 5.0 patches would also be available for the 5.1 builds, so that an upgrade would not lose you any features you’ve now come to like! Some porting work is involved there because not all are available for 5.1. With the other bugs getting fixed first, that’s eating up some time. We believe that’s ok though, as the resulting quality is higher.
- While working on one such issue (getting rid of startup warnings from one patch on 32-bit builds) we encountered an existing MySQL bug which had been hiding somewhat mislabeled. While the visible symptoms were benign, Antony Curtis was able to add insight to the root cause of things, which indicated that on some platform more serious problems could occur in the server. The story is essentially about old code and old platforms, but just in case, we made a patch and it’ll be in the next OurDelta build – to make sure it definitely can’t cause any trouble. All info and the patch have also been submitted to the MySQL bug, and the MySQL QA and bugs people have been very kind and helpful.
- Eric Bergen of Proven Scaling submitted a patch, and David Stainton from Spinn3r supplied two more that he’d extracted from the big Google patches.
- Other excited individuals have been delving into the repositories to play with various bits, or are doing some benchmarks. All good stuff!