Johannes Schlüter

Yesterday was a quite thrilling day for the PHP development team and led to some imprecise news articles so let's take a look at what happened: Over the last months many of the core contributors agreed that the current approach to bring Unicode into PHP's engine wasn't the right approach and a good thing would be to rethink it from the start. By a provocative move of one contributor the stalled situation got some more movement and Rasmus declared the current implementation to be discontinued to restart.

The past

When the foundation of what should have become PHP 6 was created a decision was made to use UTF-16 as internal encoding for "everything" inside the engine. The choice for UTF-16 was made due to the fact that PHP 6 would use the ICU library which is focused on offering UTF-16 string functions. By using UTF-16 as default encoding we'd have to convert the script code and all data passed from or to the script (request data, database results, output, ...) from another encoding, usually UTF-8, to UTF-16 or back. The need for conversion doesn't only require CPU time and more memory (a UTF-16 string takes double memory of a UTF-8 string in many cases) but makes the implementation rather complex as we always have to figure out which encoding was the right one for a given situation. From the userspace point of view the implementation brought some backwards compatibility breaks which would require manual review of the code. These all are pains for a very small gain for many users where many would be happy about a tighter integration of some mbstring-like functionality. This all led to a situation for many contributors not willing to use "trunk" as their main development tree but either develop using the stable 5.2/5.3 trees or refuse to do development at all.

The present

Yesterday the stagnation created by the situation has been resolved and it was decided that our trunk in svn will be based on 5.3 and we'll merge features from the old trunk and new features there so that 5.3 will be a true stable branch. The EOL for 5.2 has not yet been defined but I suggest you to really migrate over to 5.3, which usually can be done with very little work, as soon as possible.

The future

Right now we're starting different discussions to see what kind of Unicode support we really want. Many contributors react positive on a proposed "string class" which wraps string operations in Unicode and binary forms without going deep in the engine. In my opinion such an approach might also be a way to solve some of the often criticized inconsistencies in PHP's string APIs without the need to break old code. (new code uses the new class, old code the old functions) But that idea is far from a proper proposal or even the implementation, current status is about refocusing the development and get the requirement and design discussions going. By that it's a bit early to decide whether the next version of PHP will be called PHP 5.4, PHP 6 or maybe even PHP 7.

PHP is alive and kicking!

Just a quick heads-up: After quite some time from RC1 PHP 5.3.1RC2 has finally been packaged and released. The PHP bug tracker is welcoming reports about issues, I also welcome positive feedback.

Downloads:

• Source:
  http://downloads.php.net/johannes/php-5.3.1RC2.tar.bz2
  http://downloads.php.net/johannes/php-5.3.1RC2.tar.gz
• Windows binaries:
  http://windows.php.net/qa/

(This release candidate is not meant to be used in production systems, wait for the final release for that but please test this version)

In my blog post for the 5.3 release I already mentioned it and some others posted it, too: We're planning a release party in Munich, currently we have around 60 people who registered. That's the official announcement:

We like to invite you to the PHP 5.3 release party on Friday, the 17th of July in Munich. The release party offers a chance to talk to other PHP enthusiasts and enjoy that PHP is alive and kicking. If you just can't resist a decent barbecue together with some beer and other drinks you are also welcomed.

The happening will take place at the Waldwirtschaft beer garden irrespective of the weather. We will meet at 19:00 o'clock - open end.
The location is famous for its huge beer garden (2,500 available seats, a children’s playground) and its typical Bavarian but also international food. On sunny weather you may even enjoy live-music and listen to the sounds of Jazz, Blues, Swing or Dixi.

Catering will be provided and as a special delicacy you may enjoy a suckling pig!

If you like to join the event please register at PHPUG-Munich Wiki and follow it for updates.
Alternatively you may register at Facebook as well and follow this for updates.

For any questions please visit IRC channel: #phprp on irc.uni-erlangen.de.


The PHP 5.3. BBQ release party is sponsored by:

   
 

Supporters for the PHP 5.3 BBQ release party are:

• Sun Microsystems
• PHPUG.de

It was a long run and I'm sure it felt like an eternity for many - for me it certainly did. PHP 5.3 was branched of over two years ago and finally is ready to be called 5.3.0.

The php.net website and many other blogs discuss the features - from often loved closures, to well discussed namespaces to the sometimes hated goto - so I think I don't have to this here but instead can focus on that what really matters:

• Thanks to all the developers - Without them no new features would be there.
• Thanks to the documentation team - Without them one would have to decipher the NEWS file and guess what exactly is meant.
• Thanks to all participants during the Testfest! - During the Testfest we received many good tests for our regression test suite. Many of these tests represent what people do, not what developers think they should do which is important to reduce the risk of regressions. I hope people don't stop sending tests!
• Thanks to all the testers who reported bugs, and to the ones who reported that their applications worked without issues! - Regression tests are important but many possible issues can only be found in real life scenarios with real applications.
• Thanks to the whole community to make pressure on the developers to finally get it out as everybody is waiting for the new features.
• And last, but certainly not least: Thanks to Lukas! - Handling developers who are passionate for their work can be tough. Lukas made sure that the developers focus on what really matters and really helped me with all the work needed

So with that: In case you didn't do already: Browse over to php.net and grab your copy, it's for free!

If you want to celebrate the release and are close to Munich: We're planing a PHP Release Party on July 17th, details on that will follow.

As others wrote, or as you can read on php.net or in my twitter feed we've released the second, long awaited, release candidate of PHP 5.3.0.

5.3 is  rather big release including support for namespaces, closures, phar archives, internatioalization support via the new intl extension, improved SQLite support, mysqlnd as backend for the MySQL exensions, impressive performance improvements, ... and tons of other bigger and minor things.

Even though this server is running 5.3 already it's not suggested to be used in production evironments, yet but I'd really like to encourage everybody to test it and give feedback! I'm also interested in positive feedback, not only bug reports to support my good feeling about this release

It's done 5.3 finally, after quite some discussions, reached beta state. So if you're not already on it it's now the perfect time to test it. Problems found now can be fixed before we mark it stable. Issues not find might need workarounds in your software which will cost you more time in the end. Or the way I often put it: Test now and complain or don't test and live with the consequences - we do our best but we certainly can't cover every edge cases especially in a version like 5.3 where we did quite some internal cleanup and added tons of new features.

The NEWS entry file is, compared to some previous alpha versions, quite short this time but still worth reading if you tested a previous alpha release.

Source tarballs can be found in my download directory, Windows builds in the Windows QA area.

Happy testing!

Over the last few days I already mentioned a few hidden gems from PHP 5.3. Now at Christmas I wanted to take a look at some new language feature of the upcoming PHP version:

Added "jump label" operator (limited "goto"). (Dmitry, Sara)

The entry is a imprecise on purpose, since it's not to be advertised too much to not be abused too much, but well, you're reading this on Christmas instead of spending time with your family, so I guess you're a geek and know already: PHP 5.3 introduces not only namespaces but also goto. Yes it's a "goto label;" the limitations i, mentioned in the NEWS entry, are: You can only jump within the same execution unit (function or global part of the same file file) and you can't jump into loops.

When you know about goto I'm sure you know it's bad, so why did we added? - Well there's a very limited set of problems where it's ok. One is generated code, a code generator using goto can be written way better than without goto and nobody is supposed to read that code anyways. The second situation is when having a longer piece of code, where situations might occur where you cancel execution sin the middle of the code but want to do some cleanup nonetheless. A short pseudo-code example:

<?php
function process_file($filename) {
    $fp = fopen($filename, "r");
    if (!$fp) {
        goto cleanup;
    }

    $row = fread($fp, 1024);
    // do something with the row
    if ($error_while_processing) {
        goto cleanup;
    }

    $a_few_bytes = fread($fp, 4);
    // do something again ...
        if ($error_while_processing) {
        goto cleanup;
    }

    /* ... */cleanup:
    fclose($fp);
}

?>

There are alternatives available, like wrapping this all in loops and using break or wrap it in an try {} catch block and throw exceptions, but goto can be cleaner. So have fun and use it with care!

For the second item in my series about new 5.3 stuff I decided to take a look at Etienne Kneuss's data structure implementations.

In the programming world there are quite a few well understood and explored data structures. Which are commonly used in tons of applications, still the only things PHP offered till 5.3 in regards to structuring data were arrays (more precise: hash tables) and objects. So people had to either abuse them for other structures or implement the structures themselves on top of these. Thanks to Etienne things now are changing and PHP's Standard PHP Library (SPL) extension will offer quite a few standard implementations of data structures. The implemented data structures are implemented using these classes:

• SplDoublyLinkedList
• SplStack
• SplQueue (SplPriorityQueue)
• SplHeap (SplMinHeap, SplMaxHeap)
• SplFixedArray

Except for SplFixedArray the ideas behind these classes should be clear. If not you can read about them over at Wikipedia or in basically every general introduction to software programming book. For the exact naming and details of the implementation chosen for PHP you can read the fine manual. So why should you use these instead of implementing your own? - There are a few reasons the important ones are:

• Why do work others have done before again and again? - Code used in more places is better tested and has less bugs.
• Having code implemented native in C is faster than doing this in PHP userland.
• Using default data structures makes the code more maintainable

The only question left there might be: "Ok, that's nice and cool and I could use them, but I have to support older PHP versions for at least some time. What can I do?" - The solution there is easy as well: Most stuff in SPL doesn't do any special trick that requires an implementation in C but can also be implemented in userland. In fact most stuff is tested as userland code first to design the interfaces nicely. Others are implemented in PHP after being implemented in C as kind of documentation. These implementations are distributed with PHP source releases or via CVS, so you can simply take that code and use it.

But now let's get back to the one class which might not be clear: SplFixedArray. SplFixedArray is an implementation of a data structure similar to PHP's arrays with a big limitation: It only supports numeric indexes in a predefined range (starting at 0 and going up) like the ArrayObject from 5.2 it's class based so it can't be used with all thousands of array functions PHP offers, but within it's domain and fucntionality it's very fast and uses less memory than classic PHP arrays. so if you have a big array where numeric indexing is enough: Use it!

PHP 5.3.0 Alpha 3 has recently been released and marks the feature-complete set of stuff the upcoming release will offer. There was lots of talk about namespaces so other stuff could easily get lost in the long NEWS file so I plan to present some of the more hidden gems, sometimes just a new function, sometimes new language constructs. This series is not meant to be complete but some personal choice, these blog postings are also no replacement for documentation, but just pointers. My goal is that you try out 5.3 right now so we can fix bugs as soon as possible before releasing it

The NEWS file has a quite short entry for my first subject:

Added fileinfo extension as replacement for mime_magic extension. (Derick)

So what's fileinfo? - Fileinfo is a solution for a quite common problem: Consider you offer downloads and want to set the correct Content-Type HTTP header. How do you get the mime type to use? Many applications go by the file extension, keep a list of them and then set the header accordingly. Problem there: You have jsut a very limited set of extensions in your list and so chances are high your file isn't there. Other workarounds include calling "file" using exec() or mime_content_type() which has been deprecated for ages.

So why is it better than the old mime_content_type()? The biggest benefit is simpler configuration, for using the old function you had to configure a magic file somewhere, fileinfo includes that file so there's no need for special configuration. Additionally fileinfo can not only work on files in the filesystem but, for instance, also on strings in memory. So if you have your data in memory finfo_buffer() will tell you the mime type of some variable content or you can use PHP stream wrappers.

For people with legacy code there's a nice change included: mime_content_type() is now using fileinfo, too, internally so people using it benefit from the improvements in fileinfo without any trouble. For getting started: Take a look at the fine manual which includes examples to get you started.

Mal ein deutsches Posting zu PHP: Im Rahmen der "Weg mit magic_quotes"-Diskussion muss ich einfach Karl Valentin zitieren:

Mögen hätt ich schon wollen, aber dürfen hab ich mich nicht getraut.

Ever wondered what your PHP application and MySQL actually do? An experimental mysqlnd branch will give you full access to the network communication stream. Using a custom PHP stream filter you can then intercept the communication ... but let's start at the beginning:

When talking about mysqlnd - the mysql native driver for PHP - we always mention the fact it's native in a way that we're, when possible, using PHP infrastructure. The most common example here is the memory management. By directly using PHP's memory we can avoid unnecessary copies of data from the MySQL Client Library's memory into PHP memory.

<?php
$mysqli = mysqli_connect("localhost", "root", "", "test");
$stream = mysqli_conn_to_stream($mysqli);
stream_filter_append($stream, "mysql.server", STREAM_FILTER_READ);
?>

But there's more what we're doing. We're also using PHP's stream abstraction layer. From a development perspective the benefit is that we're using a tested abstraction from different stream implementations by different operating systems instead of writing our own. But, again, there's more to it: We can export the communication stream to PHP userland. We hesitated about exporting it for some time as it can be quite dangerous and you might easily corrupt the client-server- communication.

As Ulf mentioned during his IPC talk I recently pushed a mysqlnd branch to launchpad which adds a userspace function to mysqli which returns a PHP stream for a connection. Using that stream you can now send your own requests to the server and wait for the response. That might be nice in a way, but I guess you most likely won't have use for that. PHP streams allow you to do more: PHP streams give you the possibility to add filters to a stream. These filters allow you to intercept packages which are sent or received , read them, change them or do whatever you like. A very simple filter can be found on the launchpad site, mentioned above. That filter simply prints the information after replacing unprintable (binary) characters by dots.

Once again: Just a small step, the next one is to decode the MySQL protocol. For that I've written a simple decoder for the MySQL protocol, not complete, but enough to give an idea. The script, including the decoder and some sample code using it, is, as a sample, part of the branch. When running you will get some output like

Query:
-> 0 59: QUERY: SELECT TABLE_SCHEMA FROM INFORMATION_SCHEMA.TABLES LIMIT 1
<- 1 1: DATA
<- 2 52: FIELD INFO
<- 3 5: EOF
<- 4 19: DATA
<- 5 5: EOF
Invalid Query:
-> 0 29: QUERY: ghdfjtgfdrs tztr ttgdszthtdr
<- 1 183: ERROR: 2000You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ghdfjtgfdrs tztr ttgdszthtdr' at line 1
Prepare:
-> 0 61: PREPARE: SELECT TABLE_NAME FROM INFORMATION_SCHEMA.STATISTICS LIMIT 2
<- 1 12: OK
<- 2 52: FIELD INFO
<- 3 5: EOF
Ping:
-> 0 1: PING
<- 1 7: OK
Execute:
-> 0 11: EXECUTE
<- 1 1: DATA
<- 2 52: FIELD INFO
<- 3 5: EOF
<- 4 15: OK
<- 5 15: OK
<- 6 5: EOF
EOF
-> 0 5: CLOSE_STMT
-> 0 1: QUIT

As one can see: The protocol isn't fully decoded yet so this all might be extended but for me it served the purpose well enough. For making real use out of this we're thinking about exporting the protocol decoder which exists within mysqlnd to PHP userland.

What are your ideas for such a feature? - Sending different queries to different servers? Rewriting queries? Sharding? Replication? Easy scaling of your application while refactoring your application? Let us know!

As allow of you will have read a namespace separator for PHP was chosen. The discussions lasted more than 3 years and yes, we know most of the reasons why the chosen one is bad and §%§$&"& is way better before wasting your and our times by writing to mailing lists please read the archives. 90% of the proposals fail for simple technical reasons, 10% might technically work but have other problems which were considered. So: Decision made, let's go on and let's make sure the developer's list is there for doing productive stuff, 5.3 is delayed enough.

Thank you!

End October - time for the traditional meeting of the German and International "PHP family" - end of October? - Yes, the International PHP Conference is a bit earlier this year. Additonally the organizers moved the conference away from the industrial area of Mörfelden to the center of Mainz which sounds quite promising. Although I'll spend only around 24hrs at Mainz I'm looking really forward to the conference next week.

On Thursday morning I'll give a presentation about PHP 5.3, which will be quite interesting as one of the biggest features, namespaces, is still undergoing heavy discussions and the final syntax probably won't be clear when presenting - fortunately PHP 5.3 is much more than namespaces!

Sun will also be present at the conference, so if you're looking for an open source PHP IDE you might talk to Petr and Wen about the upcoming NetBeans 6.5 release which will feature PHP support, if you're running a startup company you might talk to Stefan Schneider who will represent the Startup Essential Program which has interesting discounts on Sun products.

Unfortunately I won't be able to attend Brian Aker's keynote about Drizzle or Ulf's session about new and hot stuff in mysqlnd, one of the new feature in PHP 5.3.

hm, I haven't blogged for a while, and no, Twitter, which I still don't get, is not an excuse, maybe I don't like telling every tiny thing I do, maybe that's the reason I don't get twitter ... anyways: In roughly 10 hrs I'll leave to St. Augustin to attend FrOSCon and give a presentation about PHP 5.3. Pre-Registration is closed but tickets (5 Euro for two days of fun) can be bought right at the entrance.

As most of you might have seen we recently announced the first alpha of PHP 5.3.0. The major changes are listed in the announcement. Source tarballs can be found on PHP's downloads site, as it's the first release I packaged I'm especially interested in feedback, whether it works or not . But we didn't only have changes on the source builds: A new team took over the creation of Windows builds. As they also did a major update to the Windows build architecture (supporting newer compiler versions on Windows and added experimental support for 64bit platforms) the build process got a bit delayed but builds should be available soon.

Remember: It's a release we packaged to get feedback from our users. So please test it. If you find issues please report them - the sooner we know about issues the sooner they can be addressed. Although our test coverage increased we can't cover all cases in our tests so please test it now and don't wait for the stable release, once the stable release is out your edge case might trigger a bug on a productive system, bugs reported now can be fixed before we release a final version.

If you want to give back to the PHP project - hey you probably make a living out of software you get for free - and can't fix bugs or assist in a similar way you probably could help our documentation team with improving the documentation, there are a few features without proper documentation, yet.