"Beta fix for foreign characters" breaks breadcrumbs in 1.8.1c again

[merktnichts]

I'm referring to this post (http://forums.opengarden.org/showpost.php?p=1510&postcount=64). Sorry for opening a new thread, but as it's not beta related anymore, the old thread would be the wrong place.

On 07-21-2007 I posted a fix for Hayes Beta to display foreign charachters in breadcrumbs properly. Unfortunately this fix seemed to be necessary for the betas only, I just downloaded 1.8.1c and just noticed that it's that fix which seems to break the stuff in the final versions again, just the opposite happens. Now there's too much conversion. The root of the problem was solved somewhere else in the meantime (after 1.8.1a, I guess), as it seems.

Now it displays: "Anfasstüte...e für Deki". It properly displays "Anfasstüte...e für Deki" if I remove the "fix" in 1.8.1c like this (just a revert to the "old" code):
function wfTruncateFilename($filename = '', $length = 16) {
if (strlen($filename) < $length) {
return $filename;
}
$fragmentLength = floor(($length - 3)/2);
$beginFragment = substr($filename, 0, $fragmentLength);
$endFragment = substr($filename, $fragmentLength * -1);
return $beginFragment.'...'.$endFragment;
}

So please trash the fix I posted, cause it seemed to be necessary for the betas only. To test it, just create a page called e.g. "Anfasstütentestseite für Deki" (don't think about the meaning of that page title, it doesn't make sense, but has two umlauts in it) and see how the breadcrumbs get displayed.

Sorry for confusion, I just noticed it today with 1.8.1c, no idea if it broke 1.8.1b as well. IIRC the fix worked ok in 1.8.1 & 1.8.1a. (It wasn't included there.)

[merktnichts]

Hrmmm, there seems to be more, but that seems to go deeper and these parts are too skilled for me, so please test the following page title:

"Anfasstütentestseite für Deki - ÄÖÜäöüß"

It produces "Anfasstüte...�Üäöüß" with the removed fix.

[SteveB]

Thanks for reporting it. We'll check it out.

[royk]

i haven't given it a thorough testing yet (i'll be back in the office tomorrow), but it looks like switching to PHP's multibyte string functions fixes the problem for my limited test cases so far:

function wfTruncateFilename($filename = '', $length = 16) {
if (mb_strlen($filename) < $length) {
return $filename;
}
$fragmentLength = floor(($length - 3)/2);
$beginFragment = mb_substr($filename, 0, $fragmentLength);
$endFragment = mb_substr($filename, $fragmentLength * -1);
return $beginFragment.'...'.$endFragment;
}

i'll set up a fresh 1.c test environment (my current one is one that's been patched time and time again, so i think the data may be in a different state than expected) and give this a deeper look tomorrow.

note: there was a missing set of db upgrades which may lead to some problems with foreign characters on upgraded systems. be sure that all your tables are using utf8_general_ci as their collation and utf_8 as their charsets. if not, you can simply do a:

alter table `tablename` convert to character set utf8;

[merktnichts]

mb_substr seems to work here too.

I didn't convert my DB as I started with Deki using Hayes Beta 2, so no "old" Gooseberry database here.

Thx again, Roy & Steve!