As one observant commenter noticed in our last post there’s a new Basic+ function called “BLen” in the version 10 compiler.  This is simply a synonym for the standard GetByteSize function, and was added to:

  1. Save me some typing effort (very important)
  2. Fit in with some of the other binary functions like BRemove and BCol2.

Of course, you may be wondering why GetByteSize/BLen is being used so much that I got tired of typing it?  It’s simply that as we progress through the v10 codebase we’re updating the code to be  “UTF8-safe” – i.e. we’re aiming to ensure that we don’t lose any performance when running in UTF8 mode, and a common Basic+ programming pattern for detecting a non-null variable is this:

   If Len( someVar ) Then
      // Variable is not null
   End

Variables in Basic+ are length-encoded, i.e. they cache the number of bytes that they occupy in memory.  When running in ANSI mode the Len statement simply returns this number (because 1-byte always equals 1 character) so if it’s zero you know you don’t have any data. However, because UTF8 is a multi-byte character-encoding format, the Len statement in UTF8-mode has to scan the contents of the entire variable to count the number of characters – it can’t use the cached byte-count.  This means that a simple check with Len could trigger this counting process when all you really want to know is if the variable contains data, and this could impact performance when dealing with large strings or arrays.

So, the best option is to use GetByteSize rather than Len, which always returns the cached byte-count regardless of ANSI or UTF8-mode, but as I don’t like typing very much you can now use BLen instead.

If you’re interested in writing UTF8-safe code and you’re not familiar with the Basic+ binary functions, you can find more details on them in a series of posts I wrote a few years ago on the Sprezzatura blog.  You may also want to check out the Internationalization section in the OI Coding Standards document too for some more UTF8-mode hints and tips.

(Disclaimer: This article is based on preliminary information and may be subject to change in the final release version of OpenInsight 10).