XmlDictionary bug

I think this is a bug in the .Net framwork.

When you serialize a class you convert its properties and values into a big string of xml, or a blob of binary data that uses less memory. The data will always contain the name of the class, xml namespaces and the names of the properties. We could make the data shorter if we substituted tokens for all these strings.

This code...

 writer = XmlDictionaryWriter.CreateBinaryWriter(stream, dic)

..will create an XmlDictionaryWriter that uses the XmlDictionary provided to perform this substitution. The problem is, that when I tried it, the binary data was the same length as when I did it without the dictionary -- the dictionary did nothing.

Digging into the reference source I discovered that the serializer will call TryLookup on the XmlDictionary to get the data required for the substitution:

 public virtual bool TryLookup(XmlDictionaryString value, out XmlDictionaryString result)
        {
            if (value == null)
                throw System.Runtime.Serialization.DiagnosticUtility.ExceptionUtility.ThrowHelperError(new ArgumentNullException("value"));
            if (value.Dictionary != this)
            {
                result = null;
                return false;
            }
            result = value;
            return true;
        }

 The problem is that this was always true:
value.Dictionary != this....

Nearly every instance of XmlDictionaryString had it's own dictionary at the time of serialization. If you test the XmlDictionaryStirng objects just after creating them, then they have the expected dictionary - the one that you set. It must get changed during serialization.

I'm using VB.Net, so I simply re-implemented that class and skipped that check, see below.


The dictionary also useful to find out exactly which strings can be tokenized - just dump out value.value to get the parts of the xml that it wants to shorten.

Imports System.Collections.Generic

Imports System.Xml

Public Class MyDIck

  Implements IXmlDictionary

  Dim lookup As Dictionary(Of String, XmlDictionaryString)

  Dim strings() As XmlDictionaryString

  Dim nextId As Integer

  Sub New()

    lookup = New Dictionary(Of String, XmlDictionaryString)

  End Sub

  Public Sub New(capacity As Integer)

    lookup = New Dictionary(Of String, XmlDictionaryString)(capacity)

    strings = New XmlDictionaryString(capacity - 1) {}

  End Sub

  Public Function Add(value As String) As XmlDictionaryString

    Dim str As XmlDictionaryString = Nothing

    If lookup.TryGetValue(value, str) = False Then

      If strings Is Nothing Then

        strings = New XmlDictionaryString(3) {}

      ElseIf nextId = strings.Length Then

        Dim newSize = nextId * 2

        If newSize = 0 Then newSize = 4

        Array.Resize(strings, newSize)

      End If

      str = New XmlDictionaryString(Me, value, nextId)

      strings(nextId) = str

      lookup.Add(value, str)

      nextId += 1

    End If

    Return str

  End Function

  Public Function TryLookup(xds As XmlDictionaryString, ByRef result As XmlDictionaryString) As Boolean Implements IXmlDictionary.TryLookup

    Return lookup.TryGetValue(xds.Value, result)

  End Function

  Public Function TryLookup(key As Integer, ByRef result As XmlDictionaryString) As Boolean Implements IXmlDictionary.TryLookup

    If key < 0 OrElse key >= nextId Then

      result = Nothing

      Return False

    End If

    result = strings(key)

    Return True

  End Function

  Public Function TryLookup(value As String, ByRef result As XmlDictionaryString) As Boolean Implements IXmlDictionary.TryLookup

    Return lookup.TryGetValue(value, result)

  End Function

End Class

HDS_FILTERBAR

There’s a ListViewFilter control in C# on codeproject from 2003. It displays a filter bar in the column headers by setting HDS_FILTERBAR.

If we start off with a ListView showing some random data in a few columns:

And set the HDS_FILTERBAR style, then it will display the filter bar:

But there’s a problem – it hasn’t resized the header controls – they need to be taller to fit the text in properly. The codeproject code makes the column resize using a hack. Here’s how it shows the filter bar:

// set/reset the flag for the filterbar
if ( hdr_filter ) style |= HDS_FILTR;
else style ^= HDS_FILTR;
SetWindowLong( Handle, W32_GWL.GWL_STYLE, style );

And here’s the self-confessed kludge that gets it to resize:

// now we have to resize this control.  we do this by sending
// a set item message to column 0 to change it's size.  this
// is a kludge but the invalidate and others just don't work.
hdr_hditem.mask = W32_HDI.HDI_HEIGHT;
SendMessage( Handle, W32_HDM.HDM_GETITEMW, 0, ref hdr_hditem );
hdr_hditem.cxy += ( hdr_filter ) ? 1 : -1;
SendMessage( Handle, W32_HDM.HDM_SETITEMW, 0, ref hdr_hditem );

It’s sending a message to alter a property of the header – using HDI_HEIGHT. If we look at the windows header files to see how it’s defined:

#define HDI_WIDTH               0x0001
#define HDI_HEIGHT              HDI_WIDTH

It’s actually the same as the width. The kludge increases the width of the column header by 1 when the filter is shown, and decreases it when the filter is not shown. Altering the width must be enough to make windows resize the control.

Searching for alternate solutions I found one that mentions using MoveWindow to resize the header control, so I tried that. Unfortunately when you resize the header it then overlaps the parent listview, hiding one or two of the ListViewItems by overlapping them.

The info needed to do it properly is in the “Header Controls” topic in the MSDN library, in the “header control size and position” paragraph. We have to send the HDM_LAYOUT message to the control, specifying the bounds of the parent. On return we get a WINDOWPOS structure which tells us the best bounds for the header so that it will sit in the bounds of the parent. All we then have to do is to resize the header control using SetWindowPos, and alter the bounds of the parent control so that its top lies below the header and its height isn’t too big.

HDLAYOUT layout = new HDLAYOUT();
RECT rect = new RECT();
rect.Right = parent.ClientSize.Width;
rect.Bottom = parent.ClientSize.Height;
layout.prc = Marshal.AllocHGlobal(Marshal.SizeOf(rect));
Marshal.StructureToPtr(rect, layout.prc, true);
layout.pwpos = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(WINDOWPOS)));
lresult = NativeMethods.SendMessage(Handle, HDM_LAYOUT, IntPtr.Zero, ref layout);
Marshal.FreeHGlobal(layout.prc);
WINDOWPOS pos = (WINDOWPOS)Marshal.PtrToStructure(layout.pwpos, typeof(WINDOWPOS));
Marshal.FreeHGlobal(layout.pwpos);
bool res = NativeMethods.SetWindowPos(Handle, IntPtr.Zero, pos.X, pos.Y, pos.Width, pos.Height, 
    SetWindowPosFlags.NoMove | SetWindowPosFlags.NoZOrder | SetWindowPosFlags.FrameChanged);
res = NativeMethods.SetWindowPos(parent.Handle, IntPtr.Zero, 0, pos.Height, parent.Width, parent.Height - pos.Height, 
    SetWindowPosFlags.NoMove | SetWindowPosFlags.NoZOrder | SetWindowPosFlags.FrameChanged);
ClearAllFilters();  

Now the filterbar is shown with the correct size, it doesn’t overlap the top of the parent listview, and it resizes correctly when the filter bar is removed.

Structures that vary in size on x86/x64 - what to do?

Some structures can be difficult to marshal, as their size varies on x86 compared to x64:

typedef struct _TBBUTTON {
   int         iBitmap;
   int         idCommand;
   BYTE     fsState;
   BYTE     fsStyle;
#ifdef _WIN64
   BYTE     bReserved[6]     // padding for alignment
#elif defined(_WIN32)
   BYTE     bReserved[2]     // padding for alignment
#endif
   DWORD_PTR   dwData;
   INT_PTR          iString;
} TBBUTTON, NEAR *PTBBUTTON *LPTBBUTTON;

The layouts in x86 and x64 would look like this:

In C the standard is for, say, an int to stay within a 4 byte boundary of the start of the structure. A short would
not cross a 2 byte boundary. Bytes can go where they like (well, they shouldn't cross the byte boundary, which I
guess they could by having bits spread over 2 bytes!) 

A pointer on x86 is a 4 byte value, and shouldn't cross a 4 byte boundary. A pointer on x64 is an 8 byte value, and
shouldn't cross an 8 byte boundary. Phew. Enough about byte boundaries...

TBBUTTON causes problems as it has the two BYTE fields in the middle. We need to pad, to get the dwData field
to start at a suitable location. Once it is in the correct place, iString will be fine.

With .Net we have a few ways to arrange structures. We can use StructLayout(LayoutKind.Sequential, Pack:=x).
Sequential means that .Net is not allowed to move the fields around at runtime, we want them to be arranged
in memory in the order that they appear in our declaration, this will match the C behaviour. Pack allows you to
specify the boundary to align the fields against. Often you would use sequential and pack = 1, then add fields of
padding to get it aligned.

Alternatively, you can use StructLayout(LayoutKind.Explicit) and hard-code where the fields should go. But,
this will only work if the structures have the same layout in x86 and x64. (Unfortunately you must use a
constant for the field offset, so you can't calculate it at runtime).

For TBBUTTON we are stuffed. There is no arrangement of pack and padding so that "one structure fits all", we will
need 2 declarations. And then we'll choose between them at runtime. (I think it might also be possible to just
compile it for x86 and run it under WOW64 on x64...)

<StructLayout(LayoutKind.Sequential, Pack:=1)> _
Public Structure TBBUTTON32
   Public bitmapIndex As Integer
   Public command As Integer
   Public state As TBStates
   Public style As TBStyles
   Public padding As UShort
   Public data As IntPtr
   Public iString As IntPtr
   Public Function Size() As Integer
       Return Marshal.SizeOf(Me)
   End Function
End Structure

<StructLayout(LayoutKind.Sequential, Pack:=1)> _
Public Structure TBBUTTON64
   Public bitmapIndex As Integer
   Public command As Integer
   Public state As TBStates
   Public style As TBStyles
   Public padding1 As Integer
   Public padding2 As UShort
   Public data As IntPtr
   Public iString As IntPtr
   Public Function Size() As Integer
       Return Marshal.SizeOf(Me)
   End Function
End Structure

The function is there for convenience. The 6 bytes of padding in the 64 bit version are spread across two variables.
We could equally use field offsets, but it takes a bit longer as you have to calculate them all. Unless you brain finds
that easy. My brain prefers arranging them in blocks and having padding bytes, and doesn't consider the numbers.

One thing that you think might work is the conditional compile thingy in VB.Net:

#if Platform="x86"

but this depends on the build target, it's not determined at runtime. So again, you would end up with two builds.

Read SD Card CID (serial number)

I just moved blogs, so here's an overview of the SD card CID stuff. I've had some success reading the CID from an SD Card:

Where:

1) The OS is XP or Vista 2) The device is not attached via any USB gizmo. It works only in a card reader attached directly to the PCI bus. (Can scsi pass through get around this? It's not an ATA or SCSI command...) 3) Admin privileges...

What not to do:

1) IOCTL_DISK_GET_STORAGEID 2) GetFileInformationByHandle

As both these return a number assigned when the volume is created. It will change next time the drive is formatted. What I did:

Use IOCTL_SFFDISK_DEVICE_COMMAND to send command 10 (see the SD spec).

Read Secure Digital (SD) Card Serial Number from CID

SD Card Musings

Read CID and CSD C# implementation

Pocket PC?

I don't think it works.

The Future?

IEEE 1667